Information processing device, information processing method, and program

Abstract

To enable control of entry and exit of persons who are not pre-registered for places where it is not possible to lock entry and exit entrances.SOLUTION: A person in a video of the person entering a room is detected by analyzing the video, and facial feature values of the detected person are obtained. The facial features of the detected person are compared with those of a pre-registered person to determine whether the person in the video is a pre-registered or a non-registered person. If a non-registered person is detected, a notification is made and the entry is recorded as an entry record. Then whether the person is a pre-registered or not is determined in the same way in a video for stopping notification, and notification is stopped if a pre-registered person is detected.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, there is a demand for managing people entering and leaving a specific room or area, such as recording people's entry and exit, or allowing only specific people to enter the room. Patent Document 1 discloses a technology for acquiring a preregistered user's face image for authentication and the user's face image, and for recording the user's entry and exit information if these face images match. This technology also prevents the entrance and exit gate from being unlocked if the authentication face image and the user's face image do not match. Patent Document 2 discloses a technology for canceling an alarm in a device that issues an alarm when face authentication is successful. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-215737 A [Patent Document 2] JP 2020-169860 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology disclosed in the above-mentioned Patent Document 1 has a problem in that it is not possible to manage or record entry and exit to a room whose entrance and exit cannot be locked for operational reasons. Also, the technology described in Patent Document 2 is a technology that makes it easy to turn off an alarm, but is not a technology for managing entry and exit to a specific area.

[0005] In view of the above-mentioned problems, the present invention has an object to enable management of entry and exit of persons who are not registered in advance to a place where the entrance and exit gates cannot be locked. [Means for solving the problem]

[0006] The information processing device of the present invention is an information processing device that manages entry and exit by generating records of people entering and exiting a specific location based on video, and is characterized in that it has an acquisition means for acquiring video footage of the entrance to the specific location and video footage of the interior of the specific location, a determination means for detecting people included in the video acquired by the acquisition means and determining whether the detected person is a pre-registered person or a non-registered person, a notification means for instructing a notification device to send a notification when the determination means determines that the video footage of the entrance includes a non-registered person heading to the specific location, and a notification stop means for stopping notification by the notification device when the determination means determines that a registered person is included in the video footage of the interior of the specific location. Effect of the Invention

[0007] According to the present invention, it is possible to manage the entry and exit of persons who are not registered in advance to a place where the entrance and exit gate cannot be locked. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device. [Diagram 2] FIG. 1 is a diagram illustrating an example of a schematic configuration of an image analysis system including an information processing device. [Diagram 3] 1 is a block diagram showing an example of a functional configuration of an information processing device according to a first embodiment. [Figure 4] FIG. 13 is a diagram for explaining an environment in which images of registered and unregistered persons entering a room are captured. [Diagram 5] 13 is a flowchart illustrating an example of a processing procedure for determining whether or not a person in a video is a registered person by using a plurality of detection results. [Figure 6] 13 is a flowchart showing an example of a processing procedure for recording an entry / exit record. [Figure 7] 13 is a flowchart showing an example of a processing procedure for issuing a notification when a person other than a registered person attempts to enter a room. [Figure 8] 13 is a flowchart illustrating an example of a processing procedure for confirming a registrant and stopping a notification. [Figure 9] FIG. 13 is a diagram for explaining a state in which a person in a passage is being photographed. [Figure 10] FIG. 13 is a diagram showing a list of combinations of people who actually passed by and determination results. [Figure 11] 13 is a flowchart showing an example of a processing procedure for correcting an entry record. [Figure 12] 13 is a flowchart showing an example of a processing procedure for correcting an exit record. [Figure 13] 13 is a flowchart showing an example of a processing procedure for recording entry and exit in an emergency. [Figure 14] FIG. 13 is a diagram showing an example of a UI for recording entry and exit in an emergency. [Figure 15] FIG. 13 is a diagram showing an example of a matching result list for each frame. [Figure 16] FIG. 13 is a diagram for explaining a list to which a determination unit refers in the fifth embodiment. [Figure 17] FIG. 13 is a diagram for explaining permission conditions for registrants for each group. [Figure 18] FIG. 13 is a block diagram showing an example of a functional configuration of an information processing device according to a sixth embodiment. [Figure 19] FIG. 13 is a diagram for explaining an environment in which images of registered and unregistered persons entering a room are captured. [Figure 20] FIG. 13 is a diagram showing an example of an entry video including registered and unregistered persons. [Figure 21] FIG. 13 is a block diagram showing an example of a functional configuration of an information processing device according to a seventh embodiment. [Figure 22] 13 is a flowchart showing an example of a processing procedure for not issuing a notification for a certain period of time after the notification has been stopped; [Figure 23] FIG. 23 is a block diagram showing an example of a functional configuration of an information processing device according to an eighth embodiment. [Figure 24] 13 is a flowchart showing an example of a processing procedure for permitting entry of an unregistered person for a certain period of time. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the embodiment described below is one example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the embodiment described below.

[0010] (First embodiment) In this embodiment, an image analysis system is described that analyzes video of entry and exit to determine whether the person is registered or unregistered, records the entry and exit results, issues a notification if a non-registered person is detected in the entry video, and stops the notification if a registered person is detected in the notification stop video. In the following embodiment, a specific room is assumed as the place to be managed, but the system can also be applied to a system that manages entry and exit records of specific places, not limited to rooms, and may generate and manage records of entry and exit to specific places.

[0011] FIG. 1 is a block diagram showing an example of the hardware configuration of an information processing device 100 according to this embodiment. The information processing device 100 according to this embodiment includes a CPU 101, a ROM 102, a RAM 103, a storage 104, and a communication I / F 105. The ROM 102, the RAM 103, the storage 104, and the communication I / F 105 are connected to an internal bus 106.

[0012] The CPU 101 is a central processing unit that performs overall control of operations in the information processing device 100. The ROM 102 is a non-volatile memory that stores programs and various initial setting data required for the CPU 101 to execute processing. The RAM 103 is a volatile memory that functions as the main memory, work area, etc. of the CPU 101, and temporarily stores programs, image data, etc. When executing processing, the CPU 101 loads the necessary programs, etc. from the ROM 102 into the RAM 103 and executes the programs, etc. to realize various functional operations.

[0013] The storage 104 is a recording device having a larger capacity than the RAM 103, such as an HDD (hard disk drive) or SSD (solid state drive). The storage 104 stores an OS (operating system) executed by the CPU 101, an entry / exit record according to this embodiment, a notification determination program, and various other data. The storage 104 can also record image data acquired via a network.

[0014] At startup, such as when the power is turned on, the CPU 101 executes a startup program stored in the ROM 102. This startup program is for reading out the OS stored in the storage 104 and loading it into the RAM 103. After the OS is started, when a user issues an instruction to start the entrance / exit record and notification determination process via, for example, the operation unit 107 shown in FIG. 2, the CPU 101 reads out the entrance / exit record and notification determination process program from the storage 104 and loads it into the RAM 103. This puts the CPU 101 in a state where it can execute the entrance / exit record and notification determination process. The CPU 101 also stores various data used in the operation of the entrance / exit record and notification determination process program in the RAM 103 and reads and writes it. The communication I / F 105 is, for example, an interface for a LAN (Local Area Network), and communicates with a network camera, other computers, etc. via the network.

[0015] In addition, images handled in the entry / exit record and notification determination process in this embodiment, entry / exit record and notification determination process program for executing the entry / exit record and notification determination process, and various data used in the entry / exit record and notification determination process may be acquired via a network. The information processing device 100 can be configured by one device having the configuration shown in FIG. 1, such as a personal computer (PC) or a tablet PC. However, the configuration shown in FIG. 1 may be configured by separate hardware. In other words, the information processing device 100 may be configured by multiple devices. In addition, the calculation process associated with the entry / exit record and notification determination process may be processed using a GPU (not shown).

[0016] FIG. 2 is a diagram showing an example of a schematic configuration of an image analysis system 200 including the information processing device 100 according to this embodiment. The image analysis system 200 includes an information processing device 100, a storage device 109, and a camera 110. The information processing device 100, the storage device 109, and the camera 110 are connected to each other via a network 111 so as to be able to communicate with each other. The network 111 may be, for example, a LAN. Note that the network 111 may have any communication standard, scale, or configuration as long as it is configured to enable communication between the information processing device 100, the camera 110, and the storage device 109. In addition, the physical connection to the network 111 may be wired or wireless. Furthermore, the connection between the information processing device 100, the storage device 109, and the camera 110 is not limited to the network 111, and may be, for example, a USB or the like.

[0017] 1, the information processing device 100 may include an operation unit 107 and a display unit 108 as a hardware configuration. Here, the operation unit 107 includes a pointing device such as a keyboard and a mouse. The display unit 108 is, for example, a monitor such as a liquid crystal display (LCD), and is a display device for a user (operator) to view when operating the information processing device 100.

[0018] The camera 110 is, for example, a network camera, and is an imaging device having a function of capturing an image within an imaging range and transmitting the captured image to the information processing device 100 via the network 111. The camera 110 may be a PTZ camera (Pan Tilt Zoom camera) configured to be able to change the imaging area. In this case, the camera 110 may have a function of transmitting imaging parameters including information indicating an imaging direction and an imaging angle of view, such as a pan angle, a tilt angle, and a zoom magnification, to the information processing device 100 via the network 111.

[0019] In addition, in this embodiment, the case where the camera 110 is a network camera serving as a surveillance camera will be described, but the present invention is not limited to this. For example, the camera 110 may be a digital still camera, a digital video camera, a smartphone or tablet terminal equipped with a camera function, an industrial camera, an in-vehicle camera, a wearable camera, etc. The information processing device 100 receives an image captured by the camera 110 as an input image from the camera 110 via the network 111, and performs processing of the received input image, which will be described later.

[0020] The camera 110 may transmit the captured image to the storage device 109 or a storage device of another device via the network 111. In this case, the information processing device 100 may receive the captured image stored in the storage device 109 or a storage device of another device as an input image via the network 111. Here, the input image is not limited to the captured image captured by the camera 110. For example, the input image may be a partial image that is a part of the captured image. In addition, the configuration of the image analysis system 200 shown in FIG. 2 is an example, and each device may be integrated like a tablet PC. In other words, the information processing device 100 may have the function of the camera 110.

[0021] The information processing device 100 in this embodiment acquires an input image in which a person is shown via the camera 110 or the like. In response to a user's operation, the information processing device 100 selects a person from the acquired input image and sets the person as a registered person. Here, the input image in which a person is shown may be acquired from the storage device 109 via the network 111. In this embodiment, an image is acquired via the camera 110, the acquired image is stored in the ROM 102, and the person is judged as a registered person or a non-registered person according to the entry / exit record and notification judgment processing program expanded in the RAM 103, and the entry / exit record is created and updated. In addition, when a person is judged to be a non-registered person, a notification is issued, and when it is further judged that a registered person has been detected, a process of stopping the notification is performed.

[0022] FIG. 3 is a block diagram showing an example of the functional configuration of the information processing device 100 according to this embodiment. As shown in Fig. 3, the information processing device 100 includes an analysis unit 301, a determination unit 302, a video acquisition unit 303, a notification unit 304, and a recording unit 305. At least some of the functions of each element of the information processing device 100 shown in Fig. 3 can be realized by the CPU 101 executing a program. However, at least some of the elements of the information processing device 100 shown in Fig. 3 may be configured to operate as dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 101. Hereinafter, each function of the information processing device 100 shown in Fig. 3 will be described in detail with reference to Fig. 4.

[0023] FIG. 4 is a diagram for explaining an environment in which the entrance and exit of registered and non-registered persons is photographed in a facility for which entry and exit management is desired in this embodiment. The structure of the facility shown in FIG. 4 is composed of a passage 401 and a room 402 for which entry and exit management is desired. A first camera 403 for acquiring an entry video including a person heading to the room 402 and a second camera 404 for acquiring an exit video including a person leaving the room 402 are installed in the passage 401. That is, the face of a person heading to the room 402 can be detected from the entry video photographed by the first camera 403, but the face of a person leaving the room 402 cannot be detected because the person is photographed facing backwards. Similarly, the face of a person leaving the room 402 can be detected from the exit video photographed by the second camera 404, but the face of a person heading to the room 402 cannot be detected because the person is photographed facing backwards. Also, near the entrance of room 402, there are installed a notification device 405 that notifies of the entry of anyone other than a registered person, and a third camera 406 that acquires a notification stop image capturing the inside of room 402. The notification stop image is an image that is referred to in order to stop the notification from notification device 405.

[0024] Here, the installation positions of the first camera 403 and the second camera 404 may be determined according to the layout of the corridors and rooms of the facility, and the installation positions can be changed within the scope of the gist. The installation positions of the notification device 405 and the third camera 406 can also be changed according to the notification target and the flow line of the facility. In this embodiment, the notification device 405 is a warning light, and the notification device 405 is installed in a position that is easy to see from the corridor when entering the room, as well as easy to see from a staff room (not shown). The notification device 405 may also be a speaker or a notification server, and in the case of a notification server, a device (not shown) may be used to send push notifications to users such as facility managers and registered users.

[0025] In Fig. 4, person 407 is a registered person, and is a facility staff member in this embodiment. Person 408 is an unregistered person, and is a customer in this embodiment. In this embodiment, person 407 is permitted to enter the room alone, and person 408 is prohibited from entering the room alone. However, person 408 can enter the room if accompanied by person 407.

[0026] 3 acquires an entry video from a first camera 403, and an analysis unit 301 detects a person 407 and a person 408 from the entry video. The analysis unit 301 analyzes the detected person, and a determination unit 302 determines whether the detected person is a registered person or not. Then, the determination unit 302 records the determination result in a recording unit 305 corresponding to the storage 104. For example, when a person 407 and a person 408 are detected from the entry video, the person 407 is determined to be a registered person and the person 408 is determined to be an unregistered person. In this case, the notification unit 304 notifies a notification device 405 of the detection of an unregistered person.

[0027] In the example of FIG. 4, person 407 allows person 408 to enter the room by accompanying him, and then person 407 and person 408 are captured in the notification stop image captured by the third camera 406. The image capture unit 303 captures the notification stop image from the camera 406, and a registered person is detected by the analysis of the analysis unit 301 and the judgment of the judgment unit 302. When the registered person is detected, the notification unit 304 stops the notification of the notification device 405. After the work in the room 402 is completed, person 407 and person 408 leave the room through the passage 401. When leaving the room, the image capture unit 303 captures the exit image from the second camera 404, and the analysis unit 301 detects person 407 and person 408 from the exit image. The analysis unit 301 analyzes the detected person, and the judgment unit 302 judges whether the detected person is a registered person or not. Then, the judgment unit 302 records the judgment result in the recording unit 305.

[0028] As described above, the image acquisition unit 303 is connected to the first camera 403, the second camera 404, and the third camera 406 in FIG. 4, and acquires images. Then, the analysis unit 301 acquires images from the image acquisition unit 303 and performs image analysis processing. In the image analysis processing, the analysis unit 301 detects a person from the acquired image and detects the face of the detected person. Then, from the detected face, organs such as the eyes, nose, and mouth are detected to acquire facial feature amounts. The determination unit 302 compares the facial feature amounts of the person in the image obtained by the analysis unit 301 with the facial feature amounts of one or more registered persons registered in advance, and determines whether or not the person corresponds to a registered person. In a list of registered persons whose similarity to the detected person is equal to or greater than a certain value, the similarity is determined by comparing the registered person with the highest similarity to the detected person. If both similarities are less than a certain value, the detected person is determined to be a non-registered person. In the processing described below, it is assumed that a list of registered persons is prepared in advance, and information on the facial feature amounts of the registered persons is stored in the storage 104 or the storage device 109 in advance. The process of determining whether or not a person is a registered person may be performed using multiple detection results of a person detected and identified in multiple frames.

[0029] Here, a method for determining whether or not a person is a registered person using a plurality of detection results in a plurality of frames will be described with reference to the flowchart shown in FIG. 5. FIG. 5 is a flowchart showing an example of a processing procedure for determining whether or not a person in a video is a registered person using a plurality of detection results. The processing in FIG. 5 may be started at the timing when the processing of the image analysis system 200 is started, or the user may decide to start the processing at his / her discretion. The information processing device 100 can realize each processing shown in FIG. 5 by the CPU 101 reading and executing a necessary program. An example of determining whether or not a person is a registered person from an entry video will be described below, but the same procedure applies to an exit video and a notification stop video.

[0030] First, in the process of S501, the video acquisition unit 303 acquires an entry video from the first camera 403. In the process of S502, the analysis unit 301 detects a person from the entry video and acquires the facial feature amount of the detected person. Note that, if multiple people are detected from the video, the following process is performed for all of the detected people.

[0031] In the processing of S503, the determination unit 302 compares the facial features of the detected person with the facial features of one or more registered persons who have been registered in advance, and obtains information on the highest similarity value when compared with the registered persons. In the process of S504, the determination unit 302 generates a matching result list for the same person. The matching result list is a list of detection results for the same person for each frame. The matching result list may be recorded in the recording unit 305, or may be held as internal data of the analysis unit 301 or the determination unit 302. Whether or not the person is the same between multiple frames is determined from the detection coordinates on the images between the frames. Alternatively, a person with similar features, such as the color or shape of clothes or hairstyle, may be determined to be the same person, and the means for this is not limited.

[0032] In the process of S505, the determination unit 302 determines whether a predetermined time has passed since the person was first detected. If the predetermined time has not passed (S505 / No), the process is repeated from S501. If the predetermined time has passed (S505 / Yes), the process proceeds to S506. In the process of S506, the determination unit 302 refers to the matching result list and determines whether the detected person is a registrant. In this process, the matching result list with the highest similarity is referenced, and if the similarity is equal to or greater than a certain value, it is determined that the person is the same as the registrant. In this case, if the similarity value in the matching result list does not exceed the certain value, it is determined that the person is not a registrant, that is, that the detected person is a non-registrant.

[0033] If the determination unit 302 determines that there is an unregistered person, the notification unit 304 notifies the notification device 405 shown in Fig. 4 that an unregistered person has been detected. The determination result obtained by the determination unit 302 is recorded in the recording unit 305 as an entry record. Note that in the case of a leaving video, it is recorded in the recording unit 305 as an leaving record.

[0034] Next, the details of the process flow of each element shown in FIG. 3 will be described with reference to the flowcharts of FIG. 6 to FIG. Fig. 6 is a flow chart showing an example of a processing procedure for recording an entry / exit record in this embodiment. The processing in Fig. 6 may be started when the processing of the image analysis system 200 is started, or the user may decide to start the processing at his / her discretion. The information processing device 100 can realize each processing shown in Fig. 6 by the CPU 101 reading and executing a necessary program.

[0035] First, in the process of S601, the video acquisition unit 303 acquires a video from the first camera 403 or the second camera 404. Next, in the process of S602, the analysis unit 301 performs image analysis processing, and the determination unit 302 determines whether the person is a registered person or a non-registered person from the video. In this process, the analysis unit 301 acquires facial features from a person in the video, and the determination unit 302 compares the facial features of the person in the video acquired by the analysis unit 301 with the facial features of one or more registered people registered in advance to determine whether the person is a registered person or a non-registered person. In this process, if multiple people are detected from the video, the facial features are compared for all the detected people. In a scene where a person leaves a room, although the person is shown in the entry video, the person is photographed facing backwards, so that the facial features cannot be acquired. Therefore, in such a case, the process is forcibly terminated. On the other hand, in a scene where a person leaves a room, the person's face can be detected in the exit video, so the process proceeds to the following S603. In a scene where a person heads toward a room, the same applies.

[0036] In the process of S603, the determination unit 302 determines whether the video acquired in S601 is an entry video. If it is an entry video (S603 / Yes), in the process of S604, the determination unit 302 records it in the recording unit 305 as an entry record. The information recorded at this time includes the detection time as the entry time, and the names of registered people or information indicating that non-registered people are included as people entering the room. In addition, images of the detected registered people and non-registered people, snapshots at the time of entry, and video including the time before and after entry may also be recorded. After the process of S604 is completed, the process of creating the entry and exit record is terminated.

[0037] On the other hand, if the result of the determination in S603 is that the video is not an entry video (S603 / No), the determination unit 302 determines that the video is an exit video, and in the process of S605, the determination unit 302 records this as an exit record in the recording unit 305. Note that the information recorded in the process of S605 is the same as the content recorded in the process of S604. After the process of S605 is completed, the process of creating the entry and exit record is terminated.

[0038] In the flowchart shown in FIG. 6, the case where the image from the first camera 403 or the second camera 404 is input has been described, but the same processing may be performed when the notification stop image is acquired from the third camera 406. In addition, in the processing of FIG. 6, the entry and exit record of the information of the registered person and the non-registered person is created, but only the information of the registered person may be created as the entry and exit record. In this case, after the processing of S602, the determination unit 302 determines whether or not the image includes a registered person, and if the image does not include a registered person, the processing ends as it is, and if the image includes a registered person, the processing proceeds to S603. Then, in S604 and S605, only the information of the registered person is recorded as the entry and exit record. In addition, in the processing procedure of FIG. 6, the information may be recorded in the recording unit 305 or in a separate recording device so that the determination results can be distinguished between registered people and non-registered people. Also, only one of the registered people and non-registered people may be recorded.

[0039] Next, a procedure for performing the notification process will be described. Fig. 7 is a flow chart showing an example of a procedure for performing a notification when a person other than a registered person attempts to enter a room in this embodiment. The process in Fig. 7 may be started at the same time as the process of the image analysis system 200 is started, or the user may decide to start the process at his / her own discretion. The information processing device 100 can realize each process shown in Fig. 7 by the CPU 101 reading and executing a necessary program. In S701, the video acquisition unit 303 acquires a video from the first camera 403, the second camera 404, or the third camera 406. The process of S702 is similar to S602 in Fig. 6, and therefore a description thereof will be omitted.

[0040] In the process of S703, the determination unit 302 determines whether or not there is an unregistered person in the image by the same procedure as described above. If the result of the determination in S703 is that there is at least one unregistered person (S703 / Yes), the process proceeds to S704. On the other hand, if there is no unregistered person in the image (S703 / No), the process ends. In the process of S704, the determination unit 302 determines whether the video in which the unregistered person was detected was an entry video. If the result of this determination is that it was an entry video (S704 / Yes), the process proceeds to S705. On the other hand, if it was not an entry video (S704 / No), the process ends.

[0041] In the process of S705, the notification unit 304 executes notification processing using the notification device 405. When the notification processing is performed, the determination unit 302 detects an unregistered person and records the fact that the notification has been performed as an entry record in the recording unit 305. The contents recorded at this time may be the date and time of detection, a cut-out image of the face part, a snapshot, etc.

[0042] Next, a process procedure for canceling a notification will be described. Fig. 8 is a flow chart showing an example of a process procedure for confirming a registrant and stopping a notification in this embodiment. The process of Fig. 7 may be started at the same time as the process of the image analysis system 200 is started, or the user may decide to start the process at his / her own discretion. The information processing device 100 can realize each process shown in Fig. 8 by the CPU 101 reading and executing a necessary program. In S801, the video acquisition unit 303 acquires a video from the first camera 403, the second camera 404, or the third camera 406. The process of S802 is similar to S602 in Fig. 6, and therefore a description thereof will be omitted.

[0043] In the process of S803, the determination unit 302 determines whether or not the person in the video has a registered person by the same procedure as described above. If the result of the determination in S803 is that there is at least one registered person (S803 / Yes), the process proceeds to S804. On the other hand, if there is no registered person in the video (S803 / No), the process ends. In the process of S804, the determination unit 302 determines whether the image in which the registrant is detected is a notification stop image. If the result of this determination is that the image is a notification stop image (S804 / Yes), the process proceeds to S805. On the other hand, if the image is not a notification stop image (S804 / No), the process ends.

[0044] In the process of S805, the notification unit 304 stops the notification process using the notification device 405. When the notification process is stopped, the determination unit 302 may record the fact that the notification has been stopped as an entry record in the recording unit 305. The contents recorded at this time may be the name of the registrant detected in this process, the detection time, a cut-out image snapshot of the face part at the time of detection, etc.

[0045] The order of the processes shown in Figures 6 to 8 may be changed as long as the gist of the process can be realized. Also, although the processes shown in Figures 6 to 8 have been described as separate flows, they may be realized in the same flow. In this case, the processes of acquiring video in S601, S701, and S801, the processes of determining whether a person is registered or not from the video in S602, S702, and S802, and the processes of determining the source of the video in S603, S704, and S804 may be common. Also, when they are realized in the same flow, the order of the processes may be changed as long as the gist of the process can be realized, or the processes may be executed in parallel.

[0046] Also, as a means for stopping a notification, a means other than stopping a notification when a registered person is shown in a notification stopping image as shown in Fig. 8 may be provided. For example, the notification device 405 may be provided with a notification stopping button, or a separate application for stopping a notification may be provided. Also, when a notification is stopped by using these buttons or applications, a record different from when the notification is stopped according to the procedure shown in Fig. 8 may be left. Specifically, when a notification is stopped by using the notification stopping button, a message such as "Forced stop by notification stopping button" and the time of the stop may be left in the record.

[0047] As described above, according to this embodiment, when an unregistered person is detected in the entry video, a notification is given, and when a registered person is detected in the notification stop image, the notification is stopped. As a result, when an unregistered person enters the room alone, a warning light or other notification can be used to make the unregistered person aware that entry is prohibited, or a registered person such as a facility staff member can stop the unregistered person. In addition, when an unregistered person enters the room accompanied by a registered person, the notification is stopped, so that it is possible to prevent the unregistered person from being mistakenly stopped even if they are in the room.

[0048] Second embodiment In the configuration shown in the first embodiment, depending on the facial feature amount acquired by the analysis unit 301 and the result of the judgment by the judgment unit 302, there is a possibility that a registered person may be erroneously judged as a non-registered person. Therefore, in this embodiment, a method of correcting an erroneous entry record using registered person information acquired in the process of stopping notification, and correcting an erroneous exit record using the entry record will be described. Note that the basic configuration of the image analysis system, the hardware configuration and the functional configuration of the information processing device according to this embodiment are the same as those in Figs. 1 to 3, respectively, and therefore the description will be omitted. Below, the points different from the first embodiment will be described.

[0049] Hereinafter, a process for correcting an erroneous entry record or exit record when a registered person is erroneously determined to be a non-registered person will be described with reference to Figs. 9 is a diagram for explaining how a person 407 in a passage 401 is photographed. In FIG. 9, a first camera 403 capturing an entry image, the person 407, and an entry record 900 are shown.

[0050] In the example shown in FIG. 9, a person 407 is shown in the entry video acquired by the video acquisition unit 303. As described in the first embodiment, in S601, the video acquisition unit 303 acquires the entry video, and in S602, the analysis unit 301 detects a person from the entry video acquired. Then, the facial feature amount of the detected person is acquired. Then, the determination unit 302 compares the facial feature amount of a registered person registered in advance, and assumes that there is no registered person registered in advance whose similarity exceeds a certain value. In this case, the determination unit 302 determines that the person 407 is an unregistered person, and in S604, information that an unregistered person has entered the room is recorded in the entry record 900. If the person 407 is actually a registered person, it means that the person has been erroneously determined to be an unregistered person. Similarly, when the person leaves the room, if the person is actually the same person as the registered person but is erroneously determined to be an unregistered person, information is recorded in the exit record as an unregistered person. In such a case, a method for correcting the entry and exit record will be described with reference to FIGS. 10 to 12.

[0051] 10 is a diagram showing a list of combinations of people who actually passed and the judgment results. Table 1001 shows a list of combinations of people who actually passed, the judgment results in the entry record, and the notification cancellation judgment. Table 1002 shows a list of people who actually passed, the judgment results in the exit record, and the entry results in the entry record.

[0052] First, a method for correcting an erroneous entry record will be described using table 1001 shown in Fig. 11 and Fig. 10. Fig. 11 is a flowchart showing an example of a processing procedure for correcting an entry record in this embodiment. The processing in Fig. 11 may be started at the same time as the processing of image analysis system 200 is started, or the user may decide to start the processing at his / her own discretion. Information processing device 100 can realize each processing shown in Fig. 11 by CPU 101 reading and executing a necessary program.

[0053] First, in the process of S1101, the determination unit 302 determines whether or not the video acquired by the video acquisition unit 303 is an entry video. If it is an entry video (S1101 / Yes), the process proceeds to S1102. On the other hand, if it is not an entry video, the process ends as it is. In the process of S1102, the determination unit 302 obtains information on a group of people who entered the room within a certain time period after the first person entered the room from the entry record. Here, if no person was detected within a certain time period in a chronologically previous time period for the detection time of any detected person, the person is determined to be the first person to enter the room. Zero or more people who entered the room within a certain time period after the first person to enter the room are considered to be a group of people who entered the room within the certain time period.

[0054] In the process of S1103, the judgment unit 302 judges whether all the people belonging to the group are unregistered. If all the people are unregistered (S1103 / Yes), the process proceeds to S1104. On the other hand, if at least one person is a registered person (S1103 / No), there is no need to correct the entry record, so this process ends.

[0055] In the process of S1104, the judgment unit 302 refers to the entry record recorded in the recording unit 305 and judges whether or not the notification stop process was performed within a certain time after the notification was made. If the notification was stopped within the certain time after the notification was made (S1104 / Yes), the entry record is corrected in the process of S1105. On the other hand, if the notification was not stopped within the certain time after the notification was made (S1104 / No), it is determined that only unregistered people actually entered the room, so the entry record is not corrected and this process ends.

[0056] Here, the details of the correction method in S1105 will be described with reference to Table 1001. In Table 1001, when the person who actually passed is a registered person and the judgment result in the entry video is a non-registered person, the person is judged to be a registered person in the notification stop video, or the notification stop button is operated by the registered person, and the notification is stopped. Therefore, the judgment unit 302 obtains information about the registered person who performed the stop, and corrects the entry record immediately before. When the person who actually passed is a non-registered person and the judgment result in the entry video is a non-registered person, the non-registered person cannot stop the notification. Therefore, since there is no record of the stop, the entry record remains the same.

[0057] Also, if the people who actually passed through were a combination of registered and non-registered people, and both were judged to be non-registered in the entry video, they will be judged to be registered in the notification stop video, or the notification stop button will be operated by the registered person, so the notification will be stopped. In this case, too, the judgment unit 302 acquires information about the registered person who stopped, and corrects the previous entry record. At this time, the previous entry record is recorded for two people. Basically, since the registered person generally enters the room first, the information of the person whose detection time is older among the recorded information is corrected. Alternatively, the facial images of the two people acquired in the entry video and the facial image acquired from the notification stop video when the notification is stopped may be compared, and the information of the one with the higher similarity in facial features may be changed. In terms of facility operation, if a registered person follows a non-registered person, the information of the person whose detection time is more recent may be corrected.

[0058] Also, in the process of S1101, it is determined whether or not the video is an entry video. However, the processing program may target only the entry video, in which case the program processing starts from the process of S1102.

[0059] Next, a method for correcting an erroneous exit record will be described with reference to FIG. 12 and table 1002 in FIG. 10. FIG. 12 is a flowchart showing an example of a processing procedure for correcting an exit record in this embodiment. The processing in FIG. 12 may be started at the same time as the processing of image analysis system 200 is started, or the user may decide to start the processing at his / her own discretion. Information processing device 100 can realize each processing shown in FIG. 12 by CPU 101 reading and executing a necessary program.

[0060] In the process of S1201, the determination unit 302 determines whether the video acquired by the video acquisition unit 303 is a video of a person leaving the room. If the video is not a video of a person leaving the room (S1201 / No), this process ends. If the video is a video of a person leaving the room (S1201 / Yes), the process proceeds to S1202. In the process of S1202, the determination unit 302 determines whether the number of people who entered recorded in the entry record matches the number of people who left recorded in the exit record. If they do not match (S1202 / No), this process is interrupted and the process waits until a new person is detected in the exit video. On the other hand, if the numbers match (S1202 / Yes), the process proceeds to S1203.

[0061] In the process of S1203, the determination unit 302 determines whether all the people who have left the room are unregistered. If the result of this determination is that all are unregistered (S1203 / Yes), the process proceeds to S1204, where the determination unit 302 obtains the entry record from the recording unit 305 and corrects the exit record as necessary. On the other hand, if at least one person is a registered person (S1203 / No), there is no need to correct the exit record, and this process ends.

[0062] Here, the details of the correction method in S1204 will be described with reference to Table 1002. In Table 1002, if the determination result in the exit video is a non-registered person, the determination unit 302 first refers to the entry record. If the person who actually passed is a registered person, the latest entry record records the information of the registered person, so it is determined that the person who left is a registered person, and the exit record is corrected to the information of the registered person at the time of entry. On the other hand, if the person who actually passed is a non-registered person, the latest entry record also records the person as a non-registered person. Therefore, in such a case, it is not necessary to correct the exit record. In this case, a notification is made from the notification device 405, and the notification is not stopped.

[0063] A case where the people who actually passed through were a combination of a registered person and a non-registered person will be described. If both of the people in the exit video are determined to be non-registered, the determination unit 302 checks the information of the past two people in the latest entry record. Then, the information of the registered person in the entry record is obtained, and the exit record is corrected using that information. At this time, the exit record with the oldest time in chronological order is corrected. Alternatively, the facial images of the two people obtained in the exit video may be compared with the registered facial image of the registered person, and the information of the one with the higher similarity in facial feature amount may be changed. As with the entry, if a registered person follows a non-registered person due to the operation of the facility, the information of the person with the newer detection time may be corrected. Alternatively, the image of the registered person obtained in the entry video may be used as the image to be compared with the facial images of the two people obtained in the exit video.

[0064] In this embodiment, an example in which a maximum of two people enter a room at the same time has been described, but a case in which one registered person and two or more unregistered people enter and leave a room can also be realized by performing the same processing as in this embodiment. In terms of facility operation, the main use cases are as described above.

[0065] As described above, according to this embodiment, the entry record or exit record is corrected, so that more accurate information can be recorded.

[0066] (Third embodiment) In the first embodiment described above, an example was described in which entry and exit records are recorded and entry and exit management is possible even for places where the entrance and exit cannot be locked due to operational reasons. However, in an emergency such as a fire in a room or the occurrence of a rescuer, only a number of unregistered people may enter the room. It is cumbersome in an emergency to perform a process of stopping notifications such as an alarm release operation each time. In addition, in an emergency, it is more appropriate to record the entry and exit records in a different format. Hereinafter, with reference to Figs. 13 and 14, a process of recording entry and exit in an emergency such as a fire in a room or the occurrence of a rescuer as a special event will be described in comparison with the example described in the first or second embodiment. Note that the basic configuration of the image analysis system, the hardware configuration and the functional configuration of the information processing device according to this embodiment are the same as those in Figs. 1 to 3, respectively, and therefore will not be described. Hereinafter, differences from the first embodiment will be described.

[0067] FIG. 13 is a flowchart showing an example of a process procedure for recording entry and exit in an emergency in this embodiment. The process of FIG. 13 may be started at the timing when an emergency such as a fire in a room or the occurrence of a rescuer is detected, or the user may decide to start the process at his / her discretion. The information processing device 100 can realize each process shown in FIG. 13 by the CPU 101 reading and executing a necessary program. FIG. 14 is a diagram showing an example of a UI for recording entry and exit in an emergency displayed on the display unit 108. In the record list 1400, a record 1401 of entry or exit and a tag 1402 indicating that the entry and exit is an emergency entry and exit are described. The record list 1400 in FIG. 14 is created by the determination unit 302 based on the entry record and exit record recorded in the recording unit 305, and is displayed on the display unit 108.

[0068] In the process of S1301 in FIG. 13, the determination unit 302 refers to the entry records and exit records recorded in the recording unit 305 and determines whether the total number of entries and exits within a predetermined time is equal to or greater than a certain value. For example, in a facility where entry and exit are conducted in pairs, the certain value may be set to 4. Alternatively, a value may be set for the total number of entries at the time of entry. In a facility where entry and exit are conducted in pairs, the certain value may be set to 3. If the total number of entries and exits within a predetermined time is equal to or greater than the certain value (S1301 / Yes), the process proceeds to S1302. On the other hand, if the total number of entries and exits within a predetermined time is less than the certain value (S1301 / No), the process proceeds to S1305.

[0069] In the process of S1302, the determination unit 302 determines whether or not an unregistered person is detected in the entry video when the total number of entries and exits within a predetermined time period is equal to or greater than a certain value. If an unregistered person is detected in the entry video (S1302 / Yes), the process proceeds to S1303. On the other hand, if an unregistered person is not detected (S1302 / No), the process proceeds to S1305.

[0070] In the process of S1303, the determination unit 302 displays the entry / exit record in an emergency on the display unit 108 so that an identification tag is added to the entry / exit record of the corresponding non-registered person. At this time, as shown in the tag 1402 of FIG. 14, a specific symbol, an icon, a message, or a color change may be used, and any method of realization is acceptable as long as it is in a form that can identify that it is a special event. In addition, the identification tag may be added from the time when the total number of entries and exits becomes a certain value or more in the process of S1301, or tagging may be performed by going back through the entry / exit history. In the example of the record list 1400 of FIG. 14, since the total number of entries and exits within a certain time (for example, 5 minutes) becomes a certain value or more (5 people in this case) at the time of entry at 11:08:30, tagging is performed by going back through 5 records. Here, only non-registered persons are given identification tags, but identification tags may also be given to registered persons, and even if only registered persons are detected in the processing of S1302 (S1302 / No), identification tags may also be given to registered persons.

[0071] In the process of S1304, the notification unit 304 restricts the notification in this process. For example, when a notification is made from the notification device 405 using an audio speaker and a warning light, the warning light is turned on but no voice is notified. Or, when a push notification is made to a smartphone or the like, if a notification has already been made once, processing is performed such that the notification is not made again. Even if an unregistered person is detected, if the notification stop processing is performed with the unregistered person accompanied as described above, the notification is kept stopped. However, if a new notification is made after that by only an unregistered person entering the room, the notification is restricted. In this embodiment, since there are many cases where only unregistered people enter the room, as in the example shown in FIG. 14, by restricting the notification in such a case, it is possible to eliminate complexity in an emergency.

[0072] In the process of S1305, the determination unit 302 determines whether or not a certain time has passed since the latest exit record. If the result of this determination is that the certain time has passed (S1305 / Yes), it is determined that emergency entry and exit has been completed, and this process ends. On the other hand, if the certain time has not passed (S1305 / No), the process returns to the process of S1302, and the subsequent processes are repeated. In addition, in the process of S1305, the recording unit 305 may record video footage from a camera capturing entry and exit from the latest exit until the certain time has passed since the latest exit, from a certain time before the time when it is determined in S1301 that the total number of entries and exits within a predetermined time is equal to or greater than a certain value.

[0073] As described above, according to this embodiment, notifications are limited in an emergency, so that in cases where a large number of unregistered people, such as emergency personnel, enter a room, the annoyance caused by notifications can be eliminated.

[0074] (Fourth embodiment) In this embodiment, with respect to the first or second embodiment described above, a method of limiting notification according to a combination detected within a certain time period when a non-registered person is detected will be described. In this embodiment, even if a non-registered person is detected in the entry video, notification is not performed if it is determined that the non-registered person entered the room together with a registered person. Hereinafter, a method of determining whether or not a non-registered person entered the room together with a registered person will be described with reference to FIG. 15. Note that the basic configuration of the image analysis system, the hardware configuration, and the functional configuration of the information processing device according to this embodiment are the same as those in FIGS. 1 to 3, respectively, and therefore description thereof will be omitted. Below, differences from the first embodiment will be described.

[0075] In this embodiment, the same person is identified across multiple frames using the procedure shown in Fig. 5. Fig. 15 is a diagram showing an example of a matching result list 1500 for each frame when both a registered person and a non-registered person enter a room. The frame No. in the title row of the matching result list 1500 is an index of the transition of matching results for each time series. The smaller the number, the older the data in the time series, and the larger the number, the newer the data in the time series. Id is shown as 1 to 4, and each index indicates the same person. The column with Id=1 indicates the transition of matching results for a specific same person.

[0076] For example, the matching result list for Id=1, i.e., matching result list 1501 for registered person A, is compared with the matching result list for Id=2, i.e., matching result list 1502 for non-registered people. At this time, registered person A with Id=1 and non-registered person with Id=2 are detected simultaneously in the same frames from frame No. 4 to No. 11. In this way, if they continue to be detected simultaneously over a certain number of frames or more, it is determined that the non-registered person has entered the room together with the registered person. The number of frames in which they continue to be detected simultaneously may be specified as any value greater than or equal to 1, and may be adjusted according to the installation position of the camera, etc.

[0077] On the other hand, the matching result list for Id=3, i.e., the matching result list 1503 for registered person B and the matching result list 1504 for the non-registered person at Id=4, are located close to each other in time series, but there are no frames detected at the same time. In this case, it is determined that the non-registered person at Id=4 did not enter the room with the registered person at Id=3. In this embodiment, the condition for simultaneous entry is that they are detected simultaneously over multiple frames, but it may be determined that they entered the room at the same time when the frame difference detected between the registered person and the non-registered person is within a certain range. For example, there is a two-frame difference between the matching result list 1503 and the matching result list 1504, but the condition for determining that they entered the room at the same time may be a difference of five frames or less. In this case, it is determined that the non-registered person at Id=4 entered the room at the same time as the registered person at Id=3, B.

[0078] In this embodiment, when it is determined by the means described in FIG. 15 that a registered person and a non-registered person have entered the room at the same time, the notification process in the notification unit 304 is not executed. Specifically, before performing the process of S705 in FIG. 7, the determination unit 302 refers to the matching result list of the person detected from the entry video and determines whether or not the above-mentioned condition is satisfied. If the above-mentioned condition is satisfied, the process is terminated as it is, and if the above-mentioned condition is not satisfied, the process proceeds to S705 and the notification process is executed. In this way, the notification process in the notification unit 304 is executed only when a registered person and a non-registered person have not entered the room at the same time. Also, in this embodiment, a method for determining simultaneous entry of two people has been described, but simultaneous entry of three or more people can also be determined by a similar method.

[0079] As described above, according to this embodiment, when a registered person and an unregistered person enter a room together, no notification is sent, and therefore processing such as stopping the notification is not required.

[0080] Fifth embodiment In this embodiment, an example will be described in which registrants are divided into groups in the first embodiment, and notifications are restricted when a registrant and a non-registrant belonging to a specific group, or two registrants, enter a room at the same time. Details will be described below with reference to Fig. 16. Note that the basic configuration of the image analysis system, and the hardware configuration and functional configuration of the information processing device according to this embodiment are the same as those in Figs. 1 to 3, respectively, and therefore will not be described here. Differences from the first embodiment will be described below.

[0081] Fig. 16 is a diagram for explaining a list referred to by the determination unit 302 in Fig. 3. The determination unit 302 performs a process of determining whether a person is a registrant or a non-registrant by referring to a list of registrants divided into groups. In the example shown in Fig. 16, the determination unit 302 refers to three groups, registrant group A1601, registrant group B1602, and registrant group C1603. The lists of registrant group A1601, registrant group B1602, and registrant group C1603 may be configured to be held by the determination unit 302, or may be acquired from the storage device 109 via the network 111, and the form thereof is not important.

[0082] Next, with reference to FIG. 17, a notification operation when a registrant and a non-registrant who belong to a specific group, or two registrants, enter a room at the same time will be described. In example 1700 shown in FIG. 17, a registrant belonging to group A is permitted to enter and leave the room alone. Also, a registrant belonging to group A can enter the room together with a non-registrant and a registrant belonging to group B. And, a registrant belonging to group A is recognized as a registrant who satisfies the condition for stopping notification in the procedure shown in FIG. 8. On the other hand, a registrant belonging to group B is permitted to enter the room only if he / she enters the room at the same time as a registrant of group A, and is not recognized as a registrant who satisfies the condition for stopping notification in the procedure shown in FIG. 8. Furthermore, a registrant belonging to group C can enter the room alone, but cannot enter with a third party, including a non-registrant, and is not recognized as a registrant who satisfies the condition for stopping notification in the procedure shown in FIG. 8.

[0083] When conditions for entry as shown in FIG. 17 are indicated for registered user group A1601, registered user group B1602, and registered user group C1603, notification is not performed for combinations that are permitted, and notification is performed for combinations that are not permitted. For example, when a registered user belonging to group A and a non-registered user attempt to enter the room at the same time, notification is not performed. On the other hand, when a registered user belonging to group C and a registered user belonging to group B attempt to enter the room at the same time, notification is performed even if there is no non-registered user. Specifically, instead of determining whether or not there is a non-registered user in the process of S703 in FIG. 7, the determination unit 302 determines whether or not the permission conditions as shown in FIG. 17 are satisfied, for example. If the permission conditions are satisfied, the process is terminated, and if the permission conditions are not satisfied, the process proceeds to S704.

[0084] In this case, the notification contents may be changed according to the combination of groups. For example, when a registered person belonging to group C and a registered person belonging to group B try to enter the room, a warning light may be turned on, but when a registered person belonging to group C and a non-registered person try to enter the room, a sound may be played over a speaker. Such a setting is useful, for example, for managing the entry of guide staff into a room that is prohibited for use by customers alone, by registering group A as guide staff and group B as customers. It is also possible to operate the system by registering machine maintenance workers in group C and permitting their entry into the room alone for machine maintenance. In addition, registered people who are recognized as meeting the conditions for stopping notifications in the procedure shown in FIG. 8 but are not permitted to enter the room alone or with a companion may be grouped separately. Furthermore, it is also possible to allow the time period and period to be specified at the same time when grouping.

[0085] As described above, according to this embodiment, it is possible to increase the variety of entry permission conditions, so that notification can be given or stopped in more certain cases.

[0086] Sixth embodiment In the first embodiment, a notification stop image is obtained separately from the entry image and the exit image, and notification is stopped when a registered person is detected in the notification stop image. In contrast, in the present embodiment, an image analysis system is described that judges whether a person is registered or unregistered from the entry image and the exit image, records the entry and exit results, and stops notification when a registered person is detected in the entry image after a non-registered person is detected in the entry image. Details of the processing of this embodiment are described below with reference to Figs. 18 to 20. Note that the hardware configuration of the information processing device and the basic configuration of the image analysis system according to this embodiment are the same as those in Figs. 1 and 2, respectively, and therefore description thereof is omitted. Differences from the first embodiment are described below.

[0087] Fig. 18 is a block diagram showing an example of the functional configuration of an information processing device 1800 according to this embodiment. The determination unit 302 in Fig. 18 has the same function as in Fig. 3, but includes a notification stop determination unit 1801 for performing the processing of this embodiment. The details of the function of the notification stop determination unit 1801 will be described later.

[0088] FIG. 19 is a diagram for explaining an environment in which the entrance and exit of registered and unregistered persons is photographed in a facility where entry and exit management is desired in this embodiment. In the example of FIG. 19, compared to the configuration of FIG. 4, a third camera for acquiring a notification stop image is not installed. In the example of FIG. 19, there is a passage 1900 and a room 1901, and the passage 1900 is photographed by an entrance camera 1902 and an exit camera 1903. In addition, a notification device 1904 that issues a notification when a person is detected is installed near the entrance of the room 1901. Also, it is assumed that person A1905 and person B1906 are passing through the passage 1900.

[0089] Fig. 20 is a diagram showing an example of an image including person A 1905 and person B 1906 passing through a passage 1900, captured by an entrance camera 1902. Here, person A 2011, person A 2021, and person A 2031 in Fig. 20 are the same person, and are registered persons. On the other hand, person B 2012, person B 2022, and person B 2032 are the same person, and are unregistered persons. Also, an image 2020 is an image of a frame chronologically later than the image 2010.

[0090] At the time of the image 2010, person A2011 is actually a registered person, but because he was walking while looking at a document in front of him, facial feature values ​​were acquired at a sideways angle. Therefore, person A2011 is determined to be an unregistered person at the time of the image 2010, and a notification is issued from the notification device 1904 by the notification unit 304, since only unregistered people have been detected in the entry image.

[0091] After that, at the time of the image 2020, the person A2021 faces the passageway travel direction, so that more accurate facial feature values ​​are acquired. As a result, the person A2021 is determined as a registered person. The notification stop determination unit 1801 issues a notification stop instruction to the notification unit 304, assuming that the person A2011 and the person A2021 are the same person and the determination result has changed from a non-registered person to a registered person. Specifically, the notification is stopped according to the procedure shown in FIG. 8, but in the process of S804, the determination is made using the entry video instead of the notification stop video. This process can be executed simultaneously even in a state where a third camera that acquires the notification stop video shown in FIG. 4 is present. At this time, the determination result of the person A2021 may be recorded as a notification stop record.

[0092] Also, as shown in the video 2030, when a person A2031, who is a registered person, takes a specific action, the notification stop determination unit 1801 may instruct the notification unit 304 to perform a notification stop process. Here, the video 2030 is a frame that follows the video 2010 in time series. At the time of the video 2010, the person A2011 is actually a registered person, but is detected as an unregistered person. At this time, the notification unit 304 issues a notification from the notification device 1904 by processing of the determination unit 302. At the time of the video 2030, the person A2031 faces so as to face the entrance camera 1902. The analysis unit 301 detects a person from the entrance video, and further performs face organ detection from the person's face. The determination unit 302 determines the line of sight from the arrangement of the eyes by face organ detection, and when the line of sight faces the entrance camera 1902, the notification stop determination unit 1801 stops the notification.

[0093] As described above, according to this embodiment, a camera for capturing notification stop video is not used, and notification is stopped when a registered person is confirmed from an entry video, or when the registered person is confirmed and faces the camera. This makes it possible to eliminate the need for a camera for capturing notification stop video, and allows for management with a simpler system.

[0094] Seventh embodiment In the above-mentioned embodiment, even if a registered person is detected and the notification is stopped, if a non-registered person is detected by continuously appearing in the entry video of the entry camera, the notification is made again. Therefore, in this embodiment, an example will be described in which, in contrast to the first to sixth embodiments described above, when a notification is stopped once, the notification is not made again for a certain period of time. This process is particularly useful in a system that notifies when a non-registered person is detected with the configuration described in the sixth embodiment. For example, after the notification is stopped in the procedure of the sixth embodiment, a non-registered person may be detected in the entry video of the entry camera and the notification is made again, but in this embodiment, such a re-notification can be avoided. Note that the method for preventing re-notification according to this embodiment is not limited to application in the sixth embodiment, and may be combined with the first to fifth embodiments. Note that the hardware configuration of the information processing device according to this embodiment and the basic configuration of the image analysis system are the same as those in FIG. 1 and FIG. 2, respectively, and therefore will not be described. Below, differences from the sixth embodiment will be described.

[0095] First, the flow of processing in this embodiment will be described with reference to Fig. 21 and Fig. 22. Fig. 21 is a block diagram showing an example of the functional configuration of an information processing device 2100 according to this embodiment. The determination unit 302 in Fig. 21 has the same function as that in Fig. 3, but includes a notification control unit 2101 for performing the processing of this embodiment. For ease of explanation, the description will be given using the configuration shown in Fig. 21, but the notification control unit 2101 may be added to the configuration of the determination unit 302 shown in Fig. 18.

[0096] Fig. 22 is a flowchart showing an example of a processing procedure for not issuing a notification for a certain period of time after the notification has been stopped. The processing in Fig. 22 is started in a state in which notification has been issued in the procedure of the above-mentioned embodiment. The information processing device 2100 can realize each processing shown in Fig. 22 by the CPU 101 reading and executing a necessary program.

[0097] In the process of S2201, the determination unit 302 determines whether or not the notification stop process has been performed. When the re-notification of this embodiment is prevented in the first to fifth embodiments, the determination unit 302 detects the registrant in the notification stop image and determines whether or not the notification stop process has been performed. The same applies when the registrant operates the notification device to stop the notification. If the notification stop process has not been performed (S2201 / No), the process waits until the notification stop process is performed. If all the people who entered the room leave the room during this processing loop, the repeated process of S2201 is stopped. On the other hand, if the notification stop process has been performed (S2201 / Yes), the process proceeds to S2202.

[0098] In the process of S2202, the notification control unit 2101 records the notification stop time and holds the information. In the process of S2203, the analysis unit 301 and the determination unit 302 perform the process of recording entry and exit, and the process of monitoring the entry of unauthorized persons. The process of S2203 is the process described in the previous embodiment, in which the entry video and the exit video are used to determine whether the person is registered or unregistered, the entry and exit record is recorded in the recording unit 305, and unregistered persons are detected in the entry video.

[0099] In the process of S2204, the determination unit 302 determines whether or not an unregistered person has been detected in the entry video. If the result of this determination is that an unregistered person has not been detected (S2204 / No), the process returns to S2203 and continues the entry monitoring process. On the other hand, if an unregistered person has been detected (S2204 / Yes), the process proceeds to S2205.

[0100] In the process of S2205, the notification control unit 2101 judges whether the current time is within a predetermined time from the notification stop time. The time used as the judgment criterion is the notification stop time recorded in S2202. If the result of this judgment is that it is within the predetermined time from the notification stop time (S2205 / Yes), in S2206, the notification control unit 2101 controls the notification unit 304 not to perform notification processing. Thereafter, the process returns to S2203 and the entry monitoring process continues.

[0101] On the other hand, if the result of the determination in S2205 is that the predetermined time has elapsed since the notification stop time (S2205 / No), in the process of S2207, the notification control unit 2101 deletes the information on the notification stop time recorded in S2202. Then, in the process of S2208, the notification unit 304 performs notification processing in the same manner as in the above-described embodiment.

[0102] As described above, according to the present embodiment, it is possible to prevent a situation in which a non-registered person appears on the screen after a notification has been stopped, and notifications are issued repeatedly in a short period of time. This makes it possible to eliminate the hassle of having to issue notifications repeatedly.

[0103] Eighth embodiment In this embodiment, a process is described in which a notification is stopped once and then the non-registered person is allowed to enter the room for a certain period of time (no notification is given). Note that the hardware configuration of the information processing device according to this embodiment and the basic configuration of the image analysis system are similar to those shown in Figs. 1 and 2, respectively, and therefore a description thereof is omitted. Below, differences from the sixth embodiment are described. Note that the process according to this embodiment may be combined with the first to fifth embodiments without being limited to application in the sixth embodiment.

[0104] First, the flow of processing in this embodiment will be described with reference to Fig. 23 and Fig. 24. Fig. 23 is a block diagram showing an example of a functional configuration of an information processing device 2300 according to this embodiment. The determination unit 302 in Fig. 23 has the same function as that in Fig. 21, but in order to perform the processing of this embodiment, the determination unit 302 holds a primary permission list 2301. The primary permission list 2301 records face images of people (non-registered people) who are temporarily permitted to enter the room, and information on the time of permission. Note that the primary permission list 2301 may be provided as a functional module separate from the determination unit 302, and the form is not important.

[0105] Fig. 24 is a flow chart showing an example of a processing procedure for permitting entry of an unregistered person for a certain period of time in this embodiment. The processing in Fig. 24 is started in a state where notification has been performed in the procedure of the above-mentioned embodiment. The information processing device 2300 can realize each processing shown in Fig. 24 by the CPU 101 reading and executing a necessary program. The processes of S2401 and S2402 are similar to those of S2201 and S2202 in FIG. 22, respectively, and therefore will not be described.

[0106] In the process of S2403, the determination unit 302 acquires face images of unregistered people detected together with the registered people when the notification stop process was performed in S2401. Then, the acquired face images of unregistered people are registered in the primary allowance list 2301. The face image to be registered may be the largest face size, or multiple images with different face angles may be registered. In addition, the notification stop time recorded in S2402 may be registered in the temporary allowance list in a form linked to the face image. The processes in S2404 to S2406 are similar to those in S2203 to S2205 in FIG. 22, respectively, and therefore will not be described.

[0107] In the process of S2407, the determination unit 302 refers to the primary permission list 2301 and compares the unregistered person detected in the process of S2405 with the person registered in the primary permission list 2301. Then, in the process of S2408, the determination unit 302 determines whether the persons match. If the result of this determination is that there is a match (S2408 / Yes), the notification control unit 2101 controls the notification unit 304 not to perform notification processing. Then, the process returns to S2404.

[0108] On the other hand, if the result of the determination in S2408 is that there is no match, the notification unit 304 performs notification processing in the process of S2410. After that, the process returns to the process of S2404. Also, if it is determined in S2406 that a predetermined time has elapsed since the notification stop time (S2406 / No), in the process of S2411, the notification control unit 2101 deletes the record of the notification stop time, and ends this process.

[0109] As described above, according to the present embodiment, when a non-registered user who was registered on the temporary permission list is detected again, it is possible to prevent notifications from being issued repeatedly in a short period of time. This makes it possible to eliminate the hassle of having to issue notifications repeatedly.

[0110] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0111] 301 Analysis unit, 302 Judgment unit, 303 Video acquisition unit, 304 Notification unit, 305 Recording unit

Claims

1. An information processing device that performs entrance and exit management by generating an entry record and an exit record of a person to a specific location based on a video, comprising: An acquisition means for acquiring an image of an entrance of the specific location and an image of an interior of the specific location; a determination means for detecting a person included in the video image acquired by the acquisition means and determining whether the detected person is a person who has been registered in advance or a non-registered person; a notification means for instructing a notification device to issue a notification when the determination means determines that the image capturing the entrance includes an unregistered person heading to the specific location; a notification stopping means for stopping a notification by the notification device when the determination means determines that a registered person is included in the image capturing the interior of the specific location; 13. An information processing device comprising:

2. An information processing device that performs entrance and exit management by generating an entry record and an exit record of a person to a specific location based on a video, comprising: An acquisition means for acquiring an image of an entrance of the specific location; a determination means for detecting a person included in the video image acquired by the acquisition means and determining whether the detected person is a person who has been registered in advance or a non-registered person; a notification means for instructing a notification device to issue a notification when the determination means determines that the image capturing the entrance includes an unregistered person heading to the specific location; a notification stopping means for stopping a notification by the notification device in response to a registered person heading to the specific location being identified from the image capturing the entrance; 13. An information processing device comprising:

3. The information processing device according to claim 2, characterized in that the notification stopping means stops the notification by the notification device when a registered person heading to the specific location is identified as facing the entrance from the image capturing the entrance.

4. An information processing device as described in any one of claims 1 to 3, characterized in that when a registered person and a non-registered person heading to the specific location are detected in the same image within a certain period of time in the same image capturing the entrance to the specific location, the notification means is controlled so as not to issue a notification.

5. The registered person belongs to any one of a plurality of groups having different permission conditions for entering the specific location; An information processing device as described in any one of claims 1 to 4, characterized in that when the judgment means determines that a registered person heading to the specific location who belongs to a group that does not satisfy the permission conditions is included in the image capturing the entrance, the notification means instructs the notification device to make a notification.

6. An information processing device as described in any one of claims 1 to 5, characterized in that if the determination means determines that the image capturing the entrance includes an unregistered person heading to the specific location within a predetermined time after the notification stopping means stops the notification by the notification device, the notification means is controlled not to issue a notification.

7. The notification device further includes a registration unit that registers, in a permission list, a non-registered user determined by the determination unit when the notification by the notification device is stopped by the notification stopping unit, An information processing device as described in any one of claims 1 to 5, characterized in that if the judgment means determines that an unregistered person registered in the permission list by the registration means is included in the video footage of the entrance within a predetermined time after the notification stopping means has stopped the notification by the notification device, the notification means is controlled so as not to issue a notification.

8. The information processing device described in claim 1, further comprising a correction means for correcting the entry record using information of a registered person determined from the image photographed inside the specific location when the determination means determines that the image photographed of the entrance does not include a registered person heading to the specific location and when the notification by the notification device is stopped by the notification stopping means.

9. The information processing device of claim 8, characterized in that after the determination means determines that the image capturing the entrance includes a registered person heading to the specific location, the correction means corrects the exit record when the determination means determines that the image capturing the entrance does not include a registered person exiting the specific location.

10. An information processing device as described in any one of claims 1 to 9, characterized in that when a certain number of entries and exits occur within a specified period of time in the entry record and exit record, the notification means is controlled so as not to issue a notification until a certain period of time has elapsed since the latest exit record.

11. 11. The information processing apparatus according to claim 10, further comprising a generating unit for generating an entrance / exit record to which a record indicating a special event is added based on the entrance record and the exit record.

12. 12. The information processing apparatus according to claim 11, further comprising a recording means for recording the video relating to the entrance and exit record in a recording section.

13. The information processing device according to any one of claims 1 to 12, characterized in that the determination means detects a person's face from the image acquired by the acquisition means, acquires facial features, and determines whether the person is a registered person or a non-registered person by comparing the facial features with the facial features of the pre-registered registered persons.

14. 14. The information processing apparatus according to claim 13, wherein the determining means detects a person from a plurality of frames of the video image acquired by the acquiring means, acquires facial feature amounts, and determines whether the person is a registered person or a non-registered person.

15. the image capturing the entrance is an image captured by a first imaging device capturing an image of a face of a person heading towards the specific location, or a second imaging device capturing an image of a face of a person exiting the specific location, The information processing device according to any one of claims 1 to 14, characterized in that when the determination means determines that the image captured by the first imaging device includes an unregistered person heading to the specific location, the notification means instructs notification.

16. An information processing method executed by an information processing device that performs entrance and exit management by generating an entry record and an exit record of a person to a specific location based on a video, the method comprising: an acquisition step of acquiring an image of an entrance of the specific location and an image of an interior of the specific location; a determination step of detecting a person included in the video acquired in the acquisition step and determining whether the detected person is a registered person or a non-registered person; a notification step of instructing a notification device to issue a notification when it is determined in the determination step that the image capturing the entrance includes an unregistered person heading to the specific location; a notification stopping step of stopping the notification by the notification device when it is determined in the determination step that the registered person is included in the image capturing the interior of the specific place; 13. An information processing method comprising:

17. An information processing method executed by an information processing device that performs entrance and exit management by generating an entry record and an exit record of a person to a specific location based on a video, the method comprising: An acquisition step of acquiring an image of an entrance of the specific location; a determination step of detecting a person included in the video acquired in the acquisition step and determining whether the detected person is a registered person or a non-registered person; a notification step of instructing a notification device to issue a notification when it is determined in the determination step that the image capturing the entrance includes an unregistered person heading to the specific location; a notification stopping step of stopping the notification by the notification device in response to a registered person heading to the specific location being identified from the image capturing the entrance; 13. An information processing method comprising:

18. A program for controlling an information processing device that performs entrance and exit management by generating an entry record and an exit record of a person to a specific location based on a video, the program comprising: an acquisition step of acquiring an image of an entrance of the specific location and an image of an interior of the specific location; a determination step of detecting a person included in the video acquired in the acquisition step and determining whether the detected person is a registered person or a non-registered person; a notification step of instructing a notification device to issue a notification when it is determined in the determination step that the image capturing the entrance includes an unregistered person heading to the specific location; a notification stopping step of stopping the notification by the notification device when it is determined in the determination step that the registered person is included in the image capturing the interior of the specific place; A program for causing a computer to execute the following.

19. A program for controlling an information processing device that performs entrance and exit management by generating an entry record and an exit record of a person to a specific location based on a video, the program comprising: An acquisition step of acquiring an image of an entrance of the specific location; a determination step of detecting a person included in the video acquired in the acquisition step and determining whether the detected person is a registered person or a non-registered person; a notification step of instructing a notification device to issue a notification when it is determined in the determination step that the image capturing the entrance includes an unregistered person heading to the specific location; a notification stopping step of stopping the notification by the notification device in response to a registered person heading to the specific location being identified from the image capturing the entrance; A program for causing a computer to execute the following.

Images