Information processing device, information processing method, and program

Abstract

To provide an information processing device which manages an event for generating an alert indicating that an object person is likely to be unauthorized.SOLUTION: An information processing device comprises: acquisition means for acquiring first feature information of a person and second feature information subsequent to the first feature information from a sensor; means for generating an event which generates an alert in response to a first value representing a relationship between the first feature information and data for identifying the person; and means for deleting an event generated within a predetermined period according to a second value representing a relationship between the second feature information and the data for identifying the person.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program that manage an event for generating an alert indicating that a target person is likely to be unauthorized. [Background technology]

[0002] Video analytics for subject identification and recognition has become increasingly popular in recent years. Using algorithms and processing hardware, video footage can be processed to obtain data for identifying subjects within the video footage. Facial recognition is one of the widely used video analytics technologies for subject identification. It is currently employed in public safety solutions to assist law enforcement investigations, authentication methods for e-commerce payments, and contactless ID recognition for physical access authentication. In particular, most facial recognition solutions acquire facial data from video footage or images to identify a subject, match the facial data identifying the subject with facial data identifying known or authorized subjects, and, if the facial data of both sets of facial data are highly correlated, generate an alert indicating that the subject in the video footage or image is likely known or authorized. In various embodiments, authorized subjects may be subjects who have permission to enter a premises or have previously entered a premises and are deemed to pose a low threat.

[0003] Facial recognition technology, based on the concept of comparing data identifying a subject with data identifying authorized subjects, can enable systems to detect unauthorized subjects. For example, if the data identifying a subject does not match the data identifying authorized subjects, an alert-generating event is detected, indicating that the subject is an unauthorized subject or is likely unauthorized. Such unauthorized subject detection systems can be useful, for example, in detecting potential intruders when an intruder or unauthorized subject enters a premises. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the accuracy limitations and challenges of conventional video analytics technologies may be amplified, hindering their application to detecting unauthorized subjects. In particular, conventional video analytics technologies may be unable to produce consistent results based on the detection of subjects detected under various environmental or shooting conditions. If an unauthorized subject is likely to be detected under such accuracy limitations, an alert is immediately generated and provided to the user. Conventional techniques typically do not retain events (commands / signals) for generating alerts, but rather manage events for generating alerts. In other words, conventional techniques do not have a method for minimizing alerts that should not be generated. [Means for solving the problem]

[0005] Accordingly, it is an object of the present disclosure to substantially overcome the above-mentioned existing challenges for managing events that generate alerts indicating that a subject is likely unauthorized.

[0006] According to a first aspect of the present disclosure, there is provided a method for managing events to generate an alert indicating that a subject is likely to be unauthorized, the events including data identifying the subject, the method detecting a recall event including data identifying the subject within a predetermined period based on a timestamp at which the event is stored in a cache, the recall event indicating that the subject is unlikely to be unauthorized, and removing the event from the cache in response to detecting the recall event including data identifying the subject within the predetermined period.

[0007] According to a second aspect of the present disclosure, there is provided an apparatus for managing events for generating an alert indicating that a subject is likely unauthorized, the events including data identifying the subject, the apparatus including a memory in communication with a processor, the memory storing a computer program recorded therein, the computer program executable by the processor, causing the apparatus to at least: detect a recall event including data identifying the subject within a predetermined time period based on a timestamp at which the event is stored in a cache, the recall event indicating that the subject is likely unauthorized, and in response to detecting the recall event including data identifying the subject within a predetermined time period, remove the event from the cache.

[0008] According to yet another aspect of the present disclosure, there is provided a system for managing events to generate an alert indicating that a subject is likely unauthorized, the system including the device of the second aspect and at least one image capture device, one motion detection sensor, and / or one infrared sensor. [Brief explanation of the drawings]

[0009] The accompanying drawings, in which like reference numbers and characters refer to identical or functionally similar elements throughout the different views, and which, together with the following detailed description, are incorporated in and constitute a part of this specification, illustrate various embodiments and serve to explain various principles and advantages according to the embodiments.

[0010] [Figure 1A] FIG. 1A illustrates a system for detecting events for generating alerts based on image capture device input, according to one embodiment. [Figure 1B] FIG. 1B illustrates a system for detecting an event to generate an alert based on an infrared sensor input, according to one embodiment. [Figure 1C] FIG. 1C illustrates a system for detecting an event for generating an alert based on motion detection sensor input, according to one embodiment. [Figure 1D] FIG. 1D illustrates a system for detecting events for generating alerts based on image capture device input with grid-based image processing for motion detection according to one embodiment. [Figure 1E] FIG. 1E illustrates a system for detecting a recall event based on image capture device input, according to one embodiment. [Figure 2] FIG. 2 shows a flow chart illustrating a method for managing events to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. [Figure 3] FIG. 3 shows a block diagram illustrating a system for managing events to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. [Figure 4] FIG. 4 illustrates an example of managing events to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. [Figure 5] FIG. 5 shows a flowchart illustrating a method for managing events to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. [Figure 6] FIG. 6 shows a schematic diagram of a computer system suitable for use in implementing the system shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] Embodiments of the present disclosure will be better understood and readily apparent to those skilled in the art from the following written description, which is provided by way of example only, and in conjunction with the drawings, in which:

[0012] Some portions of the description which follow are explicitly or implicitly presented in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm, as used herein, is generally conceived to be a self-consistent sequence of steps leading to a desired result. These steps are steps requiring physical manipulations of physical quantities, such as electrical, magnetic, or optical signals, capable of being stored, transferred, combined, compared, and otherwise manipulated.

[0013] Unless otherwise indicated, and as will become apparent hereinafter, discussions throughout this specification utilizing terms such as "scan," "search," "determine," "replace," "generate," "initialize," "output," "receive," "identify," "predict," and the like will be understood to refer to acts and processes of manipulating and transforming data represented as physical quantities within a computer system into other data also represented as physical quantities within the computer system or other information storage, transmission, or display device.

[0014] This specification also discloses apparatus for performing the method operations. Such apparatus may be specially constructed for the required purposes, or may include a computer or other device selectively activated or reconfigured by a computer program stored in a computer. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various machines may be used with programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be utilized. The structure of the computer will be apparent from the following description.

[0015] Furthermore, this specification also implicitly discloses a computer program in that it will be apparent to one skilled in the art that the individual steps of the methods described herein may be implemented by computer code. The computer program is not intended to be limited to a particular programming language and its implementation. It will be understood that a variety of programming languages ​​and their coding can be used to implement the teachings of the present disclosure contained herein. Furthermore, the computer program is not intended to be limited to a particular control flow. There are many other variations of the computer program that can use different control flows without departing from the spirit or scope of the present invention.

[0016] Furthermore, one or more steps of a computer program may be executed in parallel rather than sequentially. Such a computer program may be stored on any computer-readable medium. The computer-readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a computer. The computer-readable medium may also include wired media, such as exemplified by the Internet system, or wireless media, such as exemplified by the GSM mobile phone system. The computer program, when loaded and executed on a computer or the like, effectively provides an apparatus for performing the steps of the preferred method.

[0017] In various embodiments, data identifying a subject may refer to information associated with or used to identify a subject based on attribute or characteristic information of the subject from input detected by an input device. The attribute or characteristic information may be physical characteristics of the subject, such as height, body size, hair color, skin color, facial information, clothing, belongings, other similar characteristics or combinations, or behavioral characteristics of the subject, such as body movement, limb position, movement direction, movement speed, the subject's walking style, standing style, movement style, speaking style, other similar characteristics or combinations, or other attribute or characteristic information. Such data identifying a subject may be obtained from inputs such as images or sensor data from an image capture device. Meanwhile, data identifying each subject in the approved subject or list of approved subjects may be stored in a database and matched with data identifying detected subjects obtained from inputs to determine whether the detected subject is likely unauthorized.

[0018] In various embodiments, an alert-generating event may refer to a signal, command, or action generated, triggered, processed, or recognized by a computer system or similar electronic device that indicates whether a subject is likely to be unauthorized. Rather than immediately generating an alert upon detecting a potentially unauthorized subject, each event may undergo a further decision step and be temporarily stored in a cache or memory for a predetermined period of time before generating an alert. This allows for managing events that indicate a detected subject is likely to be unauthorized and retaining, deleting, and minimizing alerts that should not be generated. FIG. 1A illustrates an example system 100 for detecting events for generating an alert based on input from an image capture device 104. The image capture device 104 captures an image including a subject 102. The subject 102 may be identified as appearing in the image based on characteristic information, such as facial information, height, hair color, movement, or other similar characteristic information, or a combination thereof. Based on the data identifying the subject obtained from the characteristic information, the subject 102 is matched against a list of subjects or approved subjects 106 to determine whether the subject 102 is likely to be unauthorized. In this embodiment, the subject 102 is matched against a list of approved subjects 106, which includes two subjects 106a, 106b and facial information identifying each subject. The facial information of the subject 102 obtained from the image is used to match against the facial information identifying each subject 106a, 106b in the list of approved subjects 106. As a result of the matching process based on the facial information, at 108, the subject 102 does not match any of the subjects in the list of approved subjects 106. It is determined that the subject is likely to be unauthorized, and an event that generates an alert 110 is detected. In various embodiments described below, such an event that generates an alert 110 can be managed before generating the alert 110 indicating that the event is likely to be unauthorized.

[0019] 1B illustrates an embodiment of a system 112 for detecting an event for generating an alert based on input from an infrared sensor 116, and FIG. 1C illustrates an embodiment of a system 118 for detecting an event for generating an alert based on input from a motion sensor 122. Each subject 114, 120 enters the detection area and field of the infrared sensor 116 and the motion sensor 122. The subjects 114, 120 are detected by the infrared sensor 116 and the motion sensor 122 based on characteristic information such as height, body size, temperature, body movement, limb position, walking speed, movement path, or other similar characteristic information, or a combination thereof. Based on data identifying the subjects 114, 120 obtained from the characteristic information, each subject 114, 120 is matched against subjects in a database or a list of approved subjects (not shown) to determine whether the subjects 114, 120 are likely to be unauthorized. The database includes data identifying each subject in the list of approved subjects, which data may be entered by a system user and / or obtained from previous detections of approved subjects based on input data from the database or data received from sensors 116, 122. The data identifying each subject in the list of approved subjects may be a combination of height, body size, movement path, and limb movement characteristics. Such data identifying each subject in the list of approved subjects in the database is compared with data identifying the subject 114, 120 obtained by sensors 116, 122 to determine whether the subject 114, 120 is likely unauthorized. If the data identifying the subject does not match the data identifying the subject in the list of approved subjects in the database, an alert-generating event is detected. In various embodiments described below, such an event for generating an alert is managed before the event generates an alert indicating that the subject is likely unauthorized.In one embodiment, a combination of images from at least one image capture device, such as 104, and data from at least one sensor, such as infrared sensor 116 or motion detection sensor 122, can be used as input to obtain data identifying a subject and to detect events that generate an alert indicating that the subject is likely unauthorized.

[0020] FIG. 1D illustrates an embodiment of a system 124 for detecting events based on image capture device input for generating alerts using grid-based image processing for motion detection. Grid-based image processing divides an image into multiple grids, each representing a portion of the image, as represented by the light gray grid in FIG. 1D. Multiple adjacent grids can form a predefined region of interest, as shown by the dark gray grid in FIG. 1D. In this embodiment, the predefined region of interest is the entrance door in FIG. 1D. Movement in the predefined region of interest is detected by monitoring pixel changes based on historical images or video frames or by comparing the image with an empty background image, as shown by the light gray grid in FIG. 1D. This reduces processing power requirements because only the predefined region of interest is monitored, and is used in many commercial video surveillance applications that automatically capture images when changes are detected in the predefined region of interest. According to the present disclosure, a subject 126 moving toward the door is detected when the subject causes pixel changes in the predefined region of interest, as represented by the white grid in FIG. 1D. Characteristic information of the detected subjects in the white grid, such as physical characteristics of the subject, such as height, body size, hair color, skin color, and facial features, or behavioral characteristics of the subject, such as the subject's walking, standing, and movement, can be obtained from the image capture device. Based on data identifying the subject 126 obtained from the image characteristic information, the subject 126 is matched against a list of approved subjects in a subject or database (not shown) to determine whether the subject 126 is likely unauthorized. The database includes data identifying each subject in the list of approved subjects, which data can be entered by a system user and / or obtained from previous detections of approved subjects based on input of images received from the database or image capture device. The data identifying each subject in the list of approved subjects can be a single or combination of characteristic information, such as facial information and body movements when entering a door.Such data identifying each subject in the list of approved subjects in the database is used to compare with data identifying the subject 126 obtained from the imaging device to determine whether the subject 126 is likely to be unauthorized. If the subject does not match an approved subject in the database, an alert generating event is detected. Similarly, such alert generating events are managed before the event generates an alert indicating that the subject is likely unauthorized.

[0021] In various embodiments below, a recall event may refer to a signal, command, or action generated, triggered, processed, or recognized by a computer system or similar electronic device that indicates whether a subject is unlikely to be unauthorized. A recall event may be detected when a subject is determined to match or highly correlate with a recognized subject, indicating the subject is unlikely to be unauthorized. According to the present disclosure, if an event that generates an alert indicating a subject is likely unauthorized is temporarily stored in a cache or memory for a certain period of time, i.e., a predetermined period of time, a recall event detected when the subject is determined to be unlikely to be unauthorized can be deleted from the cache or memory within the predetermined period of time so that an alert for the event is not generated. FIG. 1E illustrates a system 130 for detecting recall events based on input from an image capture device, according to one embodiment. An image capture device 134 captures an image including a subject 132, and the subject 132 is identified as appearing in the image based on feature information, such as facial information, height, hair color, movement, or other similar feature information, or a combination thereof. Data identifying the subject 132 is obtained based on the feature information from the image. The facial information of subject 132 is used to match the facial information identifying each subject 136a, 136b in a list of approved subjects 136 to determine whether the subject is likely to be unauthorized. In this embodiment, at 138, subject 132 is matched to one subject in the list of approved subjects 136, for example, subject 136a. This determines that the subject is unlikely to be unauthorized, and a recall event 140 is detected. In various embodiments described below, upon detecting a recall event 140, each cached event for generating an alert is scanned to determine whether the event contains data similar to the recall event 140.For example, if the cached recall event 140 and the event for generating the alert contain similar data, such as height, hair color, etc., sufficient to associate the subject 132 of the recall event with the subject of the cached event as the same subject, the event is removed from the cache upon determining the recall event 140 and no alert is generated for the event for generating the alert. In one embodiment, if an event is stored in the cache and is not removed from the cache by a recall event such as 140 within a predetermined period of time, the event is released from the cache after the predetermined period of time has expired and an alert is generated indicating that the subject is likely unauthorized.

[0022] Various embodiments provide apparatus and methods for managing events to generate an alert indicating that a subject is likely unauthorized. FIG. 2 shows a flowchart illustrating a method 200 for managing events to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. At step 202, the method includes determining whether there is a detection of a recall event indicating that the subject is unlikely to be unauthorized, including data identifying the subject, within a predetermined period from a timestamp at which the event including data identifying the subject is stored in a cache. In response to detecting a recall event within the predetermined period, at step 204, the method includes removing the event generating the alert from the cache.

[0023] According to one embodiment, in step 202, the method may include calculating a match score for the data identifying the subject. The match score refers to the degree of correlation between the data identifying the subject and the data identifying the approved subject. In particular, the match score is used to scale the degree of correlation between the data identifying the subject and the data identifying the approved subject. In one embodiment, the match score is scaled from 0% to 100%, where a match score of 0% indicates that the data identifying the subject and the data identifying the approved subject have no correlation or common feature information. On the other hand, a match score of 100% indicates that the data identifying the subject and the data identifying the approved subject are perfectly correlated and identical to each other. The method of step 202 may further include determining whether the match score for the data identifying the subject is higher than a match threshold. A match score higher than the match threshold indicates a high correlation between the data identifying the subject and the data identifying the approved subject. The match threshold is implemented as the minimum match score that represents a significant degree of correlation between both the data identifying the subject and the data identifying the approved subject that are recognized as corresponding and correlated to each other. In response to a determination that the match score is higher than a match threshold, indicating that the data identifying the subject and the data identifying the approved subject can be significantly determined to correspond and correlate with one another, a recall event is detected, indicating that the subject is unlikely to be unauthorized. In another embodiment, in response to a determination that the match score is lower than a match threshold, indicating that there is no significant degree of correlation between the data identifying the subject and the data identifying the approved subject, an event is detected that generates an alert, indicating that the subject is likely to be unauthorized.

[0024] In other embodiments, the subject may be matched against a list of approved subjects. A match score is generated when matching the subject against each subject in the list of approved subjects. Thus, calculating the match score may include calculating a plurality of match scores, each of the plurality of match scores indicating a degree of correlation between the data identifying the subject and a corresponding one of the plurality of approved subjects. The subject's match score is then determined as the highest match score among the plurality of match scores, and used to determine whether the match score is higher than a match threshold and whether the subject is likely to be unauthorized.

[0025] According to the present disclosure, the method may further include calculating a confidence score for the event for generating an alert. The confidence score refers to the degree of likelihood that the subject is unauthorized. In one embodiment, the confidence score for an event for generating an alert indicating a high likelihood that the subject is unauthorized is scaled from 0% to 100%, where a confidence score of 0% indicates a very low likelihood that the subject is unauthorized and a confidence score of 100% indicates a very high likelihood that the subject is unauthorized. The method may further include determining whether the confidence score for the event for generating the alert is lower than a confidence threshold. A confidence score lower than the confidence threshold indicates a low likelihood that the subject is unauthorized. In one embodiment, in response to determining a confidence score lower than the confidence threshold (a low likelihood that the subject is unauthorized), the method may include caching the event for generating the alert and determining a predetermined period of time based on a timestamp at which the confidence score determination is made and the event is cached. In another embodiment, in response to determining a confidence score higher than the confidence threshold (a high likelihood that the subject is unauthorized), the method may include generating an alert without caching the event.

[0026] According to one embodiment, the step of calculating the reliability score may include calculating the reliability score based on a match score of the data identifying the subject, and the step of determining whether the reliability score is lower than a reliability threshold may be further performed using the reliability score calculated based on the match score. In other words, the degree of likelihood that the subject is unauthorized may be determined based on the match score or the degree of correlation between the data identifying the subject and authorized subjects. According to another embodiment, if the detected subject is matched with a list of authorized subjects, the step of calculating the reliability score may include calculating the reliability score based on the highest match score among a plurality of match scores. In this particular embodiment, the reliability score may be calculated based on the highest match score among the plurality of match scores using Equation 1: (Number 1) Reliability score = 100% - highest matching score Equation (1) Here, the highest match score is the highest match score among the multiple match scores, each of which represents a scaled match score from 0% to 100% between the detected subject and one subject in the list of approved subjects. Regarding the degree of correlation between the data identifying the detected subject and the data identifying a subject in the list of approved subjects. In this embodiment, if a subject has a high highest match score, indicating that the subject is highly correlated with approved subjects, a low confidence score is obtained, indicating that the subject of the event is unlikely to be unauthorized due to the relatively high match score and the degree of correlation between the data identifying the subject and the data identifying an approved subject. Events that generate an alert are cached for a predetermined period of time, allowing them to be deleted by a recall event. The above calculation of a confidence score based on the match score or the highest match score among the multiple match scores is one of many examples for calculating a confidence score. Alternatively or additionally, other parameters, indicators, scores, or calculations can be used to determine the confidence score of an event and generate an alert indicating the degree of likelihood that the detected subject is unauthorized.

[0027] FIG. 3 illustrates a block diagram of a system 300 for managing events to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. In one example, the management of inputs is performed by at least an image capture device and / or at least a sensor 302. The system 300 includes the image capture device and / or sensor 302 in communication with a device 304. The device 304 can generally be described as a physical device including at least one processor 306 and at least one memory 308 containing computer program code. The at least one memory 308 and the computer program code, together with the at least one processor 306, are configured to cause the physical device to perform the operations described in FIG. 2. The processor 306 is configured to receive images or data from the image capture device and / or sensor 302 or to retrieve images or data from a database 310.

[0028] The image capture device may be a device such as a closed-circuit television (CCTV), webcam, surveillance camera, or other similar device, and the sensor may be a device such as an infrared sensor, motion detection sensor, or other similar device, and provide a variety of characteristic information and temporal information that the system can use to identify a subject and obtain data identifying the subject. In one embodiment, the characteristic information captured by the image capture device and / or sensor 302 to identify a subject and obtain data identifying the subject may include physical characteristic information such as height, body size, hair color, skin color, facial features, clothing, belongings, other similar characteristics or combinations, or behavioral characteristic information such as body movement, limb position, direction of movement, the subject's walking style, standing style, moving style, speaking style, other similar characteristics or combinations. For example, facial features may be used to identify a subject, and other characteristic information related to the subject may be obtained and aggregated as data identifying the subject and stored in memory 308 of device 304 or in database 310 accessible by device 304. In an embodiment, the temporal information obtained from the image capture device and / or sensor 302 may include a timestamp with which each image or data is identified. To identify the subject, the image and data timestamp may be stored in memory 308 of device 304 or in a database 310 accessible by device 304, and the subject's characteristic information may be aggregated as data identifying the subject. It should be understood that database 310 may be part of device 304.

[0029] The device 304 may be configured to communicate with the image capture device and / or sensor 302 and the database 310. In one example, the device 304 may receive images or data as input from the image capture device and / or sensor 302 or retrieve data from the database 310. After processing by a processor 306 within the device 304, the device 304 identifies and compares data identifying the subject and generates an output that can be used to manage the event to generate an alert indicating that the subject is likely unauthorized.

[0030] In one embodiment, after receiving images or data from the image capture device and / or sensor 302 or retrieving images or data from the database 310, the memory 308 and computer program code stored therein are configured to cause the processor 306 to instruct the device 304 to: determine whether an event includes data identifying a subject within a predetermined period based on a timestamp at which the event was stored in the cache; detect a recall event including data identifying the subject, indicating that the subject is unlikely to be unauthorized; and remove the event from the cache in response to detecting a recall event including data identifying the subject within a predetermined period. The timestamp at which the event was stored in the cache can be retrieved from the memory 308 of the device 304 to determine the predetermined period for detecting a recall event. In one embodiment, the memory 308 of the device 304 can be configured as a cache for storing events for generating an alert, and the device 304 can be configured to remove the events for generating an alert from the memory 308 in response to detecting a recall event within a predetermined period. The events are stored in the memory 308.

[0031] The device 304 may be further configured to calculate a confidence score of the event for generating an alert and determine whether the confidence score of the event for generating the alert is lower than a confidence threshold, the confidence threshold being retrieved from a memory 308 of the device 304. In one embodiment, the device may be configured to calculate the confidence score of the event and generate the alert based on a matching score of data identifying the subject retrieved from the memory 308 of the device 304 or a database 310 accessible by the device 304. The device 304 is further configured to determine whether the confidence score of the event for generating the alert is lower than a confidence threshold using the confidence score calculated based on the matching score.

[0032] In one embodiment, after receiving images or data from the image capture device and / or sensor 302 or retrieving images or data from the database 310, the memory 308 and computer program code stored therein are configured to cause the processor 306 to cause the device 304 to calculate a match score for the subject-identifying data further based on the data identifying the approved subject and determine whether the match score for the subject-identifying data is higher than a match threshold. The data identifying the approved subject may be retrieved from a database 310 accessible by the device 304, and the match threshold may be retrieved from the memory 308 of the device 304. The data identifying the approved subject stored in the database may be entered by a system user, obtained from a previous detection of the approved subject based on entry of images or data in the database 310, or received from the image capture device and / or sensor 302. In one embodiment, the device 304 is configured to calculate a plurality of match scores, each of which refers to a degree of correlation between the subject-identifying data and the subject-identifying data of a plurality of approved subjects. Each subject-identifying data of the plurality of approved subjects may be retrieved from a database 310 accessible by the device 304 to calculate a corresponding match score of the plurality of match scores. The subject-identifying data of the plurality of approved subjects stored in the database 310 may be entered by a system user or obtained from a previous detection of the authenticated subject based on input of images or data received from the database 310 or the at least one image capture device and / or the at least one sensor 302. The device may further be configured to determine a match score of the subject-identifying data as the highest match score of the plurality of match scores.

[0033] FIG. 4 illustrates an example of managing an event to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. As previously mentioned, unauthorized subject detection based on current facial recognition technology presents certain limitations under various environmental and photographic conditions. In this embodiment, five subject images 401a-401e are detected under different conditions, and facial information is used to match the subjects against a list of authorized subjects 402 to determine whether the subjects are likely to be unauthorized. Image 401a is detected first. Image 401a is detected with the subject's face partially detected. As a result, the subject identified from image 401a did not match any subject in the list of authorized subjects 402 based on the lack of complete facial information. Because it is determined that the subject in image 401a is likely unauthorized, an event generating alert 403a is detected. Subsequently, image 401b is detected. Image 401b is detected under low light and low image quality conditions. As a result, the subject identified from image 401b, based on unclear facial information, does not match any subject in the list of approved subjects 402. Because the subject in image 401b is determined to be likely unauthorized, an event generating alert 403b is detected. Similarly, subsequent images 401c and 401d are detected under conditions of low light and low image quality, with only a partial subject's face detected, and the subjects in images 401c and 401d do not match any subject in the list of approved subjects 402. As a result, events generating alerts 403c and 403d are detected, respectively.

[0034] As shown in FIG. 4, within a predetermined period of time, each of events 403a-403d is stored in cache 403, and a recall event can remove the event from the cache so that an alert is not generated. For example, immediately after an event generating alerts 403a-403d is detected, a recall event is detected based on image 401e with clear and complete facial information that can be matched to, for example, subject 402a in list 402 of approved subjects. Once a recall event is detected, each event in cache 403 is scanned to determine whether it contains data similar to a recall event that refers to a single subject. For example, recall event 404 and events 403a-403d stored in cache 403 may contain data such as an "object tracking ID" directed to a single subject, generated by a subject detection module based on general pixel movement tracking for similar subject detection. Alternatively, the recall event 404 and events 403a-403d stored in the cache 403 may include data such as behavioral characteristic information, such as subjects moving in the same direction, and / or physical characteristic information, such as the same shirt color. The characteristic information is sufficient to associate the subject of any of events 403a-403d with the subject of the recall event 404 as a single subject. As a result, the recall event 404 can be used to remove the alert-generating events 403a-403d from the cache 403, and no alert will be generated. By using a recall mechanism that can recall events for retaining alerts and events for generating alerts indicating subjects who are likely to be unauthorized by subsequent detection of a recall event indicating that the subject is unlikely to be unauthorized, events that generate alerts are managed and the generation of alerts that should not be generated is minimized.

[0035] 5 shows a flowchart 500 illustrating a method for managing events to generate an alert indicating that a subject is unlikely to be unauthorized, according to one embodiment. In step 502, images and / or data are received from at least one image capture device, such as 104, 134, and / or at least one sensor, such as 116, 122, or images and / or data are retrieved from database 310. The subject is identified based on the image and / or data input, and data identifying the subject is obtained. In step 504, the subject is matched with each of the plurality of recognized subjects based on data identifying the subject obtained from the input and data identifying each of the plurality of recognized subjects obtained from database 310. This generates a corresponding plurality of match scores, each indicating a degree of correlation between the subject and a subject in the plurality of recognized subjects. The data identifying each of the plurality of recognized subjects stored in database 310 is input by a system user or obtained from a previous detection of the authenticated subject based on input of images and / or data received from database 310 or at least one image capture device and / or at least one sensor. In step 506, the subject's match score is selected as the highest match score among the plurality of match scores. Further, in step 508, it is determined whether the subject's match score (the highest match score) is lower than a match threshold, and an event is detected that generates an alert indicating that the subject is likely unauthorized. In step 510, a confidence score for the event can be calculated to indicate a degree of likelihood that the subject is unauthorized. In one embodiment, the confidence score for the event can be calculated based on the match scores according to Equation 1. At step 512, it is determined whether the event's confidence score is below a confidence threshold, indicating that the subject is unlikely to be unauthorized. At 514, the event is cached using a recall event so that it can be deleted within a predetermined period of time. If the event's confidence score is above a confidence threshold, indicating that the subject is likely unauthorized, at step 522, the event is not cached but directly generates an alert indicating that the subject is likely unauthorized.

[0036] In step 520, it is determined whether the duration of the event stored in the cache exceeds a predetermined duration, and in step 522, the event is released from the cache and an alert can be generated indicating that the subject is likely unauthorized, or if it is still within the predetermined duration, the event remains stored for the remainder of the predetermined duration. Returning to step 508, if the match score of the data identifying the subject is determined to be higher than the match threshold, a recall event is detected indicating that the subject is unlikely to be unauthorized. In step 518, upon detecting such a recall event, each event in the cache is scanned to determine whether the event contains similar data as a recall event identifying the same subject. If both the recall event and the event in the cache consist of similar data referring to the same subject, such a recall event causes the event to be removed from the cache. Removing the event from the cache does not result in determining whether the event's predetermined duration has expired in step 520, and therefore no alert is generated in step 522 indicating that the subject is likely unauthorized.

[0037] Figure 6 illustrates an exemplary computing device 600 (hereinafter interchangeably referred to as computer system 600 or device 600), where one or more such computing devices 600 may be used to implement system 300 shown in Figure 3. Computing device 600 is provided by way of example only and is not intended to be limiting.

[0038] 6, exemplary computing device 600 includes a processor 604 for executing software routines. While a single processor is shown for clarity, computing device 600 may include a multi-processor system. Processor 604 is connected to a communications infrastructure 606 for communicating with other components of computing device 600. Communications infrastructure 606 may include, for example, a communications bus, crossbar, or network.

[0039] The computing device 600 further includes a primary memory 608, such as random access memory (RAM), and a secondary memory 610. The secondary memory 610 may include a storage drive 612, which may be, for example, a hard disk drive, a solid-state drive, or a hybrid drive, and / or a removable storage drive 614, which may include a magnetic tape drive, an optical disk drive, a solid-state storage drive (such as a USB flash drive, a flash memory device, a solid-state drive, or a memory card). The removable storage drive 614 reads from and / or writes to the removable storage medium 618 in a well-known manner. The removable storage medium 618 may include magnetic tape, an optical disk, a non-volatile memory storage medium, or the like, which is read from and written to by the removable storage drive 614. As will be appreciated by those skilled in the relevant art, the removable storage medium 618 includes a computer-readable storage medium having computer-executable program code instructions and / or data stored therein.

[0040] In alternative implementations, secondary memory 610 may additionally or alternatively include other similar means for allowing computer programs or other instructions to be loaded into computer device 600. Such means may include, for example, a removable storage unit 622 and interface 620. Examples of removable storage units 622 and interfaces 620 include program cartridges, cartridge interfaces (such as those found in video game console devices), removable memory chips (such as EPROMs or PROMs) and associated sockets, removable solid-state storage drives (such as USB flash drives, flash memory devices, solid-state drives, or memory cards), and other removable storage units 622 and interfaces 620 that allow software and data to be transferred from removable storage unit 622 to computer system 600.

[0041] Computer device 600 also includes at least one communications interface 624. Communications interface 624 allows software and data to be transferred between computer device 600 and external devices via communications path 626. In various embodiments of the present invention, communications interface 624 allows data to be transferred between computer device 600 and a data communications network, such as a public or private data communications network. Communications interface 624 may be used by computer device 600 to exchange data with other computer devices 600 that form part of an interconnected computer network. Examples of communications interface 624 may include a modem, a network interface (such as an Ethernet card), a communications port (such as serial, parallel, printer, GPIB, IEEE 1394, RJ45, USB, etc.), and an antenna with associated circuitry. Communications interface 624 may be wired or wireless. Software and data transferred via communications interface 624 are in the form of signals, which may be electronic, electromagnetic, optical, or other signals that can be received by communications interface 624. These signals are provided to communications interface 624 via communications path 624.

[0042] As shown in FIG. 6 , computing device 600 further includes a display interface 602 for performing operations to render images on an associated display 630 and an audio interface 632 for performing operations to play audio content via associated speakers 634.

[0043] As used herein, the term "computer program product" (or computer-readable medium, which may be a non-transitory computer-readable medium) refers, in part, to removable storage medium 618, removable storage unit 622, storage drive 612, or a carrier wave that carries software to communications interface 624 via communications path 626 (wireless link or cable). Computer-readable storage medium (or computer-readable medium) refers to a non-transitory, non-volatile, tangible storage medium that provides recorded instructions and / or data to computing device 600 for execution and / or processing. Examples of such storage media include magnetic tape, CD-ROM, DVD, Blu-ray® disk, hard disk drive, ROM or integrated circuit, solid-state storage drive (such as a USB flash drive, flash memory device, solid-state drive, or memory card), hybrid drive, magneto-optical disk, or computer-readable card such as a PCMCIA card, whether these devices are internal or external to computing device 600. Transient or intangible computer-readable transmission media that may also be involved in providing software, application programs, instructions and / or data to computer device 600 include wireless or infrared transmission channels, as well as network connections to other computers or network devices, the Internet or intranet, including email transmissions and information stored on websites and the like.

[0044] Computer programs (also referred to as computer program code) are stored in primary memory 608 and / or secondary memory 610. Computer programs may also be received via communications interface 624. Such computer programs, when executed, enable computing device 600 to perform one or more features of the embodiments discussed herein. In various embodiments, the computer programs, when executed, enable processor 604 to perform the features of the above-described embodiments. Thus, such computer programs represent controllers of computer system 600.

[0045] The software may be stored on a computer program product and loaded into the computing device 600 using the removable storage drive 614, the storage drive 612, or the interface 620. The computer program product may be a non-transitory computer-readable medium. Alternatively, the computer program product may be downloaded to the computing device 600 via the communications path 626. The software, when executed by the processor 604, causes the computing device 600 to perform the functions of the embodiments described herein.

[0046] It should be understood that the embodiment of FIG. 6 is presented by way of example only. Thus, in some embodiments, one or more functions of computing device 600 may be omitted. Also, in some embodiments, one or more functions of computing device 600 may be combined together. Furthermore, in some embodiments, one or more functions of computing device 600 may be divided into one or more components. For example, primary memory 608 and / or secondary memory 610 may function as memory 308 of device 304, while processor 604 may function as processor 306 of device 304.

[0047] It will be appreciated by those skilled in the art that numerous variations and / or modifications may be made to the present invention as illustrated in the specific embodiments without departing from the spirit or scope of the invention as broadly described. For example, while the above description primarily presents alerts in a visual interface, it will be appreciated that in alternative embodiments, the method may be implemented using other types of alert presentation, such as sound alerts. Certain modifications, such as the addition of access points, modifications to login routines, etc., are contemplated and may be incorporated. The present embodiments, therefore, are to be considered in all respects as illustrative and not restrictive.

[0048] This application claims the benefit of priority to Singapore Patent Application No. 10201910218X, filed November 1, 2019, the disclosure of which is incorporated herein by reference in its entirety. [Explanation of symbols]

[0049] 100, 112, 118, 124, 130, 300 systems 102, 114, 120, 126, 132 Target 104, 134 Imaging device 106, 136, 402 List of Approved Subjects 106a, 106b, 136a, 136b Target 110 Alert 116 Infrared Sensor 122 Motion detection sensor 140 Recall Event 302 Imaging devices and / or sensors 304 Equipment 306 processors 308 memory 310 Database 401a-401e images 403a-403d alerts 403 Cache 404 Recall Event 600 Computer equipment 602 Display Interface 604 processor 606 Communications Infrastructure 608 Primary Memory 610 Secondary Memory 612 storage drive 614 Removable Storage Drive 618 Removable Storage Media 620 Interface 622 Removable Storage Unit 624 Communication Interface 630 Display 632 Audio Interface 634 Speaker 636 Communication Path

Claims

1. an acquisition means for acquiring first feature information of a person and second feature information subsequent to the first feature information from a sensor; means for generating an event for generating an alert in accordance with a first matching score representing a relationship between the first feature information and data for identifying a person; a means for deleting the generated event within a predetermined period of time in accordance with a second matching score that indicates a relationship between the second feature information and the data for identifying the person; Equipped with Information processing device.

2. further comprising an alert issuing unit that issues an alert based on the event if the event is not deleted within the predetermined period of time; The information processing device according to claim 1 .

3. The alert may be audible or visual. The information processing device according to claim 2 .

4. The event includes the first characteristic information acquired from the sensor. The information processing device according to any one of claims 1 to 3.

5. The means for generating an event generates the event when the first matching score is lower than a threshold. The information processing device according to any one of claims 1 to 4.

6. the means for deleting the event deletes the event when the second matching score is higher than a threshold. The information processing device according to any one of claims 1 to 5.

7. The information for identifying a person is information for identifying an approved person. The information processing device according to any one of claims 1 to 6.

8. A process of acquiring first feature information of a person and second feature information subsequent to the first feature information from a sensor; generating an event for generating an alert in accordance with a first matching score representing a relationship between the first feature information and data for identifying a person; a process of deleting the generated event within a predetermined period of time in accordance with a second matching score representing a relationship between the second feature information and the data for identifying the person; The computer executes Information processing methods.

9. A process of acquiring first feature information of a person and second feature information subsequent to the first feature information from a sensor; generating an event for generating an alert in accordance with a first matching score representing a relationship between the first feature information and data for identifying a person; a process of deleting the generated event within a predetermined period of time in accordance with a second matching score representing a relationship between the second feature information and the data for identifying the person; to the computer, program.

Images