Intelligent door opening system based on monitoring

Abstract

The system and technology may be used to allow a person to access a secure area, open a door, etc. The technique may include detecting a person within a certain distance of the secure area, using a camera to capture a set of images of at least the person's facial features, preprocessing the set of images to identify facial features in at least one of the images, and using a processing circuit to identify the person's gaze status based on the identified facial features in at least one of the images. The technique may also include using a trained machine learning model to determine the person's intent to access the secure area based on the gaze status, and, based on the determination of intent, granting access to the secure area.

Description

Background Art

[0001] Physical access control systems can be used to restrict entry into a physical space and permit entry to authorized individuals. For example, a physical access control system can control access to a room, floor, building, safe (e.g., floor safe, wall safe, freestanding safe, etc.), cabinet, vehicle, case, and the like. In some systems, user devices, badges, or cards and access control devices are used, and the user devices, badges, or cards are read by a reader or communicate with the reader (e.g., via wireless communication using a short-range communication technique). The access control device can determine whether the user device has appropriate authorization or authentication to access the controlled physical area. The access control device can unlock a lock (e.g., a door lock) in response to determining that the user device includes or has the appropriate authorization or authentication provided.

Brief Description of the Drawings

[0002] [Figure 1] A system for determining an access intention according to some embodiments is shown. [Figure 2] A block diagram showing access intention processing according to some embodiments is shown. [Figure 3] A flowchart showing a technique for determining an access intention according to some embodiments is shown. [Figure 4] A machine learning engine for gaze detection or training and execution related to an access intention according to some embodiments is shown. [Figure 5] An example of a block diagram of a machine in which any one or more of the techniques described herein can function according to some embodiments is generally shown.

Modes for Carrying Out the Invention

[0003] In these drawings, which are not necessarily drawn to scale, similar numbers may describe similar components in different drawings. Similar numbers with different letter suffixes may represent different instances of similar components. These drawings illustrate the various embodiments discussed herein in general terms, not as limitations.

[0004] The systems and techniques described herein provide gaze-based intelligent door opening. Human behavior is complex. When people approach a door or secure area, they may not intend to open the door or access the secure area, but rather, for example, to pass it by, chat near it, or run errands around it. Current access control systems that rely on proximity may improperly unlock or open doors when such an action is not necessary. These unnecessary door openings can increase security risks, cause excessive wear and tear on electromechanical components, or consume unnecessary power.

[0005] Camera-based systems can be used to detect human proximity and capture human body movements or facial expressions. One or more facial images acquired by the camera can be used to more accurately determine a person's intentions. While facial orientation and speed of movement can help detect intentions, they can still be inaccurate. For example, a person might walk to a door and stop with the intention of performing an action near the door (e.g., grabbing some candy from a table located next to the door). Therefore, more accurate detection of a person's intentions is desirable.

[0006] The systems and techniques described herein may be used to automatically detect human intent associated with a door or secure area. Gaze is an important nonverbal communication cue because it contains rich information about human intent. By analyzing continuous gaze through two or more images of a face, the systems and techniques described herein can more accurately determine the intent to open a door or access a secure area.

[0007] Existing eye-tracking systems, often exemplary, utilize cameras equipped with infrared illuminators. These eye-tracking systems can perform corneal reflection photography. When one eye is directed to gaze at a small point of light, that light is reflected from the front of the eyeball (cornea). In the case of a camera positioned very close to this light source, when the person is focused on the camera, the corneal reflection will appear to be located at the center of the pupil. As the person shifts their gaze to the right, left, up, or down relative to the light, the corneal reflection is captured by the camera so that it is displaced relative to the center of the pupil.

[0008] One limitation of corneal reflection pupil-center-based gaze tracking systems is that they require a user-specific calibration process to adjust the linear model parameters of the gaze projection and only allow very small head movements during gaze estimation, which can be inconvenient for the user or result in inaccurate or uncertain readings. Furthermore, accurate gaze projection can only be achieved at very short distances (e.g., within 0.5 meters). In the case of automatic door opening systems, user-specific calibration is highly undesirable, as subjects may approach the door or secure area from different angular paths, walk freely without any restrictions on head movement, and may be located more than half a meter away.

[0009] The systems and techniques described herein provide technical solutions to the technical problems of long-range gaze estimation without a calibration process. The technical solutions may be based on gaze to determine a person's intent related to a door or secure area. In one example, a door may include any obstacle, material, or barrier that prevents a person from crossing an entrance (e.g., a framed area), such as an automatic door (e.g., a sliding door), a locked door, a partial barrier, a revolving door, or a gate. A door may include wooden doors, metal doors, glass doors, etc. A secure area or safe area may include any area that is prevented from being accessed by a person (e.g., by a door, bar, fence, laser, alarm trigger, etc., as described above).

[0010] The systems and techniques described herein incorporate gaze estimation into automated door opening or secure access systems. The systems and techniques described herein may be used to capture a person's eye movements or gaze status when approaching a door or secure area. A person focusing on a camera near the access path to the door or secure area may be used to trigger the opening or unlocking of the door, or otherwise making the secure area accessible.

[0011] Figure 1 shows a system 100 for determining access intent according to several embodiments. The system 100 includes a computing device 108 communicating with a camera 102, the camera 102 may be integrated into the computing device 108 or be separate from the computing device 108. The computing device 108 may include a reader 110 (for example, for user authentication), a processor 112, memory 114, or communication circuitry 116. The computing device 108 may optionally communicate with a server 106 via a network 104 (for example, for user authentication).

[0012] Camera 102 may be used to record images of a face or eye area. Images may be captured at a distance of 2-3 meters in some examples. In other examples, images may be captured at distances ranging from a few inches to 5-10 meters. The camera may capture these images, which may be processed by a computing device 108 to perform long-range gaze estimation. Gaze estimation may be used to detect a person's intent in the captured images quickly enough to enable a seamless door-opening experience. The aperture, focal length, and focus of the camera lens may be configured to allow a depth of field of approximately 2 meters in front of the door or secure area. However, the aperture, focal length, and focus of the camera lens may be configured to allow any other suitable depth of field in front of the door or secure area. The global shutter camera sensor of camera 102 may be used to image fast-moving objects (e.g., people). Camera 102 may have relatively high resolution to capture fine details of a person's eye movements at certain distances, including distances of up to 5-10 meters or more in some examples. In some examples, the original eye region image resolution may be higher than 50 x 50 pixels when the person is 2 meters away. In some examples, camera 102 is near-infrared (NIR) sensitive. Camera 102 may include an illuminator, such as an infrared illuminator, to induce a bright corneal reflection in a distant eye. In some embodiments, this illuminator may be a light source 118. In some embodiments, the illuminator may be a component separate from camera 102. The location of the bright corneal reflection relative to the pupil center may be used as a feature for gaze estimation. In some examples, multiple infrared illuminators can be used to estimate the orientation of a person's head based on multiple corneal reflection points. The gaze status may be used in conjunction with the head orientation to determine the user's intent.

[0013] Camera 102 may provide an approximate distance between the person and the imaging sensor, such as when Camera 102 is a 3D camera or an RGB-D camera. Head orientation or gaze status may be generated via depth images of the eye surface and eye region appearance images. In other examples, the approximate distance between the person and the imaging sensor may be provided by other means, including radar, lidar, or other cameras(s) in some examples. Camera 102 or system 100 may include an embedded ultra-wideband (UWB) component. UWB is a wireless technology used in real-time location services. When a user approaches a door, the UWB component may provide the distance between the person and Camera 102, or an angle of a vector representing the distance relative to Camera 102. These measurements may be used as conditional features for a gaze estimation model.

[0014] A typical eye-tracking system maps the gaze position to a projected field based on an estimated gaze direction vector. The computing device 108 avoids the need to obtain detailed gaze positions within the projected field and instead categorizes gazes into two types: gazes on doors (or secure areas) and gazes outside of doors (or secure areas). This simplified categorization allows gaze classification to be performed without requiring a user-specific calibration process. The system can be made more generalized by using training data containing a large variety of objects with different eye geometric shapes, eliminating the need for a calibration process for each user.

[0015] In one example, the camera 102 and light source 118 may be integrated into the door hardware, for example, as part of the door frame, push bar, handle, etc. In this example, focusing on the door results in gaze movement very similar to focusing on an embedded camera, especially for distant eyes. The gaze classification model may be simplified to classify gaze on the camera or gaze on something other than the camera. In another example, the camera 102 may be separate hardware installed adjacent to the door, for example, at a different fixed position for each installation. In this example, geometric shape measurements of the camera installation and door size may be used as conditional parameters for the gaze classification model (e.g., a machine learning trained model). The position of the camera 102 may be modeled as 3D coordinates in space with the center of the door or entrance to the secure area as the origin. The light source 118 may be separate hardware installed adjacent to the door (or secure area), for example, at a different fixed position for each installation. In this example, the light source position may be modeled as 3D coordinates in the same space as described above. Each light source position may be paired with the corresponding corneal reflection position in the eye image.

[0016] The intention of a person near a door can be represented through the probability of their intention to open the door. This can be determined by analyzing a sequence of images (e.g., video frames) and integrating the results. An exemplary analysis may include extracting the eye region from the frame and calculating the brightest corneal reflection location and pupil center. In some examples, the boundary points of the iris or cornea can be determined. These features can serve as input to a classification model. In some examples, nonlinear methods such as linear regression or support vector machines with nonlinear kernels can be applied to classify gaze status.

[0017] In some examples, the gaze status of an image can be estimated using a neural network that implements a deep learning architecture such as a convolutional neural network (CNN). Layers of the neural network can automatically extract features from eye region images or face images. Data-driven features may be based on a training dataset. In some models, the head orientation vector can be generated based on a face image or eye depth image (e.g., using facial landmarks), and this head orientation vector can be concatenated with features extracted from the eye image, along with user distance and angle, to be input into the classification portion of the model.

[0018] The CNN can be trained to accommodate variations in the placement of camera 102 or light source 118. For example, the CNN may be trained using a setup that includes one or more cameras or one or more light sources positioned at various points around a door or secure area. A human subject may be asked to look at the door (or secure area) in various ways and then look away while each of the one or more cameras or one or more light sources is active. The CNN model may be built using data from different camera positions or light sources so that the CNN can predict intentions under various optical conditions. The position of the camera or light may significantly affect the eye appearance image or other distance parameters observed by camera 102. Position vectors transformed from the 3D coordinates of camera 102 or light source 118 can be concatenated with one or more other gaze-related features that are input into the classification portion of the model.

[0019] In some examples, camera or light source locations may be labeled. In such cases, during installation, the installer can specify the location of camera 102 or light source 118. The label of the training location closest to the installation location may be set as the model input. In these examples, the conditional CNN model may be constructed by converting the camera and light source installation labels into condition vectors. These conditions may be shared with the same lower layers of the model that extract eye image-related features. Each condition may have its own weight for the top layer that generates the final gaze status. The variation in the top layer represents different eye appearance images under different camera and light source locations, having the same gaze for the status of the door (or secure area). Higher-level class labels can dramatically reduce the amount of data required for training.

[0020] The model (e.g., a CNN or other type of neural network) may be trained with simulated data that can be generated using models of eye features (e.g., cornea, pupil, iris, etc.) interacting with optical models of the light source 118 and camera 102. This data may be used for training or to augment actual training data.

[0021] After generating gaze status for an image, the results of consecutive frames can be integrated to predict the final intention. In some examples, an N-out-of-M scheme may be used. For example, if there are N frames showing gaze status on a door out of a total of M consecutive video frames, the intention to open the door or enter a secure area can be predicted. In other examples, Bayesian inference can be used to predict the probability of an intention to open the door (or enter a secure area) at the next time point. Prior probabilities of consecutive gaze status outputs under different intention conditions can be generated using a training dataset. In some examples, a recurrent neural network (RNN) model may be built on top of the output of a CNN for a set of frames to predict the probability of opening the door (or entering a secure area) in a sliding window.

[0022] In some examples, consecutive 2D frames can be considered as a 3D dataset with time as an additional dimension. One-step predictions can be made based on time blocks. For example, such data blocks can be input into a 3D convolutional network to directly generate intention probabilities. In this example, larger training datasets can be used for more complex models with more trainable parameters.

[0023] Figure 2 shows a block diagram 200 illustrating access intent processing in several embodiments. Block diagram 200 shows a person 202 near a door 204. Person 202 may be determined to be within a certain distance of the door 204, for example, based on face size captured by a camera unit 206 near the door 204, or via another distance criterion. In response to triggering this detection, video frames may be captured by the camera unit 206. Video frames may include sampled streaming video frames of 2 seconds or more. In block 208, one or more captured frames may be processed (for example, frame by frame). In block 208, feature extraction may be performed using a deep neural network or other model. The classifier may indicate gaze status. After M or more frames have been processed in block 208, in block 210, M or more frames may be combined to determine the intent to open the door. Block 212 includes one or more classifiers. In one example, the classifier may use a criterion of whether N out of M frames have a gaze status toward door 204 or the secure area. In another example, the RNN may use the gaze status of consecutive frames as input to determine the intent to open the door. Based on the output of block 210, the intent may be detected. If it is determined that the intent is to open door 204 or enter the secure area, door 204 may be opened, authentication may be performed (in some examples, authentication may have already been performed), or person 202 may be granted access in other ways. If it is determined that the intent is not to open door 204 or enter the secure area, further frames may be captured or processed (for example, as long as person 202 is within a certain distance of door 204) to determine, for example, whether person 202 gave rise to the intent to open the door or enter the secure area.

[0024] FIG. 3 shows a flowchart of a technique 300 for determining an access intention according to some embodiments. In one example, the operations of technique 300 can be performed by a processing circuit, for example, by executing instructions stored in a memory. The processing circuit can include a processor, a system-on-chip, or other circuits (e.g., wiring). For example, technique 300 can be performed by a processing circuit (or one or more hardware or software components of a device) such as that illustrated and described with reference to FIG. 1.

[0025] Technique 300 includes an operation 302 for detecting a person within a specific distance of a secure area or an automatic door. Operation 302 can include determining the size of a person's face (e.g., via one or more images captured by a camera) and comparing the face size to a threshold (e.g., when the face size is larger, the person is closer). Operation 302 can include receiving communication from a person's device and determining from the communication that the person is within the threshold proximity. In some examples, the communication device can include an embedded ultra-wideband (UWB) unit to provide both the distance and angle to the intention detection unit. In other examples, the communication device can use Bluetooth Low Energy (BLE), high-frequency radio frequency identifier (RFID), or other techniques that use time-of-flight to calculate the distance.

[0026] Technique 300 includes an operation 304 of using a camera to capture a set of images of at least facial features of a person. In some examples, operation 304 may include capturing the set of images using two crossed-polarized camera lenses, which can be achieved by adding a pair of crossed-polarized filters on the lenses. The camera may be one or more of a visible light camera (e.g., color, black and white, etc.), an infrared camera, an RGB-D camera, etc. Capturing the set of images may include capturing a video (e.g., a 1-second video, a 2-second video, a 5-second video, etc.). The video may be continuously captured after a trigger (e.g., face detection). The video may include a sliding window with a width of 1 to 3 seconds that was used to sample video frames for subsequent intention detection operations. When no face is detected or a decision to open the door is made, the capture of the video may be stopped. The facial features may include one or both eyes of a person. In other examples, the facial features may include the entire face of a person. The original images of the face and body may be captured from the camera.

[0027] Technique 300 may include an optional operation 305 for performing preprocessing to identify facial features such as facial landmarks. The facial landmarks may be used to extract an eye region image from the original face image. The landmarks may be used to generate the orientation of the head relative to the camera. In some examples, the orientation of the head may be identified based on a depth image from an RGB-D camera. The facial features may include landmarks in the eye region such as one or both eyes of a person, the pupil center, the iris boundary, or the eye boundary, an image of the entire face, etc.

[0028] Technique 300 includes an operation 306 that identifies a person's gaze status based on facial features in a set of images, for example, using a processing circuit. A gaze status may be identified as being oriented toward a secure area or an automatic door. The gaze status may include a probability vector of gaze status (e.g., the likelihood of whether the gaze status is oriented toward the door area). A pre-trained deep neural network can be used to perform the classification task to generate the gaze status probability vector. The gaze status may be generated based on a single video frame.

[0029] Technique 300 includes, for example, an action 308 for determining a person's intention to access a secure area or cross an automatic doorway based on their gaze direction, using a trained machine learning model. The trained machine learning model may include a recurrent neural network. In some examples, the trained machine learning model is trained using multiple camera positions relative to an access device (e.g., a card reader, a communication device, etc.) that includes processing circuitry. Action 308 may include determining the intention based on the detected trajectory of the person, which is obtained from a set of images. In one example, action 308 may include an integrated action based on gaze status outputs of consecutive video frames. In this example, a final intention detection result may be generated. In some examples, an N-out-of-M scheme may be used to generate the final result. In other examples, a recurrent neural network may be used to integrate the results and generate a final intention signal. Multiple camera positions or light source positions may be modeled as 3D coordinates, such as relative to the center of the door. Camera and light source positions may be input to generate gaze status for a single video frame.

[0030] Technique 300 includes an action 310 to allow access to a secure area or open an automatic door in response to determining intent. Action 310 may include opening a door, barrier, or obstacle blocking access to the secure area, or opening an automatic door, or otherwise allowing access to the secure area. In some examples, action 310 may include logging or flagging a person who has been determined to have the intent to enter the secure area as entering the secure area.

[0031] Technique 300 may include determining whether a person is authorized to access the secure area before granting access to the secure area. In response to determining that a person is authorized to access the secure area, Technique 300 may include granting access to the secure area. In some examples, Technique 300 may include determining intent or selecting a sequence of actions for authorizing a person based on the person's distance to the secure area. For example, when a person is within a certain distance but outside a second threshold distance, the sequence of actions may include authorizing the person first, and then determining intent. When a person is within a second threshold distance (which in some examples may coincide with or be closer to a certain distance), the sequence of actions may include determining intent first, and then authorizing the person. This sequence or distance (e.g., a certain distance or a second threshold distance) may be customized based on, for example, the layout of the area surrounding the secure area, user preferences, etc. In one example, determining whether a person is authorized to access the secure area may include using a set of images to authenticate the person.

[0032] Figure 4 shows several embodiments of a machine learning engine for training and execution related to gaze detection or access intent. The machine learning engine may be deployed to run on a computing device (e.g., a secure access device). The system can compute one or more weights for a criterion based on one or more machine learning algorithms. Figure 4 shows an exemplary machine learning engine 400 by some examples of the present disclosure.

[0033] The machine learning engine 400 uses a training engine 402 and a prediction engine 404. The training engine 402 uses input data 406 to determine one or more features 410 after, for example, receiving a preprocessing component 408. One or more features 410 may be used to generate an initial model 412, which may be updated iteratively or (for example, during reinforcement learning) with future labeled or unlabeled data to improve the performance of, for example, the prediction engine 404 or the initial model 412. The improved model may be redeployed for use.

[0034] The input data 406 may include head orientation vectors generated based on facial images (e.g., using facial landmarks) or depth images of the eyes, user distance or angle, features extracted from eye images, camera or light source position, captured or simulated images of people looking at a door or secure area, or averting their gaze (e.g., while moving), simulated data that may be generated using a model of eye features (e.g., cornea, pupil, iris, etc.).

[0035] In the prediction engine 404, current data 414 (e.g., frames captured by a camera of people within a specific distance of a door or secure area) can be input to the preprocessing component 416. In some examples, preprocessing component 416 and preprocessing component 408 are the same. The prediction engine 404 generates a feature vector 418 from the preprocessed current data, which is input to the model 420 to generate one or more criterion weights 422. The criterion weights 422 can be used to output predictions, as will be further described below.

[0036] The training engine 402 may operate offline (e.g., on a server) to train the model 420. The prediction engine 404 may be designed to operate online (e.g., in real time, on a mobile device, on a wearable device, etc.). In some examples, the model 420 may be periodically updated based on identified future data, such as through additional training (e.g., through updated input data 406, or based on labeled or unlabeled data output in weighting 422), or by personalizing a general model (e.g., the initial model 412) for a particular installation.

[0037] The input data 406 includes the original video frames acquired within the detection proximity, or conditional parameters such as the position of the camera or light source. In some examples, the conditional parameters may include the angle between the user's standing position and the intent detection unit, such as when a UWB device is used. The input data 406 may undergo a preprocessing step 408 to generate gaze-related features such as eye region facial landmarks, paired eye region images, corneal reflection points, pupil center, head orientation, or a cropped full-face image. The labels on the input data 406 may include the gaze status of a single frame (e.g., gaze in or outside the door region). Using the labeled data, a first step model may be trained to determine the gaze status of a frame. A second step model, used to synthesize sequential single-frame gaze statuses, may be trained to predict an intent to open a door based on the single-frame results. The output of the first step model, with the final door-opening intent label, may be used to train the second step model. The generation of the initial model 412 may be stopped according to specified criteria (e.g., after sufficient input data has been used, such as 1,000, 10,000, or 100,000 data points) or when the data converges (e.g., similar inputs produce similar outputs). The initial pre-trained model 412 may be implemented or adjusted for a specific door placement. In some examples, the pre-trained model 412 may be updated with further input data 406 until a satisfactory model 420 is generated. In some examples, federative learning may be used to update a shared model using usage data acquired across different door placements. In federative learning, the model 420 (or initial model 412) may be updated using data from each of two or more sites, such as site 1 424A, site 2 424B, site N 424N, etc. The data received from these sites may include local objective functions, local weights, etc. The model 420 may use the federative learning process to form a consensus among the various sites 424A-N. Model 420 may be updated, used, or transmitted to sites 424A-N for further data collection.Using associative learning could improve Model 420 over time.

[0038] The specific machine learning algorithm used in training engine 402 (e.g., Step 1 or Step 2 model) can be selected from many different potential machine learning algorithms. Examples of machine learning algorithms include artificial neural networks, Bayesian networks, instance-based learning, support vector machines, decision trees (e.g., Iterative Bisection 3, C9.5, Classification and Regression Tree (CART), Chi-Square Auto-Interaction Detector (CHAID), etc.), random forests, linear classifiers, quadratic classifiers, k-nearest neighbors, linear regression, logistic regression, and hidden Markov models. In one example, a convolutional neural network is used to extract embeddings from paired eye region images or cropped whole-face images. These embeddings are then concatenated with other gaze-related features such as head orientation, pupil center or iris boundary point related to the eye boundary, corneal reflection point, or camera or light source position coordinates. These concatenated features can then be input into a classification top such as a fully connected neural network, support vector machine, or logistic regression model to generate the gaze status for this frame. This gaze status can be a binary output or a probability vector indicating whether the subject in this frame is looking at the door area. Single-frame gaze statuses can be aggregated within a sliding window period, etc., to predict the final intent detection result. Single-frame gaze statuses can be aggregated and the final prediction generated using an N-out-of-M scheme, a recurrent neural network, or Bayesian probabilities. Once trained, Model 420 can output a gaze intent prediction or a gaze intent probability.

[0039] Figure 5 provides a general overview of an example block diagram of machine 500 in which any one or more of the techniques (e.g., methodologies) described herein may function according to several embodiments. In alternative embodiments, machine 500 may operate as a standalone device or be connected to other machines (e.g., networked). In a networked deployment, machine 500 may operate as a server machine, a client machine, or both in a server-client network environment. In one example, machine 500 may function as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. Machine 500 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be performed by the machine. Furthermore, although only a single machine is shown, the term “machine” should also be interpreted to include any set of machines that individually or collectively execute a set (or set) of instructions in order to perform any one or more of the methods described herein, such as cloud computing, software as a service (SaaS), and other computer cluster configurations. Machine 500 could be an example of a computing device 108 or a server 106.

[0040] The examples described herein may include or operate on logical or multiple components, modules, or mechanisms. A module is a tangible entity (e.g., hardware) capable of performing a specified operation during operation. A module includes hardware. In one example, the hardware may be specifically configured to perform a particular operation (e.g., hardwired). In one example, the hardware may include a configurable execution unit (e.g., a transistor, circuit, etc.) and a computer-readable medium containing instructions, which configure the execution unit to perform a particular operation during operation. Configuration may be performed under the direction of the execution unit or a loading mechanism. Thus, the execution unit is communicatively coupled to the computer-readable medium when the device is operating. In this example, the execution unit may be a component of two or more modules. For example, during operation, the execution unit may at some point be configured by a first set of instructions to implement a first module, and may be reconfigured by a second set of instructions to implement a second module.

[0041] The machine (e.g., a computer system) 500 may include a hardware processor 502 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), main memory 504, and static memory 506, some or all of which may communicate with each other via an interlink (e.g., a bus) 508. The machine 500 may further include one or more of the following: a display unit 510, an alphanumeric input device 512 (e.g., a keyboard), and a user interface (UI) navigation device 514 (e.g., a mouse). In one example, the display unit 510, the alphanumeric input device 512, and the UI navigation device 514 may be touchscreen displays. The machine 500 may further include a storage device (e.g., a drive unit) 516, a signal generation device 518, a network interface device 520, and one or more sensors 521 such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. The machine 500 may include an output controller 528, such as a serial (e.g., Universal Serial Bus (USB)), parallel, or other wired or wireless (e.g., infrared (IR), near-field communication (NFC)) connection, for communicating with or controlling one or more peripheral devices (e.g., a card reader).

[0042] The storage device 516 may include a non-temporary machine-readable medium 522 in which one or more data structures or instructions 524 (e.g., software) that embody or utilize any one or more of the techniques or functions described herein are stored. The instructions 524 may also reside entirely or at least partially in the main memory 504, static memory 506, or hardware processor 502 during their execution by the machine 500. In one example, one or any combination of the hardware processor 502, main memory 504, static memory 506, or storage device 516 may constitute the machine-readable medium.

[0043] Although the machine-readable medium 522 is shown as a single medium, the term “machine-readable medium” may include a single or multiple mediums configured to store one or more instructions 524 (for example, a centralized or distributed database, or associated caches and servers).

[0044] The term “machine-readable medium” may include any medium capable of storing, encoding, or carrying instructions for execution by machine 500, causing machine 500 to perform one or more of the techniques of the Disclosure, or capable of storing, encoding, or carrying data structures used by or associated with such instructions. Non-limiting examples of machine-readable mediums may include solid-state memory, as well as optical and magnetic media. Specific examples of machine-readable mediums may include non-volatile memory such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, CD-ROMs and DVD-ROMs.

[0045] Instruction 524 may further be transmitted or received via a communication network 526 using a transmission medium via a network interface device 520 that utilizes one of a number of transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Illustrative networks may include, among others, local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), plain old telephone service (POTS) networks, wireless data networks (e.g., IEEE 802.11 family standards known as Wi-Fi, IEEE 802.16 family standards known as WiMAX®), IEEE 802.15.4 family standards, and peer-to-peer (P2P) networks. In one example, the network interface device 520 may include one or more physical jacks (e.g., Ethernet®, coaxial, or telephone jacks) or one or more antennas for connecting to the communication network 526. In one example, the network interface device 520 may include multiple antennas for wireless communication using at least one of the single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term “transmission medium” is to be interpreted as including any intangible medium capable of storing, encoding, or carrying instructions for execution by the machine 500, including digital or analog communication signals or other intangible mediums for facilitating the communication of such software.

[0046] Example 1 is a method that includes detecting a person within a specific distance of a secure area, using a camera to capture a set of images of at least the person's facial features, preprocessing the set of images to identify facial features in at least one of the images, using a processing circuit to identify the person's gaze status based on the identified facial features in at least one of the images, using a trained machine learning model to determine the person's intent to access the secure area based on the gaze status, and granting access to the secure area based on the determination of intent.

[0047] In Example 2, the subject of Example 1 includes detecting a person within a certain distance, which involves determining the size of the person's face and comparing the face size to a threshold. In Example 3, the subject matter of Examples 1 and 2 includes allowing access to the secure area, which involves opening a door to the secure area.

[0048] In Example 4, the subject matter of Examples 1-3 involves capturing a set of images using two cross-polarized camera lenses. In Example 5, the subject matter of Examples 1-4 includes the fact that the camera is an infrared camera.

[0049] In Example 6, the subject matter of Examples 1-5 includes the fact that the set of images includes at least one second of video. In Example 7, the subject matter of Examples 1-6 includes the facial feature being one or both of a person's eyes.

[0050] In Example 8, the subject matter of Examples 1-7 includes the fact that the trained machine learning model is a recurrent neural network. In Example 9, the subject matter of Examples 1-8 includes the training of a trained machine learning model using multiple camera positions relative to an access device that includes a processing circuit.

[0051] In Example 10, the subject of Examples 1-9 is identified as being in a direction toward the secure area. In Example 11, the subject matter of Examples 1-10 involves determining intent based on a detected human trajectory, where the detected trajectory is obtained from a set of images.

[0052] In Example 12, the subject matter of Examples 1-11 includes determining whether a person is authorized to access the secure area before granting access to the secure area, and granting access to the secure area based on the determination that the person is authorized to access the secure area.

[0053] In Example 13, the subject of Example 12 includes determining intent or selecting a sequence of actions to authorize a person based on the person's distance to a secure area. In Example 14, the subject matter of Examples 12-13 involves determining whether a person is authorized to access a secure area, which includes authenticating a person using a set of images.

[0054] Example 15 is a machine-readable medium containing a command, which, when executed by a processing circuit, causes the processing circuit to perform actions such as detecting a person within a certain distance of a secure area, receiving a set of images of at least the person's facial features from a camera, preprocessing the set of images to identify facial features in at least one of the images in the set, identifying the person's gaze status based on the identified facial features in at least one of the images, using a trained machine learning model to determine the person's intent to access the secure area based on the gaze status, and granting access to the secure area based on the determination of intent.

[0055] In Example 16, the subject of Example 15 is an infrared camera, and capturing a set of images involves using two cross-polarized camera lenses. In Example 17, the subject of Examples 15-16 includes the fact that the set of images includes at least one second of video.

[0056] In Example 18, the subject matter of Examples 15–17 includes the fact that the trained machine learning model is a recurrent neural network. In Example 19, the subject matter of Examples 15-18 includes determining whether a person is authorized to access the secure area before granting access to the secure area, and granting access to the secure area based on the determination that the person is authorized to access the secure area.

[0057] Example 20 is a system comprising a camera that captures a set of images of at least a person's facial features, a processing circuit, and a memory containing instructions, wherein, when executed by the processing circuit, the instructions cause the processing circuit to detect a person within a specific distance of an automatic door based on the set of images, to preprocess the set of images to identify facial features in at least one of the images in the set of images, to identify the person's gaze status based on the facial features in at least one of the images, to use a trained machine learning model to determine the intention of a person crossing the automatic door based on the gaze status, and to output a control signal to open the automatic door based on the determination of the intention.

[0058] Example 21 is at least one machine-readable medium containing an instruction, which, when executed by a processing circuit, causes the processing circuit to perform an action that implements any of Examples 1 to 20. Example 22 is a device that includes means for implementing any of Examples 1 to 20.

[0059] Example 23 is a system for implementing any of Examples 1 through 20. Example 24 is a method for implementing any of Examples 1 through 20. Examples of the methods described herein can be implemented, at least in part, in machines or computers. Some examples may include computer-readable or machine-readable media encoded with instructions that can be operated to configure an electronic device to perform methods such as those described above. Implementations of such methods may include code, such as microcode, assembly language code, or higher-level language code. Such code may include computer-readable instructions for performing various methods. The code may form parts of a computer program product. Furthermore, in one example, the code may be tangibly stored on one or more volatile, non-temporary, or non-volatile tangible computer-readable media, either during execution or at some other point in time. Examples of these tangible computer-readable media include, but are not limited to, hard disks, removable magnetic disks, removable optical disks (e.g., compact disks and digital video disks), magnetic cassettes, memory cards or memory sticks, random access memory (RAM), and read-only memory (ROM).

Claims

1. It is a method, Detecting people within a specific distance of a secure area, Using a camera to capture a set of images of at least the facial features of the person, The set of images is preprocessed in order to identify the facial features in at least one of the images in the set of images. Using a processing circuit, the gaze status of the person is identified based on the identified facial features in at least one image. Using a trained machine learning model, determine the intent of the person accessing the secure area based on the gaze status, Based on the determination of the aforementioned intent, access to the secure area will be permitted. A method that includes this.

2. The method according to claim 1, wherein detecting the person within the specified distance includes determining the face size of the person and comparing the face size with a threshold.

3. The method according to claim 1, wherein granting access to the secure area includes opening a door to the secure area.

4. The method according to claim 1, wherein capturing the set of images includes using two cross-polarized camera lenses.

5. The method according to claim 1, wherein the camera is an infrared camera.

6. The method according to claim 1, wherein the set of images includes a video of at least one second.

7. The method according to claim 1, wherein the facial feature is one or both of the person's eyes.

8. The method according to claim 1, wherein the trained machine learning model is a recurrent neural network.

9. The method according to claim 1, wherein the trained machine learning model is trained using a plurality of camera positions relative to an access device including the processing circuit.

10. The method according to claim 1, wherein the attention status is identified as being in a direction toward the secure area.

11. The method according to claim 1, wherein the determination of the intent is based on the detected trajectory of the person, and the detected trajectory is obtained from the set of images.

12. A method according to claim 1, further comprising determining whether a person is authorized to access the secure area before granting access to the secure area, and granting access to the secure area based on the determination that the person is authorized to access the secure area.

13. A method according to claim 12, further comprising determining the intent or selecting a sequence of actions to authorize the person based on the person's distance to the secure area.

14. The method of claim 12, wherein determining whether the person is authorized to access the secure area includes using the set of images to authenticate the person.

15. A machine-readable medium containing an instruction, wherein the instruction, when executed by a processing circuit, It detects people within a specific distance of the secure area, The camera receives a set of images of at least the facial features of the person, In order to identify the facial features in at least one of the images in the set of images, the set of images is preprocessed, Based on the identified facial features in at least one of the images, the gaze status of the person is identified. Using a trained machine learning model, the intention of a person accessing the secure area is determined based on the gaze status. Based on the determination of the intent, grant access to the secure area. At least one machine-readable medium that causes the processing circuit to perform an operation.

16. The camera is an infrared camera, and capturing the set of images involves using two cross-polarized camera lenses, wherein at least one machine-readable medium is according to claim 15.

17. The set of images comprises at least one machine-readable medium according to claim 15, including at least one second of video.

18. The at least one machine-readable medium according to claim 15, wherein the trained machine learning model is a recurrent neural network.

19. The at least one machine-readable medium according to claim 15, further comprising determining whether the person is authorized to access the secure area before granting access to the secure area, and granting access to the secure area based on the determination that the person is authorized to access the secure area.

20. It is a system, A camera that captures a set of images of at least a person's facial features, Processing circuit and The system includes a memory containing instructions, and the instructions, when executed by the processing circuit, Based on the set of images, the system detects the person who is within a specific distance of the automatic door. In order to identify the facial features in at least one of the images in the set of images, the set of images is preprocessed, Based on the facial features in at least one of the images, the gaze status of the person is identified. Using a trained machine learning model, the intention of the person crossing the automatic door is determined based on the gaze status. A system that causes the processing circuit to output a control signal for opening the automatic door, based on the determination of the aforementioned intention.

Images