Methods and systems for trajectory and intent prediction, and non-transitory computer-readable media

Abstract

A method and system for trajectory and intent prediction are provided. The method and system include operations comprising: receiving a user's observed trajectory and user behavior information; processing the observed trajectory using a machine learning technique to generate multiple predicted trajectories, the machine learning technique being trained to establish a relationship between the multiple trained observed trajectories and the trained predicted trajectories; adjusting the multiple predicted trajectories based on the user behavior information to determine a user intent to operate a target access control device; determining a threshold range for the target access control device within a given predicted trajectory among the multiple predicted trajectories; and performing an operation associated with the target access control device in response to determining that the target access control device is within the threshold range of the given predicted trajectory among the multiple predicted trajectories.

Description

[0001] Priority application

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 125,044, filed on December 14, 2020, the disclosure of which is incorporated herein by reference in its entirety. Background Technology

[0003] Trajectory prediction plays a crucial role in many tasks, such as intelligent access control systems. Trajectory prediction is generally defined as predicting the location of a mobile agent (e.g., a person, vehicle, or mobile device) at each time step within a predefined future time interval based on local trajectories observed over a certain period. Summary of the Invention

[0004] In some aspects, a method is provided, comprising: receiving, by one or more processors, a user's observation trajectory and user behavior information; processing the observation trajectory using machine learning techniques to generate multiple predicted trajectories, the machine learning techniques being trained to establish relationships between the multiple trained observation trajectories and the trained predicted trajectories; adjusting the multiple predicted trajectories based on the user behavior information to determine a user intent to operate a target access control device; determining a threshold range for the target access control device within a given predicted trajectory among the multiple predicted trajectories; and, in response to determining that the target access control device is within the threshold range of a given predicted trajectory among the multiple predicted trajectories, performing an operation associated with the target access control device.

[0005] In some respects, the target access control device includes a lock associated with a door, and the operation performed includes unlocking the door.

[0006] In some aspects, the method includes establishing a wireless communication link between a user's mobile device and a target access control device; exchanging authorization information through the wireless communication link; and performing operations after determining, based on the authorization information, that the user is authorized to access the target access control device.

[0007] In some aspects, the method includes determining, based on authorization information, that a user is authorized to access a target access control device before performing an operation; and delaying the execution of the operation after determining that the user is authorized until the target access control device is determined to be within a threshold range of a given predicted trajectory among multiple predicted trajectories.

[0008] In some aspects, the method includes determining, based on authorization information, that a user is authorized to access a target access control device before performing an operation; and, after determining that the user is authorized, preventing the operation from being performed in response to determining that the target access control device is outside a threshold range of a given predicted trajectory among multiple predicted trajectories.

[0009] In some respects, machine learning techniques include conditional variational autoencoders.

[0010] In some respects, adjusting multiple predicted trajectories based on user behavior information involves processing observed trajectories and user behavior information by a conditional variational autoencoder to generate multiple predicted trajectories, each of which is associated with a corresponding probability indicating the likelihood that the user will travel along the corresponding predicted trajectory.

[0011] In some respects, machine learning techniques include variational autoencoders.

[0012] In some respects, adjusting multiple predicted trajectories based on user behavior information involves concatenating the user behavior information with multiple predicted trajectories output by a variational autoencoder, each of the multiple predicted trajectories being associated with a corresponding probability indicating the likelihood that the user will travel along the corresponding predicted trajectory.

[0013] In some aspects, the method includes using a second machine learning technique to process connected user behavior information and multiple predicted trajectories, the second machine learning technique being trained to establish relationships between multiple trained user behavior information and predicted intentions to operate access control devices.

[0014] In some aspects, the method includes encoding the user's observation trajectory, wherein machine learning techniques are applied to the encoded user's observation trajectory.

[0015] In some aspects, the method includes determining whether the received user behavior information satisfies the minimum parameters of the user behavior information.

[0016] In some aspects, the method includes allowing the target access control device to perform an operation in response to determining that the received user behavior information satisfies the minimum parameters of the user behavior information.

[0017] In some aspects, the method includes preventing the target access control device from performing an operation in response to determining that the received user behavior information fails to meet the minimum parameters of the user behavior information.

[0018] In some respects, the minimum parameter includes the threshold amount of user behavior information for a specified type.

[0019] In some aspects, the method includes generating user behavior information by encoding a feature vector that includes at least one of the following: monitoring the user's body movements; monitoring the user's stride; identifying the time and location when the user operates different types of access control devices; identifying other client devices and other types of access control devices within the user's scope when a given access control device is being operated by the user; or identifying other users typically in his / her social network.

[0020] In some respects, machine learning techniques include first machine learning techniques, and also include: generating user behavior information through second machine learning techniques, the second machine learning techniques being trained to establish a relationship between training user behavior information and predicted user behavior information; and generating user intent to operate a target access control device through third machine learning techniques, the third machine learning techniques being trained to establish a relationship between training user behavior information connected to the same set of trajectories and predicted user intent to operate the access control device.

[0021] In some respects, each of the first, second, and third machine learning techniques is trained end-to-end.

[0022] In some aspects, a system is provided comprising: one or more processors coupled to a memory, the memory including nontransitory computer instructions that, when executed by the one or more processors, perform operations including: receiving a user's observed trajectory and user behavior information; processing the observed trajectory using machine learning techniques to generate a plurality of predicted trajectories, the machine learning techniques being trained to establish relationships between the plurality of trained observed trajectories and the trained predicted trajectories; adjusting the plurality of predicted trajectories based on the user behavior information to determine a user intent to operate a target access control device; determining a threshold range for the target access control device within a given predicted trajectory among the plurality of predicted trajectories; and performing operations associated with the target access control device in response to determining that the target access control device is within a threshold range of the given predicted trajectory among the plurality of predicted trajectories.

[0023] In some aspects, a non-transitory computer-readable medium is provided, comprising non-transitory computer-readable instructions for performing operations including: receiving a user's observed trajectory and user behavior information; processing the observed trajectory using machine learning techniques to generate multiple predicted trajectories, the machine learning techniques being trained to establish relationships between multiple trained observed trajectories and trained predicted trajectories; adjusting the multiple predicted trajectories based on the user behavior information to determine a user intent to operate a target access control device; determining a threshold range for the target access control device within a given predicted trajectory among the multiple predicted trajectories; and performing an operation associated with the target access control device in response to determining that the target access control device is within the threshold range of a given predicted trajectory among the multiple predicted trajectories. Attached Figure Description

[0024] Figure 1 This is a block diagram of an example access control system according to some implementation methods.

[0025] Figure 2 An example access control system based on trajectory prediction according to an exemplary implementation is shown.

[0026] Figures 3A to 3C It can be deployed in some implementation methods Figure 1 A block diagram of an example trajectory and intent prediction system within an access control system.

[0027] Figure 4 It can be deployed in some implementation methods Figure 1 , Figure 2 as well as Figures 3A to 3C The sample database within the system.

[0028] Figure 5 This is a flowchart illustrating an example operation of an access control system according to an example implementation.

[0029] Figure 6 This is a block diagram illustrating an example software architecture that can be used in conjunction with various hardware architectures described herein.

[0030] Figure 7 This is a block diagram illustrating the components of a machine according to some example embodiments. Detailed Implementation

[0031] Example methods and systems for trajectory and intent prediction-based access control systems (e.g., physical or logical access control systems) are described. In the following description, numerous specific details are set forth for illustrative purposes to provide a thorough understanding of the example implementations. However, it will be apparent to those skilled in the art that embodiments of the invention can be practiced without these specific details.

[0032] In a typical access control system, a user carries a physical card or device containing a set of credentials (e.g., authorization information). When the physical card or device is brought within approximately 20 centimeters of the access device (e.g., an electronic door lock), these credentials are exchanged with the access device. The access device then determines whether the credentials authorize the user to access the device, and if so, grants access (e.g., unlocks the door). While such systems generally function well, they require the user to be very close to the access device to operate it. This can introduce various delays when operating the device and can potentially frustrate the user.

[0033] As mobile devices become more common, they can be programmed to carry the same set of credentials as commonly used physical cards. These mobile devices can communicate with the access device over longer distances, for example, using Bluetooth Low Energy (BLE) communication protocols. For instance, a mobile device can send credentials and exchange credentials with the access device within a range of up to 100 meters. In such a scenario, the access device can be operated even when the user is farther away than when using a physical card or device. Thus, when the user finally arrives at the access device, the access device has already received and authorized the credentials, granting or denying the user's access. No further action from the user is required to operate the device upon arrival (e.g., the user does not need to bring the physical card close to the access device).

[0034] However, these other methods of exchanging credentials via BLE introduce another problem. That is, if multiple access devices exist within the scope of the BLE communication protocol, credentials may be exchanged with a device the user does not intend to operate. For example, multiple electronic door locks may exist within the scope of a user's mobile device, and the user has credentials to access these multiple electronic door locks. However, the user may only intend to unlock or operate one of the multiple electronic door locks. As another example, a user may pass through a given door or access control device that the user is authorized to access, but may not intend to pass through or operate that given door or access control device. In such cases, determining the user's trajectory can play a crucial role in determining which of the multiple correct access devices to operate and the user's intent regarding operating such a device.

[0035] A typical trajectory prediction system takes a few steps of observed trajectory as input and generates a number of consecutive positions in a future timeline. These typical trajectory prediction systems can provide a set of trajectories predicted to be within a user's walking path. Most traditional and modern future trajectory prediction methods aim to build a model that is considered to be a prediction for many users. Such methods are severely limited because human movement is inherently unique and dynamic. Furthermore, the way one user holds a phone device may be very different from the way another user holds it. For example, some users prefer to hold the mobile device in their hand. This means that two-dimensional (2D) or three-dimensional (3D) position coordinates are very noisy because when a user walks, they swing their arms, causing the mobile device to move back and forth. Other users may place the phone in their front pocket or back pocket. Moreover, their stride will vary based on the user's height or his / her general walking preferences. Therefore, while typical trajectory prediction systems that rely on user location generally predict multiple future trajectories well, they typically do not consider user specificity and therefore lack the accuracy requirement that prevents multiple future trajectories from being accurately applied in the case of credential exchange and access devices.

[0036] The disclosed embodiments provide an intelligent solution that can accurately predict a user's future location and determine the user's intent, thus enabling the access control system to provide a proactive and seamless user experience while maintaining high security. The disclosed embodiments also provide a trajectory prediction system that predicts a user's trajectory based on past and current user behavior information. Based on the predicted trajectory or a given set of predicted trajectories and user behavior information, if a given access device is within the trajectory and authorized for user access (e.g., determined through remote exchange of credentials such as BLE), the given access device is operated. As an example, a given access device (e.g., a door lock) may initially communicate with the user's mobile device via a communication protocol (e.g., BLE) to exchange authorization data (e.g., credentials). If the given access device is subsequently determined to be within the user's predicted trajectory, and the user normally accesses or operates the device that day / at that time, or has a preference for operating the device relative to another device near the user, then the given access device is instructed to operate (e.g., the door lock is opened). Thus, when the user arrives at the given access device, the given access device is ready to be operated, without the user having to bring an access card to the device.

[0037] In some embodiments, the disclosed embodiments provide systems and methods for performing remote access control based on trajectory and intent prediction. According to the disclosed embodiments, a user's observed trajectory and user behavior information are received. The disclosed embodiments process the observed trajectory using machine learning techniques to generate multiple predicted trajectories. The machine learning techniques can be trained to establish relationships between multiple trained observed trajectories and trained predicted trajectories. The disclosed embodiments adjust the multiple predicted trajectories based on user behavior information to determine the user's intent to operate a target access control device. In response to determining that the target access control device falls within a threshold range of a given predicted trajectory among the multiple predicted trajectories, the disclosed embodiments perform an operation associated with the target access control device.

[0038] Figure 1 This is a block diagram illustrating an example system 100 according to various exemplary embodiments. System 100 may be an access control system including client device 120, one or more access control devices 110 such as lockable doors to control access to protected assets or resources, and an authorization management system 140 communicatively coupled via a network 130 (e.g., the Internet, BLE, Ultra-Wideband (UWB) communication protocol, telephone network).

[0039] Ultra-wideband (UWB) is a radio frequency (RF) technology that uses short, low-power pulses over a wide frequency spectrum. The number of pulses is on the order of millions of individual pulses per second. The width of the spectrum is typically greater than 500 MHz or more than 20 percent of the arithmetic center frequency.

[0040] UWB can be used for communication, for example, by encoding data via time modulation (e.g., pulse position coding). Here, a symbol is specified by a pulse on a subset of time units from a set of available time units. Other examples of UWB encoding can include amplitude modulation and / or polarity modulation. Wideband transmission tends to be more robust to multipath fading compared to carrier-based transmission techniques. Furthermore, the lower power of the pulse at any given frequency tends to reduce interference with carrier-based communication techniques.

[0041] UWB can be used in radar operations, providing positioning accuracy on the order of tens of centimeters. Due to the potentially variable absorption and reflection at different frequencies within the pulse, both surface features and occlusion (e.g., coverage) features of an object can be detected. In some cases, positioning provides the angle of incidence in addition to distance.

[0042] Client device 120 and access control device 110 can be communicatively coupled via electronic messages (e.g., packets exchanged via the Internet, BLE, UWB, WiFi Direct, or any other protocol). Although Figure 1 A single access control device 110 and a single client device 120 are shown; however, it should be understood that in other embodiments, multiple access control devices 110 and multiple client devices 120 may be included in system 100. As used herein, the term "client device" can be any machine that interfaces with a communication network (e.g., network 130) to exchange credentials with access control device 110, authorization management system 140, another client device 120, or any other component to gain access to assets or resources protected by access control device 110. Client device 120 may use UWB to obtain location information and calculate the current trajectory of client device 120.

[0043] In one implementation, client device 120 may provide current trajectory information to authorization management system 140. In some implementations, access control device 110 may determine the current trajectory of client device 120 and provide such information to authorization management system 140. Client device 120 (alone or in combination with access control device 110) collects various user behavior information from the user of client device 120. Such user behavior information may include the user's physical movement, the user's stride, the time and location of the user's operation of different types of access control devices 110, and any one or more other client devices or access control device types within the user's range when the user operates a given access control device. Various user behavior information may be stored and / or collected by authorization management system 140. In some implementations, client device 120 (alone or in combination with access control device 110) collects this information by monitoring the physical movement of client device 120 and / or the user's stride. In some implementations, at least some of the user behavior information is collected by client device 120 (alone or in combination with access control device 110) and / or authorization management system 140. The authorization management system 140 allows users to opt in or out of the collection of some or all user behavior information in order to protect user privacy.

[0044] In some cases, some or all components and functions of the authorization management system 140 may be included in the client device 120 (e.g., any machine learning techniques discussed with respect to the authorization management system 140 may be implemented on the corresponding client device 120). Any component in system 100 that performs trajectory and intent prediction may be implemented as a separate component of any of the authorization management system 140, client device 120, or access control device 110. The functionality of any component performing trajectory and intent prediction in system 100 may be implemented in a distributed manner across any of the authorization management system 140, client device 120, and / or access control device 110.

[0045] The authorization management system 140 uses machine learning techniques to predict one or more trajectories based on the current trajectory. The authorization management system 140 also receives or obtains user behavior information and adjusts the predicted one or more trajectories based on the user behavior information (e.g., the authorization management system 140 concatenates the predicted trajectories with the user behavior information). In some implementations, the authorization management system 140 uses machine learning techniques to compute feature vectors based on user behavior information associated with the user. In some implementations, the authorization management system 140 applies a machine learning model to the input current trajectory and the input user behavior information to generate predictions of one or more trajectories. Each trajectory may be associated with a specific or given probability that the user will adopt a path following that trajectory. The authorization management system 140 identifies the predicted trajectory with the highest probability. The authorization management system 140 then determines whether a given access control device 110 is within a specified range of the identified predicted trajectories. If yes, the authorization management system 140 instructs the given access control device 110 to grant access or perform an operation; otherwise, the authorization management system 140 instructs the given access control device 110 (on which the user has been authorized to access) to deny access or prevent the operation from being performed.

[0046] Client device 120 may be, but is not limited to, mobile phones, desktop computers, laptop computers, portable digital assistants (PDAs), smartphones, wearable devices (e.g., smartwatches), tablets, ultrabooks, netbooks, laptops, multiprocessor systems, microprocessor-based or programmable consumer electronics, or any other communication device that a user can use to access the network.

[0047] Access control device 110 may include an access reader device that is connected to and controls a physical resource (e.g., a door locking mechanism or a back-end server). The physical resource associated with access control device 110 may include a door lock, a vehicle's ignition system, or any other device that allows or denies access to a physical component and can be operated to allow or deny access to a physical component. For example, in the case of a door lock, access control device 110 may deny access, in which case the door lock remains locked and the door cannot be opened, or access control device 110 may allow access, in which case the door lock becomes unlocked to allow the door to be opened. As another example, in the case of an ignition system, access control device 110 may deny access, in which case the vehicle ignition system remains disabled and the vehicle cannot be started, or access control device 110 may allow access, in which case vehicle ignition becomes enabled to allow the vehicle to start.

[0048] Access control encompasses a range of systems and methods for managing, for example, human access to secure areas or secure assets. Access control includes: identification of authorized users or devices (e.g., vehicles, drones, etc.); activation of gates, doors, or other facilities used to protect area security; or activation of control mechanisms (e.g., physical or electronic / software control mechanisms) to allow access to secure assets. Access control device 110 forms part of a Picture Access Control System (PACS), which may include readers (e.g., online or offline readers) that store authorization data and can determine whether credentials (e.g., credentials or key devices from cards, clips, or personal electronic devices such as mobile phones) are authorized for use with actuators or control mechanisms (e.g., door locks, door openers, software control mechanisms, alarm closing mechanisms, etc.), or the PACS may include a host server, in which, in a centrally managed configuration, readers and actuators (e.g., via controllers) are connected to the host server. In a centrally managed configuration, the reader can retrieve credentials from the credential or key device and pass these credentials to the PACS host server. The host server then determines whether to authorize access to the secure area or secure asset with credentials and commands the executor or other control mechanism accordingly. While this document uses examples of physical access control, this disclosure is equally applicable to logical access control system (LACS) use cases (e.g., logical access to personal electronic devices, rider identification in transportation services, access and asset control in unattended stores, etc.).

[0049] Wireless PACS, such as those that utilize wireless communication between the reader and the credential or key device, can use RFID or Personal Area Network (PAN) technologies, such as IEEE 802.15.1, Bluetooth, Bluetooth Low Energy (BLE), Near Field Communication (NFC), ZigBee, GSM, CDMA, Wi-Fi, etc. Many of these technologies have drawbacks for a seamless user experience. For example, NFC has such a short range that credential exchange typically doesn't occur until the user is very close to the secure area or asset and attempts to gain access. The transmission of credentials to the reader and the response from the reader or host server can take several seconds, resulting in a frustrating user experience. Furthermore, the user typically has to, for example, take the device out of their pocket and place it on or very close to the reader to begin the process.

[0050] On the other hand, BLE devices have a range of tens of meters (e.g., ten to twenty meters). Therefore, credential exchange can be completed as the user approaches the reader. However, BLE, and many other PAN standards, do not provide accurate physical tracking of the device (e.g., ranging, positioning, etc.). Therefore, without some additional evidence of intent, the reader struggles to determine whether the user actually intends to gain access to a secure area or asset. For example, a problem arises if an authorized user simply walks past the reader in the lobby and the door is unlocked, or even opened. Evidence of intent can include things like touching a doorknob, making a gesture with a key device, etc. However, this can be a less than ideal user experience compared to a user simply walking to the reader and gaining access to the secure area without requiring further action or interaction on some part of the user's part.

[0051] To help address one or more of these or other problems, location technologies (e.g., using secure UWB ranging) can be used and combined with PAN discovery and key exchange. UWB location technologies can be more accurate than some traditional technologies, and can be accurate to, for example, tens of centimeters. UWB location technology can provide both the range and orientation of the credential or key device relative to the reader. This accuracy far surpasses, for example, the approximately ten-meter accuracy of BLE when the reader is not coordinated. The precision of UWB accuracy can be a useful tool for seamlessly determining user intent (e.g., whether the user is attempting to access a secure area or asset, or is simply passing by) and the user's current or predicted trajectory. For example, several areas can be defined, such as near the reader, at the reader, etc., to provide different contexts for understanding user intent. Additionally or alternatively, the accuracy of tracking helps to provide an accurate model of user movement or the direction of user movement from which intent can be discerned. Thus, the reader can classify user movement as, for example, possibly approaching the reader or simply walking past.

[0052] Once an intent is triggered, the reader can take action, for example, based on credentials exchanged via PAN technology. For offline readers, such as those not connected to a control panel or host server, the reader can directly control actuators or other control mechanisms (e.g., disconnecting a lock on a connected door). In a centrally managed PACS, (online) readers can forward credentials to a control panel or host server for operation.

[0053] Typically, access control device 110 may include one or more of a memory, a processor, one or more antennas, a communication module, a network interface device, a user interface, and a power supply or power supply device.

[0054] The memory of access control device 110 can be used in conjunction with the execution of application programs or instructions by the processor of access control device 110, and is used for temporary or long-term storage of program instructions or instruction sets and / or credentials or authorization data, such as credential data, credential authorization data, or access control data or instructions. For example, the memory may contain executable instructions used by the processor to run other components of access control device 110 and / or for access determination based on credential or authorization data. The memory of access control device 110 may include a computer-readable medium, which can be any medium that can contain, store, transmit, or transfer data, program code, or instructions for use by or in conjunction with access control device 110. The computer-readable medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses. More specific examples of suitable computer-readable media include, but are not limited to, electrical connections or tangible storage media having one or more lines, such as portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), dynamic RAM (DRAM), any solid-state storage device typically a compact disk read-only memory (CD-ROM), or other optical or magnetic storage devices. Computer-readable media includes computer-readable storage media, but should not be confused with computer-readable storage media; computer-readable storage media is intended to cover all physical, non-transitory, or similar embodiments of computer-readable media.

[0055] The processor of access control device 110 may correspond to one or more computer processing devices or resources. For example, the processor may be provided as a silicon chip, a field-programmable gate array (FPGA) chip, an application-specific integrated circuit (ASIC) chip, any other type of integrated circuit (IC) chip, a collection of IC chips, etc. As a more specific example, the processor may be provided as a microprocessor, a central processing unit (CPU), or multiple microprocessors or CPUs configured to execute instruction sets stored in the memory and / or internal memory of access control device 110. The access control device also encapsulates sensing devices that may or may not have antennas.

[0056] The access control device 110 may have one or more antennas and may be configured to provide wireless communication between the access control device 110 and a credential or key device (e.g., client device 120). The antennas may be arranged to operate using one or more wireless communication protocols and operating frequencies, including but not limited to IEEE 802.15.1, Bluetooth, Bluetooth Low Energy (BLE), Near Field Communication (NFC), ZigBee, GSM, CDMA, Wi-Fi, RF, UWB, etc. By way of example, the antenna may be an RF antenna, and therefore, RF signals can be transmitted / received in free space for reception / transmission by a credential or key device with an RF transceiver. In some cases, at least one antenna is designed or configured to transmit and / or receive UWB signals (referred to herein as a "UWB antenna" for simplicity), enabling the reader to communicate with client device 120 using UWB technology.

[0057] The communication module of the access control device 110 can be configured to communicate with one or more different systems or devices (e.g., one or more client devices 120 and / or authorization management system 140) remotely or locally according to any suitable communication protocol.

[0058] The network interface device of access control device 110 includes hardware for facilitating communication via a communication network (e.g., network 130) with other devices such as one or more client devices 120 and / or authorization management system 140, utilizing any of a number of transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), conventional telephone (POTS) networks, wireless data networks (e.g., the IEEE 802.11 standard series known as Wi-Fi, the IEEE 802.16 standard series known as WiMax), the IEEE 802.15.4 standard series, and peer-to-peer (P2P) networks, etc. In some examples, the network interface device may include an Ethernet port or other physical jack, a Wi-Fi card, a network interface card (NIC), a cellular interface (e.g., antennas, filters, and associated circuitry), etc. In some examples, the network interface device may include multiple antennas to perform wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies.

[0059] The user interface of access control device 110 may include one or more input devices and / or display devices. Examples of suitable user input devices that may be included in the user interface include, but are not limited to, one or more buttons, a keyboard, a mouse, a touch-sensitive surface, a stylus, a camera, a microphone, etc. Examples of suitable user output devices that may be included in the user interface include, but are not limited to, one or more LEDs, an LCD panel, a display screen, a touch screen, one or more lights, speakers, etc. It should be understood that the user interface may also include a combination of user input and user output devices, such as a touch-sensitive display.

[0060] Network 130 may include or operate in combination with the following networks: self-organizing network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless network, wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), BLE, UWB, the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), ordinary old-style telephone service (POTS) network, cellular telephone network, wireless network, A network, another type of network, or a combination of two or more such networks. For example, a network or part of a network may include a wireless network or a cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling can implement any data transmission technology of various types, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimization (EVDO), General Packet Radio Service (GPRS), Enhanced Data Rate Evolution of GSM (EDGE), the 3rd Generation Partnership Project (3GPP) including 3G, fourth-generation (4G) wireless networks, fifth-generation (5G) wireless networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other short-range or long-range protocols, or other data transmission technologies.

[0061] In the example, when client device 120 approaches access control device 110 (e.g., enters the range of the BLE communication protocol), client device 120 transmits its credentials over network 130. In some cases, credentials may be selected from multiple credentials based on the current geographic location of client device 120. For example, multiple credentials associated with different geographic locations may be stored on client device 120. When client device 120 comes within a certain distance (e.g., within 10 meters) of the geographic location associated with one of these credentials, client device 120 retrieves the associated credential from local storage.

[0062] In one example, client device 120 provides credentials directly to access control device 110. In this case, access control device 110 communicates credentials with authorization management system 140. Figure 1 The authorization management system 140 includes an authorization system 142 and a trajectory and intent prediction system 144. The authorization management system 140 may also include information regarding... Figure 6 and Figure 7 The described elements, such as processors and memory, store instructions that, when executed by the processor, enable the processor to control the functions of the authorization management system 140.

[0063] The authorization management system 140 searches a list of credentials stored in the authorization system 142 to determine whether the received credential matches a credential from a list of authorized credentials used to access secure assets or resources (e.g., doors or secure areas) protected by the access control device 110. In response to determining that the received credential is authorized to access the access control device 110, the authorization management system 140 accesses the trajectory and intent prediction system 144 to determine whether the trajectory of the client device 120 is predicted to be within a specified range (e.g., 2 meters) of the access control device 110, and whether user behavior information indicates an intent for the user to follow the given trajectory and / or operate the given access control device 110, as will be discussed in more detail below. Once the trajectory and intent prediction system 144 indicates to the authorization management system 140 that the client device 120 is predicted to follow a trajectory within the specified range of the access control device 110, and the user behavior information indicates an intent for the user to access or operate the device 110, the authorization management system 140 instructs the access control device 110 to perform an operation that authorizes access to the client device 120 (e.g., instructing the access control device 110 to unlock a door).

[0064] In another example, client device 120 provides credentials to authorization management system 140. Authorization management system 140 searches a list of credentials stored in authorization system 142 to determine if the received credentials match credentials from a list of authorized credentials used to access secure assets or resources (e.g., doors or secure areas) protected by access control device 110. In response to determining that the received credentials are authorized to access access control device 110, authorization management system 140 accesses trajectory and intent prediction system 144 to determine whether the trajectory of client device 120 is predicted to be within a specified range (e.g., 2 meters) of access control device 110, and whether user behavior information indicates that the user intends to follow the given trajectory and / or operate the given access control device 110, as will be discussed in more detail below. Once the trajectory and intent prediction system 144 indicates to the authorization management system 140 that the client device 120 is predicted to follow a trajectory within a specified range of the access control device 110, and user behavior information indicates that the user intends to follow the given trajectory and / or operate the given access control device 110, the authorization management system 140 instructs the access control device 110 (associated with the received credentials and within the geographical distance of the client device 120) to perform an operation that grants the client device 120 access (e.g., instructing the access control device 110 to unlock a door).

[0065] In one example, the trajectory and intent prediction system 144 is implemented locally on the access control device 110. In this case, the access control device 110 locally determines whether to grant / deny access to the client device 120 based on hard-coded range or threshold distance information. In another example, the trajectory and intent prediction system 144 is implemented on the client device 120 and provides trajectory and intent predictions to the access control device 110. The access control device 110 then determines whether the client device 120 is within the range associated with the access control device 110 to grant / deny access to the client device 120.

[0066] The trajectory and intent prediction system 144 trains one or more machine learning techniques implemented by the authorization management system 140 to predict one or more trajectories of the client device 120 based on observed trajectories and a set of user behavior information.

[0067] In one implementation, the trajectory and intent prediction system 144 implements a first machine learning technique that receives a feature vector representing a user's current trajectory and makes conditional predictions about one or more predicted trajectories based on this set of user behavior information. The predicted trajectories are input into an intent prediction machine learning technique that indicates a prediction about whether the user intends to operate an access control device within the range of the predicted trajectory.

[0068] In another embodiment, the trajectory and intent prediction system 144 implements a first machine learning technique that receives a feature vector representing a user's current trajectory and predicts one or more trajectories based on the current trajectory. The predicted one or more trajectories are concatenated with or otherwise combined with feature vectors representing user behavior information. In some cases, the feature vectors representing user behavior information are determined or provided by another machine learning technique. The predicted trajectories concatenated with the feature vectors representing user behavior information are input to an intent prediction machine learning technique that indicates a prediction about whether the user intends to operate an access control device within the predicted trajectory range.

[0069] In another embodiment, the trajectory and intent prediction system 144 implements a first machine learning technique that receives a feature vector representing a user's current trajectory and a feature vector representing user behavior information, and predicts one or more trajectories based on the current trajectory and the feature vector representing user behavior information. In some cases, the predicted one or more trajectories are also concatenated with or otherwise combined with the feature vector representing user behavior information. In some cases, the feature vector representing user behavior information is determined or provided by another machine learning technique. The predicted trajectory concatenated with the feature vector representing user behavior information is input to an intent prediction machine learning technique that indicates a prediction about whether the user intends to operate an access control device within the predicted trajectory range.

[0070] The trajectory and intent prediction system 144 processes observed current trajectories and / or user behavior information, along with corresponding ground truth trajectory and user behavior information pairs, to train machine learning techniques. Ground truth trajectories represent subsequent trajectories following one or more observed trajectories. For example, a ground truth trajectory represents the second segment of a trajectory that follows the first segment. Ground truth user behavior information represents a feature vector comprising a set of features present and / or missing from the input dataset of user behavior information.

[0071] The disclosed machine learning techniques can be implemented by any combination of neural networks, such as Long Short-Term Memory (LSTM) neural networks, autoencoders, variational autoencoders, conditional variational autoencoders, convolutional neural networks, radial basis function networks, deep feedforward networks, recurrent neural networks, gated recurrent units, denoising autoencoders, sparse autoencoders, Markov chains, Hopfield networks, Boltzmann machines, deep belief networks, deep convolutional networks, deconvolutional neural networks, generative adversarial networks, liquid machines, extreme learning machines, echo state networks, deep residual networks, support vector machines, Korhonen networks, or any combination thereof.

[0072] The trajectory and intent prediction system 144 obtains a specified range of activation or operation of the access control device 110. For example, the trajectory and intent prediction system 144 obtains the unique identifier of the access control device 110 and searches the database 400. Figure 4 Access control device range 430 is used to identify and retrieve ranges associated with the unique identifier of access control device 110. Different access control devices 110 or access control devices of different types 110 may be associated with different activation or operation ranges, and each access control device is stored in access control device range 430 along with its respective unique identifier. In some cases, access control device range 430 stores device types with corresponding ranges. In such cases, the device type is used to retrieve the associated range from access control device range 430 rather than the unique identifier. Trajectory and intent prediction system 144 determines whether the predicted trajectory falls within a specified range of access control device 110 and whether the intent prediction indicates a user intent to operate access control device 110. If so, trajectory and intent prediction system 144 instructs authorization management system 140 to activate or operate access control device 110 to grant access to client device 120.

[0073] In another example, the trajectory and intent prediction system 144 is implemented locally on access control device 110. In this case, access control device 110 is hard-programmed with a corresponding activation range (e.g., a range for access control device 110 stored in access control device range 430). The trajectory and intent prediction system 144 implemented on access control device 110 determines whether the predicted trajectory falls within the hard-coded range. If so, the trajectory and intent prediction system 144 causes access control device 110 to grant access to client device 120. In another example, the trajectory and intent prediction system 144 is implemented on client device 120 and provides trajectory and intent predictions to access control device 110. Access control device 110 then determines whether client device 120 is within the range associated with access control device 110 to grant / deny access to client device 120.

[0074] In some cases, the trajectory and intent prediction system 144 does not access any range information, but simply provides the predicted trajectory or a set of trajectories to the authorization management system 140, client device 120, and / or access control device 110. These devices then make a decision, jointly or individually, on whether the predicted trajectory is within a threshold range.

[0075] Figure 2An example access control system 200 based on trajectory and intent prediction according to an exemplary embodiment is illustrated. For example, a user 210 may carry a client device 120 (not shown), such as a mobile device or a telephone. The client device 120 (or access control device 110) may collect a set of observed 2D and / or 3D coordinates 230. The client device 120 (or access control device 110) may calculate the user's current trajectory. The client device 120 (alone or in combination with access control device 110) also collects a set of user behavior information, such as the user's stride and / or physical movement. The client device 120 (alone or in combination with access control device 110) may also collect the time and location at which the client device 120 is used to operate various access control devices 110. Using this information, trained machine learning techniques can determine or derive the user's intent or preference regarding operating certain access control devices 110. In some cases, the access control device 110 stores identifiers of the client devices 120 that operate the access control device 110, as well as the time when each client device 120 operates the access control device 110. Then, the user behavior information is aggregated into the corresponding profile of the client device 120 to generate user behavior information associated with the corresponding client device 120.

[0076] In one example, client device 120 may determine that two access control devices 220 and 222 are within a specified range of client device 120. For example, each of access control devices 220 and 222 is within BLE communication range with client device 120. In response, client device 120 retrieves credentials for both access control devices 220 and 222 and sends these credentials to authorization management system 140. Authorization management system 140 determines that client device 120 is authorized to access both access control devices 220 and 222. In response, authorization management system 140 delays granting access to a specific one of access control devices 220 or 222 until client device 120 is determined to be traveling along a predicted trajectory within a specific range 250 of the respective access control device 220 or 222.

[0077] In another example, a single access control device 110 may exist, guaranteeing access to an area protected by that single access control device 110. In such a case, the user's intent to enter the secure area is determined before instructing access control device 110 to grant access to a given client device 120. Specifically, before instructing access control device 110 to grant access to client device 120, a determination is made as to whether the user's predicted trajectory falls within the range of access control device 110.

[0078] For example, client device 120 (or access control device 110) provides the observed current trajectory to trajectory and intent prediction system 144. Client device 120 (alone or in combination with access control device 110) also provides user behavior information associated with client device 120. In another example, access control device 110 provides the observed current trajectory to trajectory and intent prediction system 144. Access control device 110 may provide user behavior information associated with client device 120 independently or in combination with client device 120.

[0079] The trajectory and intent prediction system 144 predicts one or more trajectories based on the current trajectory and user behavior information. The trajectory and intent prediction system 144 then identifies a predicted trajectory 240 along which the client device 120 is predicted to travel. In response to determining that the predicted trajectory 240 falls within the range of the first access control device 220, the trajectory and intent prediction system 144 instructs the authorization management system 140 to authorize access to the client device 120 by the first access control device 220 (e.g., instructing the first access control device 220 to perform an operation such as unlocking an electronic door lock). In response to determining that the predicted trajectory 240 does not fall within the range of the second access control device 222, the trajectory and intent prediction system 144 instructs the authorization management system 140 to deny access to the client device 120 by the second access control device 222 (e.g., instructing the second access control device 222 to remain locked, even if the client device 120's credentials are authorized to access the second access control device 222). In some cases, the predicted trajectory falls within the range of both the first access control device 220 and the second access control device 222. However, the trajectory and intent prediction system 144 predicts the user's intent to operate the first access control device 220 (e.g., based on user behavior information). In such a case, the trajectory and intent prediction system 144 instructs the authorization management system 140 to allow the first access control device 220 to access the client device 120 (e.g., instructs the first access control device 220 to perform an operation such as unlocking an electronic door lock), and to deny access to the client device 120 to the second access control device 222.

[0080] Figures 3A to 3C It can be deployed in some implementation methods Figure 1The system includes a block diagram of example trajectories and intent prediction system 144 within the system. Training input 310 includes model parameters 312 and training data 320, which may include pairs of training datasets 322 (e.g., input-output training pairs) and constraints 326. Model parameters 312 store or provide parameters or coefficients of the corresponding model in the machine learning model. During training, these parameters 312 are tuned based on the input-output training pairs of the training data 320. After the parameters 312 are tuned (after training), they are used by the trained model 360 to implement the trained machine learning (ML) model on a new dataset 370.

[0081] Training data 320 includes constraints 326, which can limit the constraints on a given trajectory and user behavior information. Paired training data 320 can include a set of input-output pairs 322, such as pairs of training observed trajectories and training user behavior information, along with corresponding training predicted trajectories (ground truth trajectories). The ground truth predicted trajectory represents the actual trajectory at one or more future time points, following the observed trajectory and that set of user behavior information at earlier time points. For example, the observed trajectory and user behavior measurements for the first segment of a path can be obtained at a first time point. The ground truth predicted trajectory represents the actual observed trajectory for the second segment following the first segment at a second time point.

[0082] Some components of the training input 310 may be stored separately from other components of the training input 310 in different off-site facilities. Paired training data 320 may include pairs of training user behavior information and corresponding training feature vectors for user behavior information (ground-based real user behavior information). Paired training data 320 may include pairs of training predicted trajectories linked to user behavior information and corresponding training intents (ground-based real intents) for operating the access control device. Ground-based real intents are generated by collecting information indicating whether a given access device 110 is operated when a trajectory linked to a specific user behavior information is observed.

[0083] Machine learning model training 330 trains one or more machine learning techniques based on a set of input-output pairs of paired training data 322. For example, model training 330 can train a first set of ML model parameters 312 by minimizing a loss function based on one or more ground-based measurements. Specifically, the first set of ML model parameters 312 can be applied to a training set of observed current trajectories conditioned on a set of user behavior information to estimate predicted trajectories. In some implementations, the derivative of the loss function is calculated based on a comparison of the estimated predicted trajectory with the ground-based true trajectory, and the first set of ML model parameters is updated based on the calculated derivative of the loss function. The first set of ML model parameters can be applied to a first machine learning technique (e.g., a conditional variational autoencoder) to generate a first prediction given new data 370.

[0084] As another example, model training 330 can train a second set of ML model parameters 312 by minimizing the loss function based on one or more ground-based measurements. Specifically, the second set of ML model parameters 312 can be applied to a training set of observed user behavior information to estimate feature vectors representing the user behavior information. In some implementations, the derivative of the loss function is calculated based on a comparison between the estimated user behavior information and ground-based real user behavior information, and the second set of ML model parameters is updated based on the calculated derivative of the loss function. The second set of ML model parameters can be applied to a second machine learning technique (e.g., a neural network) to generate a second prediction given new data 370.

[0085] As another example, model training 330 can train a third set of ML model parameters 312 by minimizing the loss function based on one or more ground-based measurements. Specifically, the second set of ML model parameters 312 can be applied to a training set of user behavior information connected to the predicted trajectory to estimate the predicted intent of operating the access control device. In some implementations, the derivative of the loss function is calculated based on a comparison between the estimated predicted intent of operating the access control device and the ground-based true intent of operating the access control device, and the third set of ML model parameters is updated based on the calculated derivative of the loss function. The third set of ML model parameters can be applied to a third machine learning technique (e.g., a neural network) to generate a third prediction given new data 370.

[0086] The first ML model, the second ML model, and / or the third ML model may all be stored on the same device (e.g., on client device 120, on access control device 110, or centrally on authorization management system 140). In some cases, one of the ML models may be implemented by one device (e.g., on client device 120, on access control device 110, or centrally on authorization management system 140), while the other ML model may be implemented by a different device (e.g., on client device 120, on access control device 110, or centrally on authorization management system 140).

[0087] Minimizing the loss function for multiple sets of training data results in training, tuning, or optimizing the model parameters 312 of the corresponding ML model. In this way, the ML model is trained to establish relationships between multiple training data (e.g., observed trajectories, observed user behavior information, connection trajectories, and user behavior information) and corresponding multiple prediction training data (e.g., predicted trajectories, predicted user behavior information, and predicted intentions to operate access control devices).

[0088] In one implementation, an ML model is trained using supervised learning techniques to estimate trajectories from training observation trajectories and user behavior information. In this case, to train the ML model, multiple training observation trajectories and user behavior information are retrieved along with their corresponding training predictions or estimated trajectories. For example, from a database of 400 (… Figure 4 Retrieve training observation trajectories and user behavior information from the training data 410 in the dataset. Apply the ML model to the first batch of training observation trajectories and user behavior information to estimate trajectories for a given group. This batch of training observation trajectories and user behavior information can be used to train the ML model with the same parameters, and can be used across a range from one set of training observation trajectories and user behavior information to all sets of training observation trajectories and user behavior information. In some implementations, the output or result of the ML model is used to compute or predict the first set of predicted trajectories.

[0089] The first set of predicted trajectories is applied to the loss function, and the gradient or derivative of the loss function is calculated based on the expected or ground truth set of the predicted trajectories. The updated parameters of the ML model are then calculated based on the gradient or derivative of the loss function. For example, the parameters of the ML model are stored in a trained machine learning technique 420 in a database 400. The ML model, along with the updated parameters, is then applied to a second batch of training observation trajectories and user behavior information to estimate the predicted trajectories for a given set again, and the predicted trajectories are applied to the loss function for comparison with the corresponding ground truth predicted trajectories. The parameters of the ML model are updated again, and this training process iterates for a specified number of iterations or epochs, or until a given convergence criterion is met.

[0090] After training the machine learning model, new data 370 can be received, including one or more observed trajectories and user behavior information. The trained machine learning technique can be applied to the new data 370 to generate a result 380, which includes predicted trajectories and the probability that the user will follow a path along each corresponding trajectory.

[0091] Figure 3B It shows that it can be used Figure 1 One implementation 301 of the trajectory and intent prediction system 144 deployed within the system. Observed trajectories can be received from a client device 120. The observed trajectories are processed by a trajectory encoder 371 to generate a feature vector corresponding to the current trajectory. The feature vector corresponding to the current trajectory is input into a trajectory prediction training model 361. The trajectory prediction training model 361 can be operated using a first set of model parameters 312 and can implement a conditional variational autoencoder. Any other type of neural network or machine learning technique can be similarly used as the trajectory prediction training model 361. The trajectory prediction training model 361 also receives user behavior information from a user behavior information training model 362. The user behavior information training model 362 is configured to receive a set of user behavior information (e.g., encoded as vectors) and generate feature vectors representing the user behavior information. In one example, the trajectory prediction training model 361 and the user behavior information training model 362 are trained end-to-end.

[0092] The trajectory prediction training model 361 processes the feature vector corresponding to the current trajectory and the feature vector representing user behavior information, and predicts one or more trajectories. The predicted trajectories may be processed by another machine learning technique (not shown) to determine the user's intent to operate within the range of one or more of the trajectories. After authorizing user credentials to access the access control device within the range of client device 120, in response to determining the user's intent to operate within the range of one or more of the trajectories, the trajectory and intent prediction system 144 instructs the access control device within the range of client device 120 to grant access or perform an action (e.g., unlock a door lock).

[0093] Figure 3C It shows that it can be deployed in Figure 1 Another implementation 302 of the trajectory and intent prediction system 144 within the system. An observed trajectory can be received from a client device 120. The observed trajectory is processed by a trajectory encoder 371 to generate a feature vector corresponding to the current trajectory. The feature vector corresponding to the current trajectory is input into a trajectory prediction training model 361. The trajectory prediction training model 361 can operate using a first set of model parameters 312 and can implement a variational autoencoder. Any other type of neural network or machine learning technique can be similarly used as the trajectory prediction training model 361. The trajectory prediction training model 361 processes the feature vector corresponding to the current trajectory and predicts one or more trajectories. These predicted one or more trajectories are provided to a connector 392. In some cases, the trajectory prediction training model 361 processes the feature vector corresponding to the current trajectory and a feature vector representing user behavior information, and predicts one or more trajectories. This one or more trajectories (predicted based on the current trajectory and the feature vector representing user behavior information) are provided to the connector 392.

[0094] User behavior information training model 362 is configured to receive a set of user behavior information (e.g., encoded as vectors) and generate feature vectors representing the user behavior information. These feature vectors representing the user behavior information are also provided to connector 392. In some implementations, user behavior information training model 362 may output a result or feature vector indicating a very low confidence score (e.g., the user behavior information fails to meet the minimum parameters of the user behavior information). This may be the case if the amount of user behavior information collected for a given user is insufficient (e.g., less than a threshold amount for a specified type, such as the time when different types and locations of access devices are operated, the user's stride, the user's body movements, whether the user is carrying client device 120 or keeping the device in their pocket, who the user is with when the access control device is operated, etc.). In such cases, predictions about user intent and trajectory can be assigned very low probabilities, and access to the access control device within the range of client device 120 may be blocked even if the credentials of client device 120 are authorized to access the access control device.

[0095] As an example, if a user recently adds a new key to an electronic door lock, there may be a training period that needs to be performed to generate a feature vector representing the user's behavior at a high confidence level. Once a feature vector representing the user's behavior with high confidence is achieved, intent prediction can be provided at a high probability level, and when the credentials of the authorized client device 120 access the access control device, the operation (e.g., unlocking) is indicated for the access control device within range of the client device 120. That is, there may be a training period associated with each new set of credentials or keys added by the user to the client device 120 to access the corresponding access control device. During this period, the access control device can be accessed and operated solely by the client device 120 using short-range communication protocols such as those within 20 centimeters (e.g., NFC). After training the user behavior information training model 362 for the newly added credentials is complete, the client device 120 can access and operate the access control device using long-range communication protocols such as those within 10 meters (e.g., BLE).

[0096] In one example, connector 392 combines (e.g., adjusts) one or more trajectories (based solely on the current trajectory prediction) based on feature vectors representing user behavior information. In another example, connector 392 combines (e.g., adjusts) one or more trajectories (based on the current trajectory and based on feature vectors representing user behavior information prediction) based on feature vectors representing user behavior information. The concatenation results of connector 392 are fed to intent prediction training model 363. Connector 392 can be implemented as any device that combines, multiplies, aggregates, sums, and / or otherwise generates a single representation from multiple input datasets.

[0097] Intent prediction training model 363 determines the user's intent to operate an access control device within a range of one or more trajectories. Intent prediction training model 363 may also be a neural network trained end-to-end. Intent prediction training model 363 is trained to output a prediction about whether the user intends to operate a given access control device within the range of client device 120 (e.g., based on a set of predicted trajectories adjusted by feature vectors representing user behavior information). Intent prediction training model 363 may output whether the user will operate a given access control device or has the intent to operate a given access control device (e.g., yes or no) and a probability indicating the likelihood that the user will operate the access control device. After authorizing user credentials to access an access control device within the range of client device 120, in response to a specified probability that the user's intent to operate the access control device within a range of one or more trajectories has a greater than a threshold probability, trajectory and intent prediction system 144 instructs the access control device within the range of client device 120 to grant access or perform an operation (e.g., unlock a door).

[0098] In one example, the intent prediction training model 363 is trained to predict user intent to operate an access control device at a specific time of day and when the user is within range of one or more other client devices 120. Specifically, the user behavior information training model 362 can provide the time (e.g., 9 AM and 5 PM) when the user operates a particular access control device and a set of identifiers for other client devices 120 within a specific range (e.g., 5 meters) of client device 120 when the access control device is operated. The intent prediction training model 363 can identify access control devices within range of the user's predicted trajectory and determine if the current time is within a specified threshold of 9 AM or 5 PM. In such a case, regarding social network information contained in the user behavior information, the intent prediction training model 363 also determines whether other client devices with identifiers matching the set of identifiers are within a specified range of client device 120. If so, the intent prediction training model 363 determines that the user's intent to operate the access control device is very high (e.g., 90%) and causes the access device to grant the user access. The intent prediction training model 363 can also determine that the user's intent to operate on another access control device within the range of the predicted trajectory is very low (e.g., less than 10%) (e.g., because the user has not operated the device in the past at a specific time and when within the range of that set of identifiers of other client devices 120), and cause the access device to deny the user access, even if the user has credentials for accessing or operating the access device.

[0099] In some implementations, user behavior data is collected over time after one or more trained models are implemented in the system including client device 120. In this case, the user behavior model (e.g., user behavior information training model 362) and the trajectory model (e.g., trajectory prediction training model 361) are decoupled. This may be because the user behavior model is unavailable due to a lack of training data. In this case, only the trajectory model is used to perform trajectory prediction so that the access device grants or denies user access. In such a case, the user behavior model includes a classifier that receives user behavior information (e.g., stride length, time of day, preferences, social network information, target gate) as input and outputs a probability of the user's intention to access a given access device. This probability is then combined with the trajectory probability to generate a prediction of the user's intention to operate a given access device.

[0100] In some implementations, the user behavior model receives multiple doors or access control devices instead of a single door or access control device as input. In this case, the user behavior model outputs the probability of a user's intention to operate each of the multiple access control devices. That is, it outputs a probability for each access control device, indicating the likelihood of a user operating a given access control device. This probability is combined with a trajectory probability to determine the user's intention to access a given access control device among the multiple access control devices or not to access it at all. In some implementations, the combination of the trajectory and the probability output by the user behavior model is generated as a weighted average of predictions made by two models—a user behavior information training model 362 and a trajectory prediction training model 361. The weights can be configured by the system administrator and / or can automatically and dynamically evolve over time as the model becomes more accurate given more user behavior data collected.

[0101] Figure 5 This is a flowchart illustrating an example operation of a process 500 of an access control system 100 according to an exemplary embodiment. Process 500 can be implemented as computer-readable instructions executable by one or more processors, such that the operation of process 500 can be performed partially or entirely by functional components of system 100; therefore, process 500 is described below by way of example with reference to system 100. However, in other embodiments, at least some operations of process 500 can be deployed on various other hardware configurations. Some or all operations of process 500 can be performed in parallel, out of order, or completely omitted.

[0102] At operation 501, the authorization management system 140 receives the user's observed trajectory and user behavior information. For example, the authorization management system 140 receives the current trajectory of the client device 120 and a set of user behavior information (e.g., the user's physical movement; the user's stride; preferences for accessing certain access devices at certain times of the day; the user's social network, such as who the user is with at different times of the day or on different days of the week; preferences for opening certain doors in a specific order, etc.).

[0103] At operation 502, the authorization management system 140 processes the observed trajectory using machine learning techniques to generate multiple predicted trajectories. The machine learning techniques are trained to establish relationships between multiple training observed trajectories and training predicted trajectories. For example, trajectory prediction training model 361 processes the current trajectory to generate predictions for one or more trajectories.

[0104] At operation 503, the authorization management system 140 adjusts multiple predicted trajectories based on user behavior information to determine the user's intent to access the target control device. For example, trajectory prediction training model 361 processes the current trajectory conditioned on user behavior information to adjust the prediction of one or more trajectories.

[0105] At operation 504, the authorization management system 140 determines a target access control device within a threshold range of a given predicted trajectory among multiple predicted trajectories. For example, the authorization management system 140 determines that the location of the first access control device 220 has a range 250 within one or more predicted trajectories.

[0106] At operation 505, in response to determining that the target access control device falls within a threshold range of a given predicted trajectory among multiple predicted trajectories, the authorization management system 140 performs an operation associated with the target access control device. For example, the authorization management system 140 instructs the access control device 220 to grant access to the client device 120 (e.g., by unlocking an electronic door lock). In some cases, the authorization management system 140 directly bypasses the access control device 110 to control the lock or security resource.

[0107] Figure 6 This is a block diagram illustrating an example software architecture 606 that can be used in conjunction with various hardware architectures described herein. Figure 6 This is a non-limiting example of a software architecture, and it will be understood that many other architectures can be implemented to facilitate the functionality described herein. Software architecture 606 can be implemented in hardware such as... Figure 7 The execution occurs on a machine 700, which includes a processor 704, a memory 714, and input / output (I / O) components 718, etc. A representative hardware layer 652 is shown, and this representative hardware layer 652 can represent, for example... Figure 7The machine 700. A representative hardware layer 652 includes a processing unit 654 having associated executable instructions 604. The executable instructions 604 represent executable instructions of the software architecture 606, including implementations of the methods, components, etc., described herein. Hardware layer 652 also includes a memory and / or storage device 656, which also has executable instructions 604. Hardware layer 652 may also include other hardware 658. Software architecture 606 can be deployed on... Figure 1 In any one or more of the components shown.

[0108] exist Figure 6 In the example architecture, software architecture 606 can be conceptualized as a stack of layers, each providing specific functionality. For example, software architecture 606 may include layers such as operating system 602, library 620, framework / middleware 618, application 616, and presentation layer 614. Operationally, application 616 and / or other components within a layer can activate API call 608 via the software stack and receive message 612 in response to API call 608. The layers shown are representative in nature, and not all software architectures have all layers. For example, some mobile or dedicated operating systems may not provide framework / middleware 618, while other operating systems may provide such a layer. Other software architectures may include additional or different layers.

[0109] Operating system 602 can manage hardware resources and provide general services. Operating system 602 may include, for example, a kernel 622, services 624, and drivers 626. Kernel 622 can serve as an abstraction layer between the hardware layer and other software layers. For example, kernel 622 can be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Services 624 can provide other public services to other software layers. Drivers 626 are responsible for controlling or interfacing with the underlying hardware. For example, depending on the hardware configuration, drivers 626 may include display drivers, camera drivers, BLE drivers, UWB drivers, etc. Drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Drivers, audio drivers, power management drivers, etc.

[0110] Library 620 provides common infrastructure used by application 616 and / or other components and / or layers. Library 620 provides functionality that allows other software components to perform tasks more easily than by directly interfacing with the functions of the underlying operating system 602 (e.g., kernel 622, services 624, and / or drivers 626). Library 620 may include system libraries 644 (e.g., the C standard library), which provide functions such as memory allocation, string manipulation, and mathematical functions. Additionally, library 620 may include API libraries 646, such as media libraries (e.g., libraries supporting the rendering and manipulation of various media formats such as MPREG4, H.264, MP3, AAC, AMR, JPG, and PNG), graphics libraries (e.g., OpenGL frameworks for rendering 2D and 3D graphical content on a display), database libraries (e.g., SQLite providing various relational database functionalities), web libraries (e.g., WebKit providing web browsing functionality), and so on. Library 620 may also include a wide variety of other libraries 648 to provide many other APIs to application 616 and other software components / devices.

[0111] The framework / middleware 618 (sometimes also called middleware) provides a higher level of common infrastructure that can be used by application 616 and / or other software components / devices. For example, the framework / middleware 618 can provide a variety of graphical user interface functions, advanced resource management, advanced location services, etc. The framework / middleware 618 can provide a wide range of other APIs that can be used by application 616 and / or other software components / devices, some of which may be specific to a particular operating system 602 or platform.

[0112] Application 616 includes built-in applications 638 and / or third-party applications 640. Examples of representative built-in applications 638 may include, but are not limited to: contact applications, browser applications, book reader applications, location applications, media applications, messaging applications, and / or game applications. Third-party applications 640 may include those used by entities other than the platform-specific vendor using Android. TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on platforms such as iOS. TM ANDROID TM , Mobile software running on the phone's mobile operating system or other mobile operating systems. Third-party application 640 may activate API calls 608 provided by the mobile operating system (e.g., operating system 602) to facilitate the functions described herein.

[0113] Application 616 can use built-in operating system functions (e.g., kernel 622, services 624, and / or drivers 626), libraries 620, and frameworks / middleware 618 to create a UI for interacting with the system's user. Alternatively or additionally, in some systems, interaction with the user can occur through a presentation layer, such as presentation layer 614. In these systems, the application / component "logic" can be separated from the aspects of the application / component that interact with the user.

[0114] Figure 7 This is a block diagram illustrating components of a machine 700 according to some example embodiments, the machine 700 being capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and executing any or more of the methods discussed herein. Specifically, Figure 7 A schematic representation of machine 700 is shown as an example of a computer system, within which instructions 710 (e.g., software, program, application, applet, application program, or other executable code) can be executed to cause machine 700 to perform any or more of the methods discussed herein.

[0115] Similarly, instruction 710 can be used to implement the devices or components described herein. Instruction 710 transforms a general, unprogrammed machine 700 into a specific machine 700 programmed to perform the described and illustrated functions in the described manner. In alternative embodiments, machine 700 operates as a standalone device or can be coupled (e.g., networked) to other machines. In network deployments, machine 700 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 700 can include, but is not limited to, server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, STBs, PDAs, entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart home appliances), other smart devices, web devices, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing instruction 710, which specifies the action to be taken by machine 700. Furthermore, although only a single machine 700 is shown, the term "machine" should also be considered to include a collection of machines that individually or jointly execute the instructions 710 to perform any or more of the methods discussed herein.

[0116] Machine 700 may include processor 704, memory / storage device 706, and I / O components 718, which may be configured to communicate with each other, for example, via bus 702. In example embodiments, processor 704 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processors 708 and 712 capable of executing instructions 710. The term "processor" is intended to include multi-core processor 704, which may include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although Figure 7 Multiple processors 704 are shown, but machine 700 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0117] Memory / storage device 706 may include memory 714, such as main memory or other memory storage devices, and storage cells 716, which processor 704 can access, for example, via bus 702. Storage cells 716 and memory 714 store instructions 710 embodying any one or more methods or functions described herein. During execution of instructions 710 by machine 700, instructions 710 may also reside wholly or partially within memory 714, within storage cells 716, within at least one processor in processor 704 (e.g., within the processor's cache memory), or any suitable combination thereof. Therefore, memory 714, storage cells 716, and the memory of processor 704 are examples of machine-readable media.

[0118] I / O component 718 may include various components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O component 718 included in a particular machine 700 will depend on the type of machine. For example, a portable machine such as a mobile phone will likely include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It should be understood that I / O component 718 may include... Figure 7Many other components are not shown. The I / O components 718 are grouped by function only for the sake of simplicity in the following discussion, and this grouping is by no means limiting. In various example embodiments, the I / O components 718 may include output components 726 and input components 728. Output components 726 may include visual components (e.g., displays, such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), auditory components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. Input components 728 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), haptic input components (e.g., physical buttons, touchscreens or other haptic input components that provide position and / or force for touch or touch gestures), audio input components (e.g., microphones), etc.

[0119] In other example implementations, I / O component 718 may include biometric component 739, motion component 734, environmental component 736, or positioning component 738, as well as various other components. For example, biometric component 739 may include components for detecting expressions (e.g., hand gestures, facial expressions, vocal expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 734 may include accelerometer components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), etc. Environmental component 736 may include, for example, a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an hearing sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects the concentration of hazardous gases for safety purposes or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. Position component 738 may include a positioning sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer from which air pressure can be used to determine altitude), an orientation sensor component (e.g., a magnetometer), etc.

[0120] Various technologies can be used to implement communication. I / O component 718 may include communication component 740, which is operable to couple machine 700 to network 737 or device 729 via coupling 724 and coupling 722, respectively. For example, communication component 740 may include a network interface component or other suitable device that interfaces with network 737. In other examples, communication component 740 may include wired communication component, wireless communication component, cellular communication component, near field communication (NFC) component, Bluetooth component (e.g., Bluetooth Low Energy), etc. Components and other communication components that provide communication via other modes. Device 729 can be any peripheral device from other machines or a wide variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0121] Furthermore, the communication component 740 can detect identifiers or include components operable to detect identifiers. For example, the communication component 740 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, data matrices, data symbols, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying audio signals from tags). Additionally, various information can be obtained via the communication component 740, such as location obtained via Internet Protocol (IP) geolocation, etc. Location obtained through signal triangulation, location obtained by detecting NFC beacon signals that can indicate a specific location, etc. Glossary:

[0122] In this context, "carrier signal" refers to any intangible medium capable of storing, encoding, or carrying transient or non-transient instructions executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of such instructions. Instructions can be sent or received over a network using transient or non-transient transmission media via network interface devices and using any of a number of well-known transmission protocols.

[0123] In this context, "client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to, mobile phones, desktop computers, laptop computers, PDAs, smartphones, tablet computers, ultrabooks, netbooks, laptops, multiprocessor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0124] In this context, "communication network" refers to one or more parts of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), LAN, BLE network, UWB network, wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a Polite Telephone Service (POTS) network, a cellular telephone network, a wireless network, etc. A network, another type of network, or a combination of two or more such networks. For example, a network or part of a network may include a wireless network or a cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling can implement any data transmission technology of various types, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimization (EVDO), General Packet Radio Service (GPRS), Enhanced Data Rate Evolution of GSM (EDGE), the 3rd Generation Partnership Project (3GPP) including 3G, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other long-distance protocols, or other data transmission technologies.

[0125] In this context, "machine-readable medium" refers to a component, device, or other tangible medium capable of temporarily or permanently storing instructions and data, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage devices (e.g., erasable programmable read-only memory (EEPROM)) and / or any suitable combination thereof. The term "machine-readable medium" should be considered to include a single medium or multiple media capable of storing instructions (e.g., a centralized or distributed database or associated cache memory and server). The term "machine-readable medium" will also be considered to include any or a combination of media capable of storing machine-executable instructions (e.g., code) such that, when executed by one or more processors of the machine, the instructions cause the machine to perform any or more of the methods described herein. Accordingly, "machine-readable medium" refers to a single storage device or apparatus, and a "cloud-based" storage system or storage network comprising multiple storage devices or apparatuses. The term "machine-readable medium" does not include signals themselves.

[0126] In this context, a “component” refers to a device, physical entity, or logic having boundaries defined by functional or subroutine calls, branch points, APIs, or other technologies that provide partitioning or modularity for specific processing or control functions. Components can interface with other components via their interfaces to perform machine processing. Components can be encapsulated functional hardware units designed for use with other components, and part of a program that typically performs a specific function related to that function. Components can constitute software components (e.g., code implemented on a machine-readable medium) or hardware components. A “hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various example implementations, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., an application or application portion) to operate to perform certain operations as described herein.

[0127] Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic permanently configured to perform certain operations. A hardware component may be a dedicated processor, such as a field-programmable gate array (FPGA) or an ASIC. A hardware component may also include programmable logic or a circuit system that is temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific part of a machine) uniquely tailored to perform the configured function and is no longer a general-purpose processor. It will be understood that decisions to implement hardware components mechanically in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., software-configured) circuit system may be driven by cost and time considerations. Therefore, the phrase “hardware component” (or “hardware-implemented component”) should be understood to include tangible entities, i.e., entities physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain way or perform certain operations described herein. Consider implementations where hardware components are temporarily configured (e.g., programmed), eliminating the need to configure or instantiate each hardware component at any given time. For example, where the hardware components include a general-purpose processor that is configured by software to become a dedicated processor, this general-purpose processor can be configured at different times as its own distinct dedicated processor (e.g., including different hardware components). The software accordingly configures one or more specific processors to constitute a particular hardware component at one time and different hardware components at different times.

[0128] Hardware components can provide information to and receive information from other hardware components. Therefore, the described hardware components can be considered communicatively coupled. In the presence of multiple hardware components, communication can be achieved through signal transmission between or among two or more hardware components (e.g., via appropriate circuitry and buses). In an example embodiment where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, by storing information in a memory structure accessible to the multiple hardware components and retrieving information from the memory structure. For example, one hardware component can perform an operation and store the output of that operation in a communicatively coupled memory device. Other hardware components can then access the memory device at a subsequent time to retrieve and process the stored output.

[0129] Hardware components can also initiate communication with input or output devices and operate on resources (e.g., collections of information). Various operations of the example methods described herein can be performed, at least partially, by one or more processors configured, either temporarily (e.g., by software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented, at least partially, by processors, where a particular processor or one or more processors are examples of hardware. For example, at least some operations of the method can be performed by one or more processors or processor-implemented components. Furthermore, one or more processors can also operate to support the execution of relevant operations in a "cloud computing" environment or as "Software as a Service" (SaaS). For example, at least some operations can be performed by a group of computers (as an example of a machine including processors), where these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs). The execution of certain operations can be distributed across processors, not just residing within a single machine, but deployed across multiple machines. In some example implementations, the processor or processor-implemented components may reside in a single geographic location (e.g., within a home environment, office environment, or server cluster). In other example implementations, the processor or processor-implemented components may be distributed across several geographic locations.

[0130] In this context, "processor" refers to any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (e.g., "commands," "opcodes," "machine codes," etc.) and generates corresponding output signals used to operate the machine. For example, a processor can be a CPU, RISC processor, CISC processor, GPU, DSP, ASIC, RFIC, or any combination thereof. A processor can also be a multi-core processor with two or more independent processors (sometimes called "cores") that can execute instructions simultaneously.

[0131] In this context, a "timestamp" refers to a sequence of encoded information or characters that identifies when an event occurred, such as a date and time of day, sometimes accurate to a fraction of a second.

[0132] Changes and modifications may be made to the disclosed embodiments without departing from the scope of this disclosure. These and other changes or modifications are intended to be included within the scope of this disclosure as set forth in the appended claims.

[0133] Furthermore, as can be seen in the foregoing specific embodiments, various features are combined in a single embodiment for the purpose of simplifying this disclosure. The method of this disclosure is not to be construed as reflecting an intention that the claimed embodiment requires more features than expressly listed in each claim. Rather, as reflected in the appended claims, the inventive subject matter may be presented with fewer features than in a single disclosed embodiment. Therefore, the appended claims are thus incorporated into the specific embodiments, wherein each claim exists on its own as a separate embodiment.

Claims

1. A method for trajectory and intent prediction, comprising: The user's observation trajectory and the user's user behavior information are received by one or more processors; The observation trajectory is processed by machine learning technology to generate multiple predicted trajectories. The machine learning technology is trained to establish a relationship between multiple trained observation trajectories and trained predicted trajectories. The first predicted trajectory among the multiple predicted trajectories represents the first future path that the user will follow from the observation trajectory. The second predicted trajectory among the multiple predicted trajectories represents the second future path that the user will follow from the observation trajectory. The multiple predicted trajectories are adjusted based on the user behavior information to determine the user's intention to operate the target access control device. The target access control device is determined to be within a threshold range of a given predicted trajectory among the plurality of predicted trajectories; as well as In response to determining that the target access control device is within a threshold range of a given predicted trajectory among the plurality of predicted trajectories, an operation associated with the target access control device is performed.

2. The method according to claim 1, wherein, The target access control device includes a lock associated with a door, and wherein performing the operation includes unlocking the door.

3. The method according to claim 2, further comprising: Establish a wireless communication link between the user's mobile device and the target access control device; Authorization information is exchanged via the wireless communication link; as well as The operation is performed after determining, based on the authorization information, that the user is authorized to access the target access control device.

4. The method according to claim 3, further comprising: Before performing the operation, it is determined, based on the authorization information, that the user is authorized to access the target access control device; as well as The operation is delayed after the user is determined to be authorized until the target access control device is determined to be within a threshold range of a given predicted trajectory among the plurality of predicted trajectories.

5. The method according to any one of claims 3 to 4, further comprising: Before performing the operation, it is determined, based on the authorization information, that the user is authorized to access the target access control device; as well as After determining that the user is authorized, in response to determining that the target access control device is outside a threshold range of a given predicted trajectory among the plurality of predicted trajectories, the operation is blocked.

6. The method according to any one of claims 1 to 4, wherein, The machine learning technique mentioned includes a conditional variational autoencoder.

7. The method according to claim 6, wherein, Adjusting the plurality of predicted trajectories based on the user behavior information includes processing the observed trajectory and the user behavior information by the conditional variational autoencoder to generate the plurality of predicted trajectories, wherein each of the plurality of predicted trajectories is associated with a corresponding probability indicating the likelihood that the user will travel along the corresponding predicted trajectory.

8. The method according to any one of claims 1 to 4, wherein, The machine learning technique mentioned includes variational autoencoders.

9. The method according to claim 8, wherein, Adjusting the plurality of predicted trajectories based on the user behavior information includes connecting the user behavior information to the plurality of predicted trajectories output by the variational autoencoder, wherein each of the plurality of predicted trajectories is associated with a corresponding probability indicating the likelihood that the user will travel along the corresponding predicted trajectory.

10. The method of claim 8, further comprising using a second machine learning technique to process the connected user behavior information and the plurality of predicted trajectories, the second machine learning technique being trained to establish a relationship between the plurality of trained user behavior information and the predicted intent to operate the access control device.

11. The method according to any one of claims 1 to 4, further comprising encoding the user's observation trajectory, wherein, The machine learning technique was applied to the coded observation trajectory of the user.

12. The method according to any one of claims 1 to 4, further comprising: Determine whether the received user behavior information meets the minimum parameters for user behavior information.

13. The method of claim 12, further comprising: In response to determining that the received user behavior information satisfies the minimum parameters of the user behavior information, the target access control device is allowed to perform the operation.

14. The method of claim 12, further comprising: In response to determining that the received user behavior information fails to meet the minimum parameters of the user behavior information, the target access control device is prevented from performing the operation.

15. The method according to claim 12, wherein, The minimum parameter includes a threshold amount for a specified type of user behavior information.

16. The method according to any one of claims 1 to 4, further comprising generating the user behavior information by encoding a feature vector, the feature vector comprising at least one of the following: Monitor the user's body movements; Monitor the user's stride; Identify the time and location when the user operates different types of access control devices; When a given access control device is being operated by the user, identify other client devices and other types of access control devices within the user's scope; or Identify other users who are typically on the user's social network.

17. The method according to any one of claims 1 to 4, wherein, The machine learning technique includes a first machine learning technique, and the method further includes: The user behavior information is generated using a second machine learning technique, which is trained to establish a relationship between the trained user behavior information and the predicted user behavior information; and The user intent to operate the target access control device is generated by a third machine learning technique, which is trained to establish a relationship between trained user behavior information connected to a set of trajectories and the predicted user intent to operate the access control device.

18. The method according to claim 17, wherein, Each of the first, second, and third machine learning techniques is trained end-to-end.

19. A system for trajectory and intent prediction, comprising: One or more processors coupled to memory, the memory including non-transitory computer instructions that, when executed by the one or more processors, perform operations including: Receive the user's observation trajectory and the user's user behavior information; The observation trajectory is processed by machine learning technology to generate multiple predicted trajectories. The machine learning technology is trained to establish a relationship between multiple trained observation trajectories and trained predicted trajectories. The first predicted trajectory among the multiple predicted trajectories represents the first future path that the user will follow from the observation trajectory. The second predicted trajectory among the multiple predicted trajectories represents the second future path that the user will follow from the observation trajectory. The multiple predicted trajectories are adjusted based on the user behavior information to determine the user's intention to operate the target access control device. Determine the threshold range of the target access control device for a given predicted trajectory among the plurality of predicted trajectories; and In response to determining that the target access control device is within a threshold range of a given predicted trajectory among the plurality of predicted trajectories, an operation associated with the target access control device is performed.

20. A non-transitory computer-readable medium, the non-transitory computer-readable medium comprising non-transitory computer-readable instructions for performing operations, the operations comprising: Receive the user's observation trajectory and the user's user behavior information; The observation trajectory is processed by machine learning technology to generate multiple predicted trajectories. The machine learning technology is trained to establish a relationship between multiple trained observation trajectories and trained predicted trajectories. The first predicted trajectory among the multiple predicted trajectories represents the first future path that the user will follow from the observation trajectory. The second predicted trajectory among the multiple predicted trajectories represents the second future path that the user will follow from the observation trajectory. The multiple predicted trajectories are adjusted based on the user behavior information to determine the user's intention to operate the target access control device. The target access control device is determined to be within a threshold range of a given predicted trajectory among the plurality of predicted trajectories; as well as In response to determining that the target access control device is within a threshold range of a given predicted trajectory among the plurality of predicted trajectories, an operation associated with the target access control device is performed.

21. A method for trajectory and intent prediction, comprising: The user receives the user's observation trajectory; The observation trajectory is processed by machine learning technology to generate multiple predicted trajectories. The machine learning technology is trained to establish a relationship between multiple trained observation trajectories and trained predicted trajectories. The first predicted trajectory among the multiple predicted trajectories represents the first future path that the user will follow from the observation trajectory. The second predicted trajectory among the multiple predicted trajectories represents the second future path that the user will follow from the observation trajectory. Determine if user behavior information is available; In response to determining that the user behavior information is unavailable, the target access control device is determined to be within a threshold range of a given predicted trajectory among the plurality of predicted trajectories generated solely based on the machine learning technique; In response to determining that the user behavior information is available, The multiple predicted trajectories are adjusted based on the user behavior information to determine the user's intention to operate the target access control device. Determine the target access control device within a threshold range of a given predicted trajectory among a plurality of adjusted predicted trajectories; as well as In response to determining that the target access control device is within the threshold range, an operation associated with the target access control device is performed.

22. The method according to claim 21, wherein, The user behavior information is output by a classifier that receives multiple user behaviors as input and outputs the probability of an intent to access a given access control device.

23. The method according to any one of claims 21 to 22, wherein, The user behavior information is output by a classifier that receives multiple access control devices as input and outputs the probability of an intention to access a given access control device among the multiple access control devices.

24. The method according to any one of claims 21 to 22, wherein, Adjusting the multiple prediction trajectories includes: calculating a weighted average of the predictions of a user's intent to access a given access control device provided by a user behavior classifier model and the predictions of a user's intent to access the given access control device provided by another machine learning technique, based on the multiple prediction trajectories.

Images