Modeling of preprogrammed scenario data for gesture-based entry control systems

Abstract

A method of training a gesture-based entry control system includes repeatedly performing, by a user of a mobile device, an intentional gesture indicative of an intentional gesture type. The intentional gesture represents an intent of the user to gain entry. A model of the selected intentional gesture type is then formed from the repeated performance of the intentional gesture. The model is configured to be employed by a software-based application to confirm future performance of the intentional gesture.

Description

Technical Field

[0001] This disclosure relates to entry control systems, and more specifically, to modeling pre-programmed scenario data for gesture-based entry control systems. Background Technology

[0002] Access control systems are used in a variety of applications, including structures, buildings, and / or components, such as safes, subway turnstiles, child-safe storage containers, and many others. In non-restricted examples of buildings, many such structures must be secure, meaning the identity and number of people entering and leaving the building at any given time must be known. One known way to achieve this is to distribute identification cards to all individuals needing access. Each person is then required to swipe their identification card at a reader located near any entrance. In one example, the reader can identify the identification card via a magnetic stripe. Another example is using RFID to read the identification card. A disadvantage of such a process is that each person must swipe their identification card individually, for example, before being allowed entry. This task can be time-consuming.

[0003] Many current access control systems use smartphones instead of identification cards. The key technology behind this smartphone use is Near Field Communication (NFC), which allows for short-range communication. For this application, both the smartphone and the local access control reader must have NFC hardware. Other options may include a human-machine interface (HID) device for the reader, capable of detecting, in a controlled manner, such as the twisting of the smartphone in front of the reader to indicate intent. However, both the smartphone and the reader must be able to detect intent independently. Furthermore, current methods still require the user to remove the smartphone and perform a specific action using it. Such removal and / or action can be frustrating and time-consuming for the user.

[0004] Improvements are needed to the access system to further optimize ease of operation with or without reducing the number of components. Summary of the Invention

[0005] A method for training a gesture-based access control system according to a non-limiting embodiment of the present disclosure includes: repeatedly executing a gesture indicating an intentional gesture type by a user of a mobile device, wherein the intentional gesture represents the user's intention to gain access; and forming a model of a selected intentional gesture type from the repeated execution of the intentional gesture by a software-based application of the gesture-based access control system, wherein the model is adopted by the software-based application to confirm future execution of the intentional gesture.

[0006] In addition to the aforementioned embodiments, the method includes selecting an intentional pose type from a pre-programmed library of multiple intentional pose types before repeated execution.

[0007] As an alternative or supplement, in the foregoing embodiments, the method further includes employing at least one machine learning algorithm to analyze data collected from repeated executions of intentional poses.

[0008] As an alternative or supplement, in the foregoing embodiments, the method includes employing at least one machine learning algorithm to analyze data collected from repeated executions of intentional poses.

[0009] As an alternative or supplement, in the foregoing embodiments, data is collected from at least one detection system of the mobile device.

[0010] As an alternative or supplement, in the foregoing embodiments, data is collected from at least one detection system of the mobile device.

[0011] As an alternative or supplement, in the foregoing embodiments, the data includes motion data detected by the inertial measurement unit (IMU) sensing system of at least one detection system.

[0012] As an alternative or supplement, in the foregoing embodiments, the data includes audible data detected by the microphone of the mobile device.

[0013] As an alternative or supplement, in the foregoing embodiments, the intentional gesture is for the user to tap on the mobile device.

[0014] As an alternative or supplement, in the foregoing embodiments, the software-based application prompts the user to select the desired gesture type via the human-machine interface (HID) device of the mobile device.

[0015] As an alternative or supplement, in the foregoing embodiments, the software-based application notifies the user when the model is completed based on confidence and via the human-machine interface (HID) device of the mobile device.

[0016] As an alternative or supplement, in the foregoing embodiments, the pre-programmed library includes at least one of user tapping on a mobile device, user knocking on a door, and user voice commands.

[0017] As an alternative or supplement, in the foregoing embodiments, the software-based application is configured to perform in an intentional manner to confirm the user's intent and authenticate the user.

[0018] As an alternative or supplement, in the foregoing embodiments, the method further includes authenticating the user by analyzing the frequency of intentional gestures through software-based application analysis, wherein the intentional gesture is one of tapping and striking.

[0019] Unless otherwise expressly stated, the foregoing features and elements can be combined in various, not exclusively, combinations. These features and elements, and their operation, will become more apparent from the following description and accompanying drawings. However, it should be understood that the following description and drawings are exemplary in nature and not limiting. Attached Figure Description

[0020] Various features will become apparent to those skilled in the art from the following detailed description of the disclosed non-limiting embodiments. The accompanying drawings, accompanying the detailed description, can be briefly described as follows:

[0021] Figure 1 This is a schematic diagram of a deviceless gesture applied to a door access control system;

[0022] Figure 2 This is another schematic diagram of entering the control system;

[0023] Figure 3 This is a flowchart illustrating the method for accessing the control system.

[0024] Figure 4 It is a flowchart of a method for determining the movement, location, and position of mobile devices entering a control system;

[0025] Figure 5 This is a schematic diagram of another embodiment of the entry control system for the application device posture;

[0026] Figure 6 This is a schematic diagram of the first example of equipment posture;

[0027] Figure 7 This is a schematic diagram of a second example of device posture;

[0028] Figure 8 This is a schematic diagram of the third example of equipment posture;

[0029] Figure 9 This is a schematic diagram of the fourth example of the device posture;

[0030] Figure 10 This is a schematic diagram of a first type of container for a user carrying a mobile device that includes access to the control system;

[0031] Figure 11 Is with Figure 10 A schematic diagram of the relevant entry control system and the execution of the first deviceless posture;

[0032] Figure 12 Is with Figure 10 A schematic diagram of the relevant entry control system and the execution of the second deviceless posture;

[0033] Figure 13 This is a schematic diagram of a user carrying a second type of container containing a mobile device that enters the control system and performing a first container posture;

[0034] Figure 14This is a schematic diagram of a user carrying a second type of container containing a mobile device that enters the control system and performing a second container posture;

[0035] Figure 15 This is a schematic diagram of a user carrying a second type of container containing a mobile device that enters the control system and performing a third container posture;

[0036] Figure 16 This is a user schematic diagram, illustrating various locations, positions, and uses of the mobile device 26 in relation to the adaptive intent pattern detection features of the gesture-based entry control system;

[0037] Figure 17 This is a schematic diagram illustrating a pose-based entry control system with adaptive intent pattern detection features;

[0038] Figure 18 This is a flowchart illustrating the sequence of inherent postures of a seamless entry control system as an embodiment of a posture-based entry control system;

[0039] Figure 19 This is a schematic diagram illustrating a cloud-based embodiment of a gesture-based entry control system;

[0040] Figure 20 This is a schematic diagram of another embodiment of a gesture-based entry control system, which is a tapping gesture entry control system.

[0041] Figure 21 This is a perspective view of mobile device 26;

[0042] Figure 22 This is a flowchart of a method for operating a posture-based entry control system, which is another embodiment of a posture-based entry control system.

[0043] Figure 23 This is a flowchart of a method for training a posture-based entry control system; and

[0044] Figure 24 It is a graph showing a user-specific model as part of pre-programmed scenario data for a software-based application of a gesture-based entry control system. Detailed Implementation

[0045] refer to Figure 1A gesture-based access control system 20 is illustrated in a non-limiting application of door 22, which provides an entrance for a user to enter or exit a building, structure, room, etc. In this embodiment, the access control system 20 is adapted to unlock the door when a user 23 (e.g., a person) intending to enter makes a detected intentional gesture. Although this application is applied to door 22, it is conceivable and understood that the access control system 20 can also be applied to anything requiring access control, including, for example, computers, subway turnstiles, safes, child safety storage compartments, etc. As will become more apparent, in some embodiments, the intentional gesture can be a deviceless gesture (see [link to relevant documentation]). Figure 1 Arrow 25 in the figure), or in other embodiments, it may be the device orientation (see [reference]). Figure 6 (Arrow 94 in the image).

[0046] refer to Figure 1 and 2 In one embodiment, the access control system 20 includes a lock or access component 24, a mobile device 26 carried by a user 23, and a wireless interface 28. The mobile device 26 is adapted to communicate wirelessly with the lock component 24 via the wireless interface 28. The lock component 24 may include a latch 30 (e.g., a locking bolt), an actuator 32, a controller 34, and a receiver 36, which may be a transceiver with bidirectional communication capabilities and includes an antenna. The receiver 36 is configured to receive wireless access or command signals (see arrow 38) from the mobile device 26 via the wireless interface 28. The access signal 38 is sent to the controller 34. The controller 34 may process the signal 38 and, based on the signal, activate the actuator 32 to move the latch 30 from a non-access state to an access state (i.e., a locked position and an unlocked position). In one embodiment, the access component 24 is an access reader (e.g., an RFID reader). Examples of the signal 38 may be Bluetooth, Wi-Fi, or other short-range communication signals. The access component 24 may be a local access component 24 and is typically located near a door or other component, and the component 24 is adapted to control access to the door or other component.

[0047] The controller 34 can be any combination of one or more of a central processing unit (CPU), a multiprocessor, a microcontroller unit (MCU), a digital signal processing unit (DSP), an application-specific integrated circuit (ASIC), and other devices capable of executing software instructions or otherwise controllable to operate according to predetermined logic. In one example, the driver 32 is a motor with a relay operated by the controller. In another example, the driver 32 is an electromagnetic driver. The wireless interface 28 is any current or future wireless interface that allows communication between the mobile device 26 and the locking assembly 24. Non-limiting examples of the wireless interface 28 include Bluetooth, Bluetooth Low Energy (BLE), radio frequency identification (RFID), near field communication (NFC), any of the standards in the IEEE 802.11 standard, and others.

[0048] In one embodiment, the mobile device 26 includes a transmitter 40, a controller 42, and at least one detection system (i.e., three shown as 46, 48, and 50), whereby the transmitter 40 may be a transceiver with an antenna. The at least one detection system may include an inertial measurement unit (IMU) sensor system 46, an environmental detection system 48, an internal activity (i.e., usage) notification module 50, and other modules for generally determining the movement, position, location, and use of the mobile device 26 relative to the user 23. Non-limiting examples of the mobile device 26 include smartphones, mobile phones, keychains, watches (i.e., smartwatches), and other similar devices typically carried by the user 23.

[0049] The controller 42 of the mobile device 26 includes a processor 56 and a storage medium 58. Optionally, the processor 56 is any combination of one or more of a central processing unit (CPU), a multiprocessor, a microcontroller unit (MCU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), and other devices capable of executing software instructions or otherwise controllable to operate according to predetermined logic. The storage medium 58 is optionally any combination of read-and-write memory (RAM) and read-only memory (ROM). The storage medium 58 may also include permanent memory, which may be any one or a combination of solid-state memory, magnetic memory, or optical memory storing a computer program (i.e., an application) with software instructions.

[0050] In one embodiment, and similar to the controller 42 of the mobile device 26, the controller 34 of the lock assembly 24 may include a processor 70 and a storage medium 72. Optionally, the processor 70 is any combination of one or more of a central processing unit (CPU), a multiprocessor, a microcontroller unit (MCU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), and other devices capable of executing software instructions or otherwise controllable to operate according to predetermined logic. The storage medium 72 is optionally any combination of read-and-write memory (RAM) and read-only memory (ROM). The storage medium 72 may also include permanent memory, which may be any one or a combination of solid-state memory, magnetic memory, or optical memory storing a computer program (i.e., an application) with software instructions. It is contemplated and understood that in one embodiment, the controller 42 may not include the storage medium 72 and may only include control circuitry capable of receiving a signal 38 from the mobile device 26 as a command signal to initiate the actuation of the lock assembly 24.

[0051] The gesture-based entry control system 20 may also include an application 60. In one embodiment, the application 60 is software-based and at least partially stored in storage medium 58 for retrieval and execution by the processor 56 of the controller 42. The application 60 may include a database of computer instructions 62 and pre-programmed data. For example, the pre-programmed data includes credential data 64 and scene data 66. In one embodiment, scene data 66 indicates a “composite” movement of the user 23 that does not necessarily include a gesture but depends on (i.e., according to) the carrying position of the mobile device 26 on the user 23.

[0052] In another embodiment, application 60 may be stored at least partially in at least one storage medium contained in the cloud (i.e., a remote server) and executed at least partially by at least one processor of the cloud.

[0053] For clarity, the term "intentional gesture" as used here refers to an action (e.g., physical movement) performed by user 23 to gain entry. In one example, the gained entry can be achieved through door 22 (see...). Figure 1 However, it can also be incorporated into any physical structure and / or electronic system (e.g., a computer). For the purposes of this disclosure, examples of intentional poses may include device-free poses, device poses, and inherent poses.

[0054] The term "device-free posture" refers to an intentional posture that is not physically present (see [reference]). Figure 1 (e.g., gesture 25). For example, if user 23 makes a deviceless gesture 25 as a wave of the right hand 74, then the mobile device 26 is not in the right hand 74, but can be located anywhere else on user 23's body. Conversely, the term "device gesture" (see [reference needed]). Figure 6 The gesture 94 in the example indicates that the mobile device 23 itself is being used as part of an intentional gesture. In this example, device gesture 94 would include waving of the mobile device 26. More specifically, and consistent with this example, the mobile device 26 would be in the waving right hand 74 (see [reference]). Figure 5 and 6 Finally, the term "inherent posture" (see...) Figure 18 The gesture 341 in the text is a gesture applied as part of a seamless access control system. That is, a typical action such as opening a door (or a typical movement made in preparation for opening a door) is a gesture. An inherent gesture is "intentional" in the sense that the user 23 intends to gain access. A specific example of an inherent gesture could be reaching for or pulling a door handle.

[0055] Determine the movement, location, and position of the mobile device relative to the user:

[0056] It is necessary to determine the motion (i.e., compound motion) of the mobile device 26 in order to identify the intentional gestures made by the user 23 by distinguishing one or more simultaneous motions made by the user. Determining the position and / or location of the mobile device 26 relative to the user 23 can help distinguish the multiple motions made by the user 23 from the measured compound motion of the mobile device 26. As a supplement or alternative, when two access components 24 are present, it may be advantageous to determine the location of the mobile device 26 relative to the user 23 if the corresponding doors 22 are closely positioned together. In this case, knowing the location of the mobile device 26 will prevent or reduce the chance of the user 23 gaining access through the wrong door via an intentional gesture without the device.

[0057] The inertial measurement unit (IMU) sensor system 46 may include one or more of an accelerometer 80, a gyroscope 82, and other means adapted to detect acceleration and thus movement in at least one dimension and optionally in three dimensions. The environmental detection system 48 may include one or more of a vision camera 84 (i.e., a computer vision system), a temperature sensor 86, a light sensor 88, and a proximity sensor 90, adapted to at least increase the level of confidence when distinguishing complex motions to determine whether the user 23 is making an unintentional gesture.

[0058] The internal activity notification module 50 can also help optimize the confidence level and can be part of the application 60 or a separate computer software instruction. For example, the activity notification module 50 can notify the application 60 that user 23 is sending a text message or making a phone call via mobile device 26. When distinguishing compound motions, the application 60 can then attribute a portion of the motion to, for example, the texting activity. In one embodiment, and depending on how the application 60 processes the information data, the visual camera 84 can be part of the IMU sensor system 46 (i.e., taking multiple photos to determine motion), and / or can be part of the internal activity notification module 50 (i.e., user 23 is experiencing an activity of taking photos for fun).

[0059] In one embodiment, the vision camera 84 is adapted to detect movement by capturing images of the surrounding environment and analyzing differences in the images over time. The temperature sensor 86 is adapted to measure temperature. In one embodiment, the temperature data at least partially indicates the user 23's body temperature. For example, if the mobile device 26 is located in the back pocket 56 of the clothing worn by the user 23 (see...). Figure 1 In this context, temperature data can be correlated with a temperature higher than that of the mobile device 26 located in the wallet or backpack carried by the user 23. The proximity sensor 90 is adapted to determine the proximity of the mobile device 26 to the user 23. For example, the mobile device 26 may be placed on a table, in a back pocket 56, in a wallet, or in a backpack. The proximity sensor 90 can also be used to determine whether a large portion of the user 23 is located between the sensor 90 and the access component 24, which may cause some attenuation of the signal between the component 24 and the mobile device 26.

[0060] The light sensor 88 is adapted to measure the light level near the mobile device 26. The light data sent from the light sensor 88 to the processor 42 can indicate the location of the mobile device 26 when the user 23 makes a gesture. For example, the mobile device 26 may be in the back pocket 56 of the clothing worn by the user 23.

[0061] In operation, the IMU sensor system 46 enables the recognition of gesture-based intentions, and the environment detection system 48 and optionally the activity notification module 50 are used to improve the reliability of intentional gesture recognition. In one example, this is achieved by application 60 fusing information obtained from systems 46, 48, and module 50, and using machine learning algorithms and / or pre-programmed scene data 66. (Reference) Figure 4 The method for determining the location and / or position of the mobile device 26 relative to the user 23 includes, at frame 200, when the activity of the motion device 26 is in standby mode or otherwise blocked.

[0062] In block 202, the IMU sensor system 46 detects periodic movement (i.e., compound motion) and sends the information to the controller 42. In block 204, the application 60 determines, via at least one algorithm and at least a portion of pre-programmed scene data 66, that at least a portion of the compound motion is a characteristic of walking. In block 206, the temperature sensor 86 and / or light sensor 88 of the environmental detection system 48 send information (i.e., confirmation parameter data) to the controller 42 used by the application 60 to determine that the mobile device 26 is, for example, in a back pocket or backpack (i.e., the light sensor 88 detects a dark environment). Furthermore, the IMU sensor system 46 can also help detect the relative position of the mobile device 26. For example, the angle of the mobile device 26 relative to the ground or floor surface can indicate the position of the front pocket relative to the back pocket, etc. In block 208, the activity notification module 50 can provide the application 60 with information indicating that the user 23 is currently using (e.g., sending a text message) the mobile device 26. This current use can provide the possible position of the mobile device 23 (i.e., vertical, horizontal, or somewhere in between) and / or an indication of the movement of the mobile device as part of a compound motion, which can ultimately be distinguished from an intentional posture. To complete boxes 206 and 208, application 60 may employ algorithms and / or pre-programmed scenario data 66.

[0063] Training based on software applications :

[0064] refer to Figure 2 Furthermore, during operation, application 60 may include a human-machine interface device (HID) 91 via mobile device 26 (see...). Figure 2 Training instructions (i.e., setup or calibration instructions) are transmitted to user 23. These training instructions may be given when user 23 carries mobile device 26 in various locations (e.g., back pocket, front pocket, left hand when right hand is in a gesture, etc.) or in various ways (e.g., backpack, wallet, etc.) and / or while performing certain activities with mobile device 26 (e.g., texting, making calls, etc.). While user 23 performs various movements and / or routines, application 60 may construct scene data 66 using information received from at least one of the IMU sensor system 46, environmental detection system 48, and internal activity notification module 50, and thereby pre-program it.

[0065] For example, application 60 can instruct user 23 to walk with mobile device 26 in back pocket 56. Then, at least one of systems 46, 48, and module 50 detects motion and other parameters and pre-programs the resulting information as part of scene data 66. As part of another event, application 60 can then instruct user 23 to perform the same walking motion with mobile device 26 in the same position, but simultaneously performing a selected gesture designed to trigger access component 24 (i.e., unlock). Again, the resulting motion detected by one or more of systems 46, 48, and module 50 is recorded as part of scene data 66. Similar instructions can be given to user 23 to reposition mobile device 26 on his or her and to perform various movements with and without a gesture. Once the instructions are trained, scene data 66 can typically resemble a matrix or array of data.

[0066] In one embodiment, application 60 may include machine learning techniques and / or algorithms (e.g., deep learning). Machine learning algorithms can increasingly train gesture recognition for specific interactions of a given user. Furthermore, through some form of "continuous" training, application 60 has the ability to adapt to changing user habits (i.e., potentially caused by impairment) over a period of time.

[0067] In one example, application 60 may include one or more machine learning algorithms configured to determine or confirm user intent based on explicit intent signals generated by one or more of detection systems 46, 48, 50, and to determine user authentication (i.e., that mobile device 26 actually belongs to user 23) by matching the intent signals with user-specific, predefined patterns. User intent and user authentication can be inferred from IMU signals, audio signals, RSSI (e.g., Bluetooth), and other data from, for example, wearable mobile device 26. In another embodiment, while user intent can be confirmed by the number of taps or the pattern of taps, user authorization can be confirmed by the intensity of the taps, the delay between taps, and / or the change in intensity from one tap to the next.

[0068] refer to Figure 23In one embodiment, application 60 may include a training operation mode. At box 500, and via HID 91, user 23 can select a training mode. In this mode, and at box 502, user 23 is prompted by application 60 via HID 91 and can select an intentional pose type from a library of supported pose types as part of scene data 66. At box 504, application 60 prompts user 23, and user 23 can perform repetition of the selected pose type for the intention. At box 506, a machine learning algorithm collects and analyzes data from the repetition of the selected pose type to build a user-specific model associated with the selected pose type, as part of scene data 66. At box 508, the machine learning algorithm determines that the user-specific model has sufficiently high quality and confidence, and application 60 notifies user 91 via HID 91 that the model is complete. Non-limiting examples of pose types may include tapping a fixed number of times by user 23 on mobile device 26 (i.e., a predefined pattern, see...). Figure 20 ), knocking 22, user-specific voice commands issued into the microphone 130 of the mobile device 26 (see Figure 2 (and other types of poses).

[0069] After training in operational mode, application 60 can enter deployment mode. In this mode, statistical machine learning techniques are deployed via algorithms, which can then be run on the cloud (i.e., remote servers, see [link]). Figure 19 The application 60 is located within and supported by Cloud 360. In this example, at least a portion of the application 60 can reside in Cloud 360, and the cloud is used to build user-specific models. In one embodiment, the user-specific model can be improved over time using machine learning algorithms. Thus, over time, a particular user 23 becomes easier to identify. At box 510, the user 23 can then perform a list of pre-trained gestures (i.e., pre-programmed into the application 60) to signal intents and authenticate them.

[0070] More specifically, in the training operation mode, data reflecting specific actions forced upon user 23 for training purposes is collected. This can be considered as defining the fundamental facts of the "correct way" to perform the actions. Optionally, application 60 may also collect data on how not to perform specific actions to further enhance learning.

[0071] Once the training pattern is complete and data is collected, the data is used to train the algorithm to extract relevant information / features that detect whether a specific action or gesture was performed correctly. The result is the trained model that is subsequently deployed (i.e., the user-specific model).

[0072] refer to Figure 24The diagram 118, consisting of three sections 118A, 118B, and 118C, is shown as an example of a modeling process where the gesture type can be a tap on the mobile device 26. The X-axis of each section 118A, 118B, and 118C is over a common duration. Section 118A shows the raw accelerometer data caused by movement of the mobile device 26 during the tap. Section 118B shows the corresponding audio data. Section 118C also shows the extracted features of the tap confirmation, highlighted with a star symbol. The spike pattern and the time interval between spikes are unique to the user 23 and can be used as authentication (i.e., a code).

[0073] The completion of training and deployment results in a user-specific detection model that uses observed signals from one or more of detection systems 46, 48, and 50 for both gesture confirmation and user authentication. The model also provides a confidence level for user authentication, which can be improved with further use. This confidence level can be used to allow or deny entry to, for example, building areas.

[0074] Distinguishing individual user movements from composite motion measured from mobile devices:

[0075] In one embodiment, application 60 may rely on the observation that deviceless gestures (e.g., waving) generate minute, periodic movements of the human body (i.e., part of a complex motion) that can be captured by the IMU sensor system 46, environmental detection system 48, and / or internal activity notification module 50 of mobile device 26. A machine learning algorithm is trained to distinguish the minute movements associated with the gesture from other, more obvious body movements that may be observed during walking or talking.

[0076] Optionally, the controller 42 of the mobile device 26 can receive data from the optical system 54. In one example, the optical data can be applied to determine whether the mobile device 26 is carried in a hand or in a pocket, backpack, or wallet. The temperature sensor 86 of the environmental detection system 48 can output temperature data to the controller 42 to determine, for example, whether the mobile device 26 is in a hand or pocket, and not in a backpack or wallet. When the application 60 compares or attempts to match composite motion with pre-programmed scene data 66, temperature and / or optical data can be applied as additional data to the composite motion to improve the level of confidence in the match.

[0077] In one embodiment, the selected deviceless intentional posture can be the hand 74 without the mobile device 26 (see...). Figure 1The user 23 does not need to remove his / her mobile device 26 to perform any device functions or input. The user 23 only needs to perform the correct intentional gesture to gain access, for example, through door 22. Other examples of intentional gestures may include left-to-right movements of the arm, top-to-bottom movements of the hand 74, head and / or shoulder movements, or any other unique movements.

[0078] In one embodiment, the intentional gesture can be a secret gesture, thus eliminating the need for further authentication between the mobile device 26 and the access component 24. In this example, the access component 24 can be relatively simple and does not require pre-programming.

[0079] In another embodiment, the access component 24 may be pre-programmed to accept only command signal 38, which carries or is accompanied by an authentication code typically pre-programmed into the two controllers 34, 42. Therefore, controller 34 is able to match the authentication code (i.e., a portion of signal 38) received from mobile device 26 with code 76 pre-programmed into storage medium 72.

[0080] refer to Figure 2 and 3 And during normal operation of the attitude entry control system 20; at box 100, the controller 34 of entry component 24 can broadcast a beacon signal via transceiver 36 (see Figure 2 (See arrow 78 in the image). In one example, the beacon signal 78 may be encoded as part of the authentication process between the mobile device 26 and the access component 24. In one example, the broadcast beacon signal 78 may be a Bluetooth radio type. In other examples, the signal 78 may be Wi-Fi / cell radio or may be audible spectrum. It is also conceivable and understood that, while maintaining the novelty of the approach process, other methods known to those skilled in the art for authenticating the mobile device 26 using the access component 24 may be applied.

[0081] At block 102, the transceiver 40 of mobile device 26 can receive beacon signal 78 when it is generally within a specified distance. Once received, at block 104, mobile device 26 typically launches application 60. In another embodiment, application 60 may not need to be launched by the beacon signal. Therefore, in some applications, entry component 24 may not be suitable for broadcasting beacon signals.

[0082] At box 106, when in the general vicinity of entry component 24 and / or when application 60 is active, application 60 may receive and process composite motion data from the IMU sensor system 46 of mobile device 26 to determine the activity of user 23 (i.e., walking, talking, standing still, etc.) and other influencing data or information from environmental detection system 48 and / or internal activity notification module 50 to determine influencing parameters such as the location, position, and / or use of mobile device. At box 108, application 60 matches the composite motion data and influencing parameter data with pre-programmed scene data 66 at a predetermined confidence level to determine whether user 23 is performing an intentional gesture indicating entry intent (e.g., an intentional gesture without device).

[0083] At box 110, and in one example, user 23 can walk with mobile device 26 in a back pocket while simultaneously performing a deviceless intentional gesture with right hand 74. At box 112, application 60 determines where mobile device 26 is located on user 23, determines that user 23 is walking, and determines that a deviceless intentional gesture is being performed by comparing composite motion and other influencing parameter data (e.g., light, temperature, etc.) with scene data 66. At box 114, and after controller 42 of mobile device 26 recognizes the deviceless intentional gesture, mobile device 26 broadcasts command signal 38 to entry component 24. At box 116, entry component 24 is actuated from a non-entry state to an entry state, whereby door 22 can be opened by user 23.

[0084] In one embodiment, the prerequisite may be that the user 23 does not walk before the mobile device 26 can recognize or accept the gesture. In this embodiment, the accelerometer system and / or gyroscope system of the mobile device 26 may be used to confirm that the user 23 is generally still except for the movement of the gesture itself.

[0085] Intentional postures are detected and / or confirmed via RSSI:

[0086] Refer again Figure 2 The beacon signal 78 broadcast by the entry component 24 via transceiver 36 can be received by the controller 42 via transceiver 40 and is typically used as a Received Signal Strength Indicator (RSSI). More specifically and as an optional embodiment, the gesture-based entry control system 20 may also include an RSSI module 92, which may be software-based and part of application 60. In other embodiments, the RSSI module 92 may be implemented by a separate sensor system of the mobile device 26, which may include both software and hardware.

[0087] In operation, in addition to the additional features provided by RSSI module 92, the attitude-based entry control system 20 can be configured as shown in boxes 100-116 (see...). Figure 3It is performed as described in [the document]. More specifically, the beacon signal 78 received by the mobile device 26 at block 102 is also processed by the RSSI module 92, which is configured to detect periodic changes in the signal strength indicating an intentional gesture of crossing (i.e., approaching the entry component 24 and repeatedly crossing in front of it) via signal 78. In one example, it could be the user 23's arm crossing back and forth in front of the entry component 26. In another embodiment, the placement of the user 23's hand on the entry component 24 can also implement RSSI.

[0088] As described in box 110 above, scene data 66 may also include pre-programmed RSSI data, which indicates periodic changes in expected signal strength detected when performing a deviceless gesture. RSSI module 92 can compare the measured periodic changes in signal strength with the pre-programmed RSSI data to further confirm or improve the confidence level of the occurrence of the deviceless gesture.

[0089] In another embodiment, scene data 66 may include only pre-programmed RSSI data. In this embodiment, application 60 determines the execution of a deviceless pose based solely on pre-programmed RSSI data. Therefore, the IMU sensor system 46 may not be required.

[0090] Mobile devices housed in containers carried by the user:

[0091] As previously mentioned, the mobile device 26 may be located close to the intentional posture being performed (i.e., the deviceless posture 25). For example, the mobile device 26 may typically rest against the user 23's body (e.g., in a back pocket) rather than being carried in the hand 74 performing the deviceless posture (see [link to previous text]). Figure 1 ).

[0092] refer to Figure 10 and 11 User 23 can perform gestures 25 normally without a device, but mobile device 26 is located in a container 95 carried by the user. Non-limiting examples of container 95 include a handbag (see...). Figure 10-12 ), backpack (see Figure 13-15 ) and other containers suitable for storing and / or carrying personal items (including mobile devices 26) for user 23.

[0093] In one embodiment, the container 95 is adapted to be carried by a specific body part of the user 23. For example, a handbag is carried by the user 23's hand 74, and a backpack is carried by the user 23's back or torso 96. For high-confidence detection of the deviceless posture 25, the container 95 is carried by a body part performing the deviceless posture 25 (i.e., an intentional body posture). For example, if the container 95 is a handbag or wallet, the hand 74 holding the handbag can perform the deviceless posture 25, thus holding the handbag with the hand in the correct posture.

[0094] The motion of the mobile device 26 is typically measured using at least the IMU sensor system 46 as previously described. In one case, the measured motion of the mobile device 26 may be a complex motion dynamically created by the user 23 walking when the user performs an intentional body posture 25 (i.e., a device-free posture). In this case, the walking motion may be achieved by the user 23 swinging their arms and hands 74 in the forward and backward directions (i.e., a regular body movement, see...). Figure 10 Arrow 97 in the middle). The swinging hand 74 carries the handbag 95, thereby causing the mobile device to undergo associated conventional containment movement (see arrow 97 in the middle). Figure 10 (Arrow 98 in the image).

[0095] refer to Figure 11 And in a continuation of the example of the hold 95 of the handbag, the intentional body posture 25 may be a twisting of the wrist associated with the hand 74 of the user 23 holding the handbag 95. The intentional body posture 25 creates an associated hold posture (see arrow 99). In one embodiment, the hold posture 99 may be an amplification of the intentional body posture 25. In other embodiments, posture 99 may be substantially the same as posture 25, or may be different but desired to be the same.

[0096] Therefore, the measured motion of the mobile device 26 is a composite motion, which includes a container posture 99 directly associated with the intentional body posture 25 and a conventional container movement 98 associated with the conventional body movement 97. Thus, the composite motion represents the conventional body movement 97 and the intentional body posture 25 multiplied by a parameter factor. The parameter factor may represent the type of the container 95 (i.e., a backpack or handbag) and the position and location of the mobile device 26 relative to the user 23 and the container 95. The parameter factor may be part of the scene data 66, and the environmental detection system 48 can help determine the position and location of the mobile device 26 and the type of container 95.

[0097] In one embodiment, an intentional body posture 25 causes the associated receiver posture 99 to be opposite to the conventional receiver movement 98. For example, the orientation of posture 99 is laterally or perpendicular to the direction of movement 98. This will contribute to a higher level of confidence by applying 60 improved motion discrimination.

[0098] refer to Figure 12 Another example of the receiving posture 99 is shown, in which the handbag is being rocked vertically. In this example, the intentional body posture could be the repeated raising and lowering of the hand 74.

[0099] refer to Figure 13-15 Another example of the housing 95 is shown as a backpack worn on the back or torso 101 of the user 23. Figure 13 In this context, the orientation 99 of the receiving component can be caused by the twisting of the torso 101 (i.e., intentional body posture 25). Figure 14 In the middle, the orientation of the receiving part 99 can be caused by the bending of the user's waist 23. Figure 15 In the middle, the position 99 of the receiving part can be caused by the left and right bending of the user's torso 101 or waist.

[0100] Equipment posture for testing:

[0101] As previously mentioned, determining the occurrence of a deviceless posture can be achieved by analyzing the combined motions of the measured mobile device 26 and other influencing parameters. For example, if the mobile device 26 is in the back pocket 56 and the right hand 74 is performing a deviceless posture, then analyzing the combined motions experienced by the mobile device 26 serves as an indirect indication of the occurrence of the deviceless posture.

[0102] refer to Figure 2 and Figure 5 In another embodiment, the mobile device 26 can be used to perform a gesture (i.e., a device gesture). In this example, the device gesture is typically measured directly as the motion of the mobile device 26. However, it remains to be understood that the motion measured by the mobile device 26 can still be a complex motion.

[0103] For example, device orientation (see Figure 6 Arrow 94 in the diagram can typically represent a roughly horizontal wave of the mobile device 26. If the user 23 remains completely still except when performing the device gesture 94, the mobile device 26 can directly measure the device gesture 94 without requiring motion differentiation of the composite motion. However, if the user 23 is walking while performing the device gesture 94, the walking motion will also be measured using the device gesture 94, resulting in a measured composite motion. That is, the walking motion generates noise that may interfere with a reliable interpretation of the intention to enter.

[0104] As previously mentioned, the compound motion in this example can be analyzed using appropriate scene data 66 established under the presupposed condition that the intentional posture is device posture 94. Other non-limiting examples of device posture 94 may include waving the mobile device 26 in a substantially vertical direction in front of the entry component 24 (i.e., simulating swiping a virtual entry card, see...). Figure 7), towards and away from the entry component 24, repeatedly move the mobile device 26 (see Figure 8 ), and rotate the mobile device approximately 90 degrees in front of the component (see Figure 9 (and other postures.)

[0105] Similar to the example of device-free posture, in the example of device posture 94, entering component 24 may not involve motion detection or measurement. All such analysis can be retained in application 60, which is part of mobile device 26. Optionally, mobile device 26 may include RSSI module 92, which can measure periodic changes in the signal strength of beacon signal 78 as a result of mobile device 26 repeatedly moving through beacon signal path or wireless interface 28.

[0106] Tapping / light tapping gestures enter the control system:

[0107] refer to Figure 2 and 20 In one embodiment, the gesture-based entry control system 20 may be a tapping gesture entry control system. In this embodiment, the user 23 of the mobile device 26 performs a tapping action, which may be a predefined tapping frequency. The term "tapping" in this embodiment will include a light tapping motion. The tapping may occur on the mobile device 26, the entry component 24, or the door 22 (see [link to documentation]). Figure 1 It may be performed near the wall area of ​​the access component 24 and / or door 22, or conveniently located on any other surface near the access point.

[0108] The tapping gesture entry control system 20's mobile device 26 may also include a microphone 130 and a tapping module 132 of the application 60. The microphone 130 may be sensitive enough to detect a wide range of frequencies and magnitudes (i.e., loudness) to track sounds produced by repeatedly tapping surfaces such as the front surface of the mobile device 26, the surface of the door 22, the surface of the door frame 136, the surface of the entry device 24, the surface of the wall 138 through which entry is made, or other surfaces. The tapping is an intentional gesture performed by the user 23 (see...). Figure 20 (Tap gesture 140). Tapping or tapping the mobile device 26 can be considered as a device gesture of the intentional gesture type, while tapping any other surface can be considered as a non-device gesture of the intentional gesture type.

[0109] In one embodiment, the tapping module 132 of application 60 is configured to receive a signature or related information of the audible sound caused by the tapping gesture 140. The tapping module 132 can then compare the measured frequency pattern of the audible sound (i.e., the tapping or the frequency of the tapping) with a pre-programmed frequency pattern. In one embodiment, if the measured frequency pattern sufficiently matches or substantially matches the pre-programmed frequency pattern, the tapping module 132 can determine that the tapping gesture 140 was performed by user 23 and initiate the transmission of command signal 38 to the input component 24.

[0110] In another embodiment, the tapping gesture entry control system 20 can be configured to further confirm (e.g., independently confirm) the execution of the tapping gesture to enhance reliability and reduce or eliminate erroneous gesture confirmations. One such confirmation may include the use of an IMU sensor system 46 similar to that described previously. For example, if the mobile device 26 is in the back pocket 56 (see...) Figure 1 If user 23 performs a tapping gesture 140 on door 22, mobile device 23 can still measure motion attributed to the tapping action (i.e., motion of the mobile device). In some cases (e.g., user walking), the measured actual motion can be a composite motion, and application 60 is configured to decrypt multiple motions from the composite motion. Once decrypted, the frequency pattern of the motion attributable to the tapping is compared with a pre-programmed motion frequency pattern (i.e., which can be the same as the audible frequency pattern). If the motion frequency pattern matches or substantially matches the pre-programmed frequency pattern, the confirmation of the tapping gesture is reaffirmed.

[0111] In another embodiment, the tapping gesture entry control system 20 may use other sensing data to reconfirm the gesture confirmation. For example, light sensor data from the environmental detection system 48 and / or RSSI data generated by the fluctuations of the beacon signal 78 and produced by the RSSI module 92, as previously described. In one embodiment, the tapping gesture 140 may be a deviceless gesture. In this example, and if the IMU sensing system 46 is applied, the location of the mobile device 26 can also be determined in the manner previously described. The detection process applied to detect the tapping gesture 140 may incorporate the various methods described and optional mobile device localization methods to provide good intent marking as part of application 60.

[0112] refer to Figure 2 , 20 And 21, and in another embodiment, the tapping gesture 140 can be performed on the front surface 148 of the mobile device 26. The mobile device 26 and Figure 21The XYZ coordinates are shown in the diagram. If the tapping gesture 140 is performed on surface 148, the audible tapping sound is evaluated as previously described. Reconfirmation of the detection using the IMU sensing system 46 and the tapping module 132 allows for the evaluation of motion only along the Z-axis to mask motion noise generated along other coordinates. That is, the tapping is performed on the front surface 148, and the direction of the tapping is substantially perpendicular to the front surface 148.

[0113] It is understandable and conceivable that knocking on the mobile device 26 instead of the door 22 could prevent disturbing the person on the other side of the door 22 that the user 23 wants to enter. It should also be understood that preconditions may apply before the knocking gesture 140 is accepted. Such preconditions could be requiring the user 23 to be within a predetermined proximity range of the entry component 24 or the door 22. Furthermore, the mobile device 26 can be knocked before the user 23 reaches the door. Conversely, an example of knocking on the door is when the user 23 has already arrived. Therefore, in the example of knocking on the mobile device 26, the user 23 is able to perform the action as they approach the door 22. Then, when the user 23 arrives, the door 22 can be unlocked.

[0114] Adaptive Intent Pattern Detection

[0115] refer to Figure 16 and 17 The posture-based entry control system 20 can be flexible and can be adapted to various orientations, including device posture 94 (see [reference]). Figure 6 ) and no-device posture 25 (see Figure 1 The system automatically adjusts to different intentional postures of the mobile device 26. In addition, when it is determined whether the user 23 is performing intentional postures 25 and 94, the system can adjust the motion array (i.e., compound motion), location, and position of the mobile device 26.

[0116] Figure 16 Multiple locations and uses of mobile devices 26 are shown without limitation, wherein application 60 is able to adapt to determine whether an intentional gesture 25, 94 is being performed. Thus, by determining the movement, location, position, and / or use of the mobile device, application 60 is also able to select an appropriate pre-programmed gesture from multiple pre-programmed gestures.

[0117] As previously mentioned, the inertial measurement unit (IMU) sensor system 46, the environmental detection system 48, and the internal activity notification module 50 together provide information used by the application 60 to determine whether an intentional gesture 25, 94 is being performed.

[0118] exist Figure 16Examples of potential locations, positions, and uses of various mobile devices 26 are shown, and may include depictions 300 showing the mobile device 26 located at the user 23's ear 302 and in a generally vertical position for making calls or calls. Depiction 304 shows the mobile device 26 located in a front shirt pocket 306, thus in a generally vertical position and in a relatively dark environment. Depiction 308 shows the mobile device 26 in the user 23's hand 74, positioned at approximately 30 degrees for texting, and in use of the texting function. Depiction 310 shows the mobile device 26 located in a front trouser pocket 312, thus in a generally vertical position and in a relatively dark environment. Depiction 314 shows the mobile device 26 located in a back trouser pocket 56 (see also...). Figure 1 Therefore, it has a basically vertical position and is in a relatively dark environment. Depiction 316 represents a suspended mobile device 26. For example, user 23 can simply hold the mobile device 26 in hand 74. Depiction 318 is a handbag (i.e., a container 95, see also...) Figure 10 The mobile device 26 is in a dark environment, and the depiction of 320 is a backpack (i.e., a container 95, see also) Figure 13 Mobile devices 26 in )

[0119] refer to Figure 17 The application 60 entering the control system 20 may include an activity notification module 50, an environment module 322, a motion module 324, a selection module 326, and multiple mode modules (i.e., five shown as 328A, 328B, 328C, 328D, and 328E). The activity notification module 50 is configured to determine the current use of the mobile device 26 and / or categorize the current use of the mobile device 26. Examples of use include sending text messages, making calls, and being in standby mode. The environment module 322 is configured to receive and categorize environmental information (see arrow 330) from the environment detection system 48. As previously described, the environmental information 330 may include light level data, temperature data, location data, location data, photographic data, sound data, and other data. The motion module 324 is configured to receive and categorize motion information (see arrow 332) from the IMU sensor system 46. Non-limiting examples of motion information include the composite motion described earlier, and this composite motion can occur in a variety of scenarios, including when user 23 walks, stands still, carries container 95, performs use, and a variety of other events that can generate motion. One or more of modules 50, 322, and 324 may include algorithms (which may be self-learning algorithms) and pre-programmed data (i.e., a portion of scenario data 66) to refine and / or classify information 330, 332, and other data available for use by selection module 326.

[0120] Selection module 326 is configured to apply information output from modules 50, 322, and 324 to select one of the mode modules 328. In one embodiment, each of the mode modules 328 may be at least partially associated with a corresponding depiction 300, 304, 308, 310, 318, and 320. Selection module 326 may include a pre-programmed data matrix 334 and an algorithm. The pre-programmed data matrix 334 may represent motion and parameter (i.e., environmental and usage) data received from modules 50, 322, and 324. From at least the data matrix 334, the selection module is able to select the appropriate mode module 328. This selection may occur before or during the execution of the intentional pose 25, 94.

[0121] Each mode module 328A, 328B, 328C, 328D, 328E may include corresponding pre-programmed scene data 66A, 66B, 66C, 66D, 66E of the preceding scene data 66. Each of the plurality of mode modules 328 may also include a corresponding one of a series of intent detection algorithms 336 for each of the respective mode modules shown (i.e., see 336A, 336B, 336C, 336D, 336E). In operation, the selection module 326 is configured to substantially activate appropriate algorithms 336A, 336B, 336C, 336D, 336E by selecting appropriate modules 328A, 328B, 328C, 328D, 328E. Each algorithm 336A, 336B, 336C, 336D, 336E is characterized according to the context in which it is applied. For example, algorithm 336A might be suitable when user 23 holds mobile device 26 in hand 74, but it might be less suitable when mobile device 26 is in back pocket 56. Therefore, selection module 326 enables and disables different mode modules 328 in real time.

[0122] During operation, when the appropriate selected mode module 328 conditionally detects the intentional postures 25 and 94, the mode module can output the command signal 38 to the input component 24.

[0123] Seamless access to the control system:

[0124] refer to Figure 2 and 18 In one embodiment, the gesture-based entry control system 20 may be a seamless entry control system adapted to provide an inherent gesture 334 (see [reference needed]) indicating, for example, an intentional expectation and initial action of the user to open door 22. Figure 18 This allows user 23 to enter. More specifically, the inherent posture 334 is the initial part of the typical user exercise 336 that is guided to gain entry.

[0125] The mobile device 26 that seamlessly enters the control system 20 can be a wearable mobile device. Examples of wearable mobile devices 26 include smartwatches, smart glasses, and smart shoes. The term "smart" is intended to indicate that the wearable mobile device 26 includes a processor 56 and other features / components previously described.

[0126] Access component 26 may also include a near-field communication (NFC) device 337 for generating a beacon signal 78. In one example, the NFC device 337 may be a Bluetooth device, the beacon signal 78 may be a Bluetooth signal, and the wearable mobile device 26 may be configured to process the Bluetooth signal. In one example, a proximity sensor 90 of the environmental detection system 48 may be used to measure the strength of the beacon signal 78, and through this measurement, the application may determine the proximity of the wearable mobile device 26 to access component 24.

[0127] Mobile device 26 may also include magnetometer 338 and a basic fact verification module 340, as part of application 60 (see...). Figure 2 Part of the magnetometer 338. The magnetometer 338 can be used to confirm, for example, grasping the door handle 342 as part of an inherent posture 334. (e.g.) Figure 18 As best illustrated, the inherent posture 334 portion of user practice 336 can be an ordered set of movements performed by the user. The ordered set of movements can depend on the type of wearable mobile device 26 and the type of activity desired.

[0128] For the sake of simplicity of explanation and understanding, this is only a non-limiting embodiment of the application, and the obtained entry type is described as through gate 22 (see Figure 1 The user 23 can enter the door 22 by walking at frame 342, starting with a slowdown and / or a complete stop. In frame 344, the user 23 can raise hand 74, holding the smartwatch 26, to reach the door handle 346. In frame 348, hand 74 can grasp the handle 346, preparing to pull or push open the door 22. This grasping motion of the inherent posture 334 can be sensed by the magnetometer 338 of the wearable mobile device 26.

[0129] During operation, and after application 60 executes and confirms the inherent posture 334, wearable mobile device 26 sends command signal 38 to entry component 24 to actuate from non-entry state to entry state, as described above. With entry component 24 in entry state, and at box 350, user 23 can complete entry exercise 336 by pulling open door 22 (see arrow 352).

[0130] Application 60 confirms basic facts module 340 (see Figure 2The module 340 is configured to receive information from the IMU sensing system 46 indicating a pull 352, which specifies the final step into exercise 336. This confirmation pull 352 can be verified via a pre-programmed confirmation pull, which may be part of the previously described scene data 66. By confirming that user 23 did indeed pull 352, module 340 is able to further confirm the accurate determination of the inherent posture. This confirmation can then be used to further refine the machine learning algorithm 336 (see...). Figure 17 ) and / or other application algorithms executed by application 60.

[0131] In an example where the wearable mobile device 26 is smart glasses, the smart glasses can be worn on the head of the user 23, and a portion of the inherent posture 334 may include the user gazing when approaching the entry component 24 and tilting the head when approaching the handle 346 of the door 22.

[0132] In the example where the wearable mobile device 26 is a smart shoe, the smart shoe can be worn on the foot of user 23, and a portion of the inherent posture 334 may include the light stepping of user 23's foot.

[0133] Entry control system based on prior posture:

[0134] refer to Figure 2 and 22 The gesture-based entry control system 20 can be a gesture-based entry control system. In this embodiment, the mobile device 26 is configured to prepare itself before the user performs a device gesture or a no-device gesture (i.e., the primary gesture). That is, the system combines explicit gestures from multiple gestures to apply implicit behavior detection. The prior event or process can be or may include the inherent gesture 334 (see Figure 18 The execution of the inherent posture 334. After user 23 performs the inherent posture 334, user 23 needs to perform the primary posture for a specified duration. A non-limiting example of the inherent posture 334 could be the action of slowing down walking as user 23 approaches and enters component 24.

[0135] refer to Figure 2 Application 60, which has any relevant hardware, may also include a timer or clock 142 and a satellite-based positioning module 144 (e.g., Global Positioning System (GPS)). In another embodiment, the satellite-based positioning module 144 may be a separate device from application 60, configured to send relevant location information to application 60.

[0136] To detect prior events (i.e., inherent posture 334), the IMU sensing system 46 can be active. Activation of the IMU sensing system 46 can be triggered when the user 23 approaches a designated vicinity of component 24. The presence of the user 23 in the vicinity can be demonstrated in any of a variety of ways. For example, any one or more of the following can be used: detection of a satellite-based positioning module 144, a proximity sensor 90 of the environmental detection system 48, detection of a beacon signal 78 generated from an NFC device 337 entering component 24, and others.

[0137] In a non-limiting embodiment, implicit detection of user 23's entry intent can rely on the user's intuition that they will slow down and stop as they approach the destination door 22 associated with the entry component 24, and perform a primary, intentional gesture to indicate their position. This intuition can be used to improve the reliability of gesture detection.

[0138] refer to Figure 22 The diagram illustrates a method for operating an entry control system 20 based on a prior posture. In block 400, the IMU sensing system 46 is activated, initiating an IMU analysis performed by the motion module 324 of application 60. In block 402, the motion module 324 determines, for example, whether user 23 is walking. At block 404, and if user 23 is walking, the motion module 324 determines whether user 23 is slowing down (i.e., inherent posture 334). If walking is slowing down, inherent posture 334 is detected (in this example).

[0139] In box 406, and after detecting or acknowledging user 23 via inherent gesture 334, application 60 may start timer 142, thereby running for a predetermined duration. In box 408, and during the predetermined duration, mobile device 26 monitors for the occurrence of a primary, intentional gesture. If a primary, intentional gesture is detected, and at box 410, application 60 implements command signal 38 to the output of entry component 24 (e.g., opening door 22). It is envisioned and understood that the primary, intentional gesture can be a device gesture, a no-device gesture, and / or another inherent gesture.

[0140] At box 412, as an optional step, and if no primary intentional pose has been detected, the applied motion module 324 (or otherwise) may determine whether user 23 has, for example, completely stopped walking. If not, application 60 continues to monitor the execution of the primary intentional pose. This optional step may be helpful when pose detection is not at a high confidence level. If user 23 has stopped walking, and at box 414, application 60 determines whether the duration has expired. If the duration has not expired, application 60 continues to monitor the execution of the primary intentional pose. If the duration has expired, the process is deactivated, or if user 23 remains near entry component 24, motion module 324 is restarted to detect the prior inherent pose (i.e., the prior event performed by user 23).

[0141] It is conceivable and understandable that at any stage during the process (e.g., in box 408), mobile device 26 may provide auditory and / or visual notifications to user 23. For example, mobile device 26 may notify user 23 that the mobile device is waiting for the execution of a primary, intentional gesture. As another example, and at the end of the duration, mobile device 26 may notify user 23 that a primary, intentional gesture failure has been detected.

[0142] In one embodiment, the preceding events can be pre-programmed, and the primary intentional gesture can be pre-selected by user 23 from a plurality of pre-programmed gestures. Non-limiting examples of the primary, intentional gesture may include: a waving motion of the hand 74 approaching and entering component 24 (i.e., a deviceless gesture or body gesture 25, see [link to relevant documentation]). Figure 1 ); knock lightly on the door 22 or enter component 24 (a device-free or body posture 25, see Figure 20 ); specific body postures that trigger inertial motion, where the mobile device is attached to the user's body (see also...) Figure 1 ); Applying body movement to the housing 95, which contains the mobile device 26 and is carried by the user 23 (i.e., housing movement 99, see Figure 12-15 ); the waving of mobile device 26 near entry component 24 (i.e., the type of device posture 94, see Figure 6-9 ).

[0143] Cloud-based, gesture-based entry control system:

[0144] refer to Figure 19The gesture-based entry control system 20 may include the use of a cloud 360 (i.e., a remote server). In this embodiment, the application 60 may reside in the cloud 360, thus wirelessly transmitting information 330, 332 collected by the IMU sensing system 46, the environmental detection system 48, and other components from the mobile device 26 to the cloud 360 for processing. Command signal 38 may be transmitted directly from the cloud 360 to the entry component 24, or returned to the mobile device 26, which then transmits signal 38 to the entry component 24.

[0145] The advantages of a cloud-based architecture include the performance of some or all of the computation and the storage of data in the cloud. This allows for the use of potentially more powerful algorithms, but may come at the cost of communication latency. Another advantage is that the mobile device 26 does not need to communicate directly with the entry component 24; instead, the cloud 360 directly sends command signals to the entry component 24 for entry authorization.

[0146] The advantages and benefits of this disclosure include enabling gesture detection without requiring the mobile device 26 to be held in the hand. Another advantage includes the ability to identify, for example, the door 22 that the user 23 intends to enter, as part of intent detection. Other advantages include reliable intent detection and a relatively inexpensive and robust design.

[0147] The aforementioned functions can be implemented or supported by a computer program, which is formed by computer-readable program code and embodied in a computer-readable medium. Computer-readable program code can include source code, object code, executable code, etc. A computer-readable medium can be any type of media that can be accessed by a computer, and can include read-only memory (ROM), random access memory (RAM), hard disk drive, optical disc (CD), digital video disc (DVD), or other non-transitory forms.

[0148] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a,” “an,” and “the,” and “the,” are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It should also be understood that, when used in this specification, the terms “comprising,” “including,” and / or “having” specify the presence of the recited features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof.

[0149] As used herein, depending on the context, the term "if" may optionally be interpreted as "when" or "in" or "in response to determination" or "in response to detection." Similarly, the phrases "if determined" or "if [the stated condition or event] is detected" may optionally be interpreted as "when [the stated condition or event] is determined" or "in response to determination" or "when [the stated condition or event] is detected" or "in response to detection," depending on the context.

[0150] The terms used herein, such as component, application, module, system, etc., are intended to refer to computer-related entities, or hardware, a combination of hardware and software, or software execution. As an example, an application can be, but is not limited to, a process, processor, object, executable file, execution thread, program, and / or computer running on a processor. An application running on a server and the server itself can be components. One or more applications may reside within a process and / or execution thread, and applications may be localized on a single computer and / or distributed across two or more computers.

[0151] While this disclosure has been described with reference to one or more exemplary embodiments, those skilled in the art will understand that various changes can be made and elements can be substituted with equivalents without departing from the scope of this disclosure. Furthermore, many modifications can be made to adapt particular situations or materials to the teachings of this disclosure without departing from the basic scope of this disclosure. Therefore, this disclosure is intended to be limited to the specific embodiments disclosed as the best mode for carrying out this disclosure, but rather to include all embodiments falling within the scope of the claims.

Claims

1. A method for training a software-based application for a posture-based entry control system, comprising: Choose an intentional pose type from a pre-programmed library of multiple intentional pose types; A first training instruction is transmitted to the user of the mobile device via the human-machine interface device of the mobile device, wherein the training instruction instructs the user to perform a first movement at a first position on the user using the mobile device. The first movement is performed at the first position on the user using the mobile device; When the user performs the first movement, scene data including a user-specific model is pre-programmed using information received from at least one detection system of the mobile device, wherein the information includes motion data; The second training instruction is transmitted to the user via the human-machine interface device, wherein the second training instruction instructs the user to perform the first movement at a first position on the user using the mobile device in a selected intentional posture that indicates the user's intention to enter an access component; The first movement is performed using the selected intentional posture at the first position on the user's body via the mobile device; When the user performs the first movement with the selected intentional posture, the scene data is preprogrammed using information received from the at least one detection system, wherein the information includes motion data; and The scene data was used to train a machine learning algorithm for pose recognition; A user of a mobile device repeatedly performs an intentional gesture of a selected intentional gesture type, wherein the intentional gesture indicates the user's intention to gain access; and Through a software-based application of the gesture-based entry control system, a model of the selected intentional gesture type is formed from the repeated execution of the intentional gesture, wherein the software-based application uses the model to determine the future execution of the intentional gesture. The machine learning algorithm is configured to determine or confirm the user's intent based on an explicit intent signal generated by the at least one detection system, and is configured to determine user authentication based on the intent signal generated by the at least one detection system and the user-specific model.

2. The method according to claim 1, wherein, The formation of the model includes: The machine learning algorithm is used to analyze the data collected from the repeated execution of the intentional pose.

3. The method according to claim 2, wherein, The data is collected from at least one detection system of the mobile device.

4. The method according to claim 3, wherein, The data includes motion data detected by the inertial measurement unit (IMU) sensing system of the at least one detection system.

5. The method according to claim 4, wherein, The data includes audible data detected by the microphone of the mobile device.

6. The method according to claim 5, wherein, The intentional gesture is the user tapping on the mobile device.

7. The method according to claim 1, wherein, The software-based application prompts the user to select the desired gesture type via the HID (Human Interface Device) of the mobile device.

8. The method according to claim 1, wherein, When the software-based application completes the model based on confidence, it notifies the user via the HID (Human-Machine Interface) device of the mobile device.

9. The method according to claim 7, wherein, The software-based application notifies the user via the HID when it completes the model based on confidence level.

10. The method according to claim 1, wherein, The pre-programmed library includes at least one of the user tapping the mobile device, the user knocking on the door, and the user's voice commands.

11. The method according to claim 1, wherein, The software-based application is configured to perform the intentional gesture to confirm the user's intent and authenticate the user.

12. The method of claim 11, further comprising: The frequency of the intentional gestures is analyzed by the software-based application to authenticate the user, wherein the intentional gestures are one of tapping and striking.

Images