Determining intent to open door

Through image machine learning models, the position and orientation of the human body near the door are analyzed, and whether there is an intention to open the door is determined, which solves the problem of inaccurate intention identification in the prior art, and realizes intelligent management and energy efficiency improvement of the door.

CN118901089APending Publication Date: 2024-11-05ASSA ABLOY AB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380027673.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-24
Filing Date
2023-03-23
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

The prior art is difficult to accurately determine whether a person has the intention to open the door, especially in remote system management and unlocked automatic door scenarios, which can easily lead to unnecessary door opening, affecting energy efficiency and safety.

Method used

The image near the door is obtained through the camera device, the human body is determined using the image machine learning model, the data structure is constructed and the analysis is repeated, and the door operation is not performed until the intention is determined to be true.

Benefits of technology

It realizes reliable determination of people's intention to open the door, avoids unnecessary energy consumption and safety risks, and ensures the rational use and management of the door.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118901089A_ABST
    Figure CN118901089A_ABST
Patent Text Reader

Abstract

A method for determining an intention to open a door is provided. The method is performed by an intent determiner. The method comprises: obtaining an image of a physical space near a door; determining a position and orientation of the person in the image by providing the image to an image machine learning model, where the image machine learning model is configured to determine the position and orientation of the person in the image, where the image machine learning model is configured to determine a stick figure of the person based on the image; adding a data item including an indicator of the location and an indicator of the orientation to the data structure; determining whether there is an intention of a person to open the door based on the data structure; and repeating the method until the exit condition is true.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of determining human intent, and in particular to determining when there is a human intent to open a door. Background Art

[0002] Locks and keys have evolved from traditional purely mechanical locks. Today, there are wireless interfaces for electronic locks, for example, by interacting with portable key devices. For example, radio frequency identification (RFID) has been used as a wireless interface.

[0003] When using RFID, the user needs to present the portable key device near a reader connected to the lock. RFID requires a relatively large antenna in the reader next to the lock and uses a large amount of energy. In addition, RFID is not an interface that can be used for remote system management of the lock; only system management using an RFID device close to the lock can be used for such tasks. Therefore, in order to allow remote system management (e.g., configuration and monitoring), a second radio interface needs to be added.

[0004] Another solution is to use Bluetooth Low Energy (BLE) or Ultra Wideband (UWB) or Ultra High Frequency (UHF). However, for these communication protocols, the range is long and it is difficult to determine intent for a specific door. One problem if the door lock is unlocked whenever a valid portable key device is within range is that when a person inside the electronic lock walks past the electronic lock, the electronic lock will open and an unauthorized person outside can enter the restricted physical space.

[0005] Additionally, for an unlocked automatic door (e.g., a swing door or sliding door with an opener), it is desirable to determine an intent to open the door when there is an intent, but to avoid opening the door when there is no intent, such as to save energy used to regulate the climate of the indoor space. Summary of the invention

[0006] One object is to improve the determination of a person's intention to open a door.

[0007] According to a first aspect, a method for determining an intention to open a door is provided, the method being performed by an intention determiner. The method comprises: obtaining an image of a physical space near a door; determining a position and orientation of a person in the image by providing the image to an image machine learning model, wherein the image machine learning model is configured to determine a position and orientation of a person in the image, wherein the image machine learning model is configured to determine a stick figure of the person based on the image; adding a data item to a data structure, the data structure comprising an indicator of the position and an indicator of the orientation; determining whether there is an intention of the person to open the door based on the data structure; and repeating the method until an exit condition is true.

[0008] In the step of adding data items, the data items may include coordinates of anatomical features represented by the stick figures.

[0009] In the step of adding a data item, the data item may include an orientation of at least one anatomical feature represented by the stick figure.

[0010] Adding the data item may include adding the data item while maintaining the position of the data item in the sequence relative to any previous data item in the data structure.

[0011] The exit condition may be true when the intention determiner determines that there is an intention of the person to actually open the door, in which case the method further comprises: sending a signal to perform a door opening procedure.

[0012] The exit condition may be true when a person is no longer determined to be in the image.

[0013] Determining the position and orientation may include determining a center point of the person in the image, in which case the center point is an indicator of the position.

[0014] Determining the position and orientation may include determining an orientation of the person in the image, the orientation of the person in the image indicating the orientation of the person's torso, in which case the orientation of the person in the image is an indicator of the orientation.

[0015] Adding the data item may include adding an indicator of the time to the data structure in association with the data item.

[0016] Determining whether intent is present may include providing the data structure to the intent machine learning model, in which case the intent machine learning model is configured to infer intent, or lack thereof, based on the input data structure.

[0017] According to a second aspect, there is provided an intention determiner for determining an intention to open a door. The intention determiner comprises: a processor; and a memory storing instructions that, when executed by the processor, cause the intention determiner to: obtain an image of a physical space near a door; determine a position and orientation of a person in the image by providing the image to an image machine learning model, wherein the image machine learning model is configured to determine a position and orientation of a person in the image, wherein the image machine learning model is configured to determine a stick figure of a person based on the image; add a data item to a data structure, wherein the data item comprises an indicator of a position and an indicator of an orientation; determine whether there is an intention of a person to open the door based on the data structure; and repeat the instructions until an exit condition is true.

[0018] The data items may include coordinates of anatomical features represented by stick figures.

[0019] The data item may include an orientation of at least one anatomical feature represented by the stick figure.

[0020] The instructions for adding an indicator may include instructions that, when executed by the processor, cause the intent determiner to send a signal to add the data item while maintaining the position of the data item in the sequence relative to any previous indicators in the data structure.

[0021] The exit condition may be true when the intention determiner determines that there is an actual intention of the person to open the door, in which case the intention determiner further includes instructions that, when executed by the processor, cause the intention determiner to send a signal to perform a door opening procedure.

[0022] The exit condition may be true when a person is no longer determined to be in the image.

[0023] The instructions for determining position and orientation may include instructions that, when executed by the processor, cause the intent determiner to determine a center point of a person in the image, in which case the center point is an indicator of position.

[0024] The instructions for determining position and orientation may include instructions that, when executed by the processor, cause the intent determiner to determine an orientation of a person in the image, the orientation of the person in the image indicating an orientation of the person's torso, in which case the orientation of the person in the image is an indicator of orientation.

[0025] The instructions for adding the data item may include instructions that, when executed by the processor, cause the intent determiner to add an indicator of the time to the data structure in association with the data item.

[0026] Instructions for determining whether intent exists may include instructions that, when executed by a processor, cause an intent determiner to provide a data structure to an intent machine learning model, in which case the intent machine learning model is configured to infer intent, or the lack of intent, based on the input data structure.

[0027] According to a third aspect, a computer program for determining an intention to open a door is provided. The computer program includes computer program code that, when executed on an intention determiner, causes the intention determiner to: obtain an image of a physical space near a door; determine a position and orientation of a person in the image by providing the image to an image machine learning model, wherein the image machine learning model is configured to determine a position and orientation of a person in the image, wherein the image machine learning model is configured to determine a stick figure of the person based on the image; add a data item to a data structure, the data item including an indicator of a position and an indicator of an orientation; determine whether there is an intention of the person to open the door based on the data structure; and repeat the computer program code until an exit condition is true.

[0028] According to a fourth aspect, there is provided a computer program product comprising the computer program according to the third aspect and a computer readable device comprising a non-transitory memory storing the computer program.

[0029] Generally, unless otherwise expressly defined herein, all terms used in the claims should be interpreted according to their ordinary meaning in the technical field. Unless otherwise expressly stated, all references to "a / an / the element, device, component, method, step, etc." will be openly interpreted as referring to at least one instance of an element, device, component, method, step, etc. Unless expressly stated otherwise, the steps of any method disclosed herein do not have to be performed in the exact order disclosed. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various aspects and embodiments will now be described by way of example with reference to the accompanying drawings, in which:

[0031] Figure 1 is a schematic diagram showing an environment in which the embodiments proposed herein can be applied;

[0032] FIG. 2A to FIG. 2D is a schematic diagram illustrating an embodiment in which an intent determiner may be implemented;

[0033] Figure 3 is to show a person (e.g. Figure 1 Schematic diagram of the position and orientation of a person);

[0034] Figure 4 is a schematic diagram showing a stick figure derived from an image of a user captured by a camera;

[0035] Figure 5 is a flow chart illustrating an embodiment of a method performed by an intention determiner for determining an intention to open a door;

[0036] Figure 6 It is shown FIG. 2A to FIG. 2D a schematic diagram of components of an intent determiner; and

[0037] Figure 7 An example of a computer program product comprising computer readable means is shown. DETAILED DESCRIPTION

[0038] Various aspects of the present disclosure will now be described more fully below with reference to the accompanying drawings, in which certain embodiments of the present invention are shown. However, these aspects may be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that the present disclosure will be comprehensive and complete and fully convey the scope of all aspects of the present invention to those skilled in the art. Throughout the specification, the same reference numerals refer to the same elements.

[0039] Embodiments presented herein provide an intention determiner for determining when there is an intention of a person to open a door. A camera (or other imaging device) repeatedly obtains images of the area near the door. For each image, the intention determiner abstracts the person's anatomical structure into a stick figure to determine the person's position and orientation and stores it in a data structure. The intention to pass through the door is then determined based on the data structure (e.g., using a machine learning model).

[0040] Figure 1 1 is a schematic diagram showing an environment in which the embodiments presented herein may be applied. Access to a physical space 16 is restricted by an openable door 15, which may be selectively unlocked. The door 15 is located between the restricted physical space 16 and the accessible physical space 14. In other words, the restricted physical space 16 is inside the door 15, and the accessible physical space 14 is outside the door 15. Although Figure 1 The door 15 in the figure is shown as a double sliding door, but the door 15 can be any suitable door type, such as a single sliding door, a swing door, a revolving door, etc. In order to unlock the door 15 or lock the door 15, an electronic lock 12 is optionally provided. The electronic lock 12 can be in an unlocked state or a locked state. The door 15 is set in a surrounding fixed structure 11 (such as a wall or a fence).

[0041] When access is granted by the access control of the electronic lock 12, the electronic lock 12 is set to an unlocked state. When the electronic lock 12 is in the unlocked state, the door 15 can be opened, and when the electronic lock 12 is in the locked state, the door 15 cannot be opened. In this way, access to the closed space 16 is controlled by the electronic lock 12. It should be noted that the electronic lock 12 can be installed in the fixed structure 11 through the door 15 (as shown in the figure), or installed in the door 15 itself (not shown).

[0042] A camera 4 or other image capture device is provided to capture images of the area near the door 15. This allows the camera 4 to capture when the person 5 moves from the first position 2a to the second position 2b and to the third position 2c. The camera can be provided to be mounted to the physical structure 11 (as shown), or mounted in any other suitable location, such as above the door 15, or mounted to the ceiling above the space just outside the door to capture people near the door. The camera can be configured to capture images based on visual light / IR light and / or depth information (e.g., based on LiDAR, radar, stereo vision, etc.). Therefore, the image from the camera can be a 2D (two-dimensional) image or a 3D (three-dimensional) image.

[0043] As described in more detail below, images from the camera 4 are sequentially analysed to thereby track the position and orientation of the person 5 over time, thereby enabling a determination to be made as to whether the person 3 demonstrates an intention to enter through the door 5 .

[0044] The electronic lock 12 and / or the camera device 4 optionally comprise communication capabilities connected to a server 6 of the electronic access control system 10 via a communication network 7. The communication network 7 may be a wide area network such as the Internet, to which the portable key devices 2, 3 may be connected, for example, via WiFi (e.g. any of the IEEE 802.11x standards) or a cellular network (e.g. LTE (Long Term Evolution), next generation mobile networks (fifth generation, 5G), UMTS (Universal Mobile Telecommunications System) using W-CDMA (Wideband Code Division Multiplexing)), etc.

[0045] FIG. 2A to FIG. 2D is a schematic diagram showing an embodiment in which the intention determiner 1 may be implemented. The intention determiner 1 is used to determine when there is an intention of a person to open a door.

[0046] exist Figure 2A In FIG. 1 , the intention determiner 1 is shown to be implemented in an electronic lock 12. Therefore, the electronic lock 12 is a host device of the intention determiner 1.

[0047] exist Figure 2B , the intent determiner 1 is shown to be implemented in a server 6. Therefore, the server 6 is a host device of the intent determiner 1.

[0048] exist Figure 2C , the intention determiner 1 is shown as being partially implemented in the server 6 and partially implemented in the electronic lock 12. Therefore, both the server 6 and the electronic lock 12 are host devices of parts of the intention determiner 1.

[0049] exist Figure 2D , the intention determiner 1 is shown to be implemented as a standalone device.

[0050] Figure 3It is shown by Figure 1 A schematic diagram of an image captured by a camera device, in Figure 3 People can be seen in Figure 1 4. The position and orientation of the person 3) in the image is shown in Figure 2. Position 2 is shown as the center position of the user 5 (in the coordinate system xy of the image from the camera 4). The orientation is shown as the direction 8 of the torso of the user 5. A bounding box 10 is also shown, which is a rectangle that just contains the person in the image. The bounding box 10 can be set so that the bounding box 10 touches the outer edges of the person 5, or a small margin can be set between the bounding box 10 and the person 5. The phrase bounding box is sometimes also called a minimum bounding box.

[0051] Figure 4 is a schematic diagram showing a stick figure 9 derived from an image of a user captured by a camera using an image machine learning (ML) model. The stick figure shows anatomical features such as joints and other anatomical features, as well as limbs and other body parts between the anatomical features.

[0052] The identified anatomical features may be a predefined set of joints and other anatomical features. Figure 4 An example of such a set of anatomical features is shown in FIG. There are shown a left shoulder 20a, a right shoulder 20b, a left elbow 20c, a right elbow 20d, a left wrist joint 20e, a right wrist joint 20f, a left hand end 20g, a right hand end 20h, a left hip joint 20i, a right hip joint 20j, a left knee 20k, a right knee 20l, a left heel 20m, a right heel 20n, a left foot end 20o, a right foot end 20p, and a head 20q. Another anatomical feature that can be derived from other anatomical features is a torso 21. It should be noted that any other suitable set of anatomical features can be used. For example, the hand ends 20g, 20h and / or the foot ends 20o, 20p can be omitted.

[0053] Optionally, the image ML model not only identifies anatomical features, but also identifies the orientation of selected features. For example, the orientation of the head 20q can be determined based on the identification of facial features (e.g., nose, eyes, mouth, etc.), for example, by image processing of an image of the user. In addition, the orientation of the torso 21 can be determined based on image processing of an image of the user and / or based on the relative coordinates of other anatomical features.

[0054] When analyzing the same user over time, stick figures can be used as a clear indication of movement. Abstracting the movement of body postures and body features (when analyzing across several images) is not only efficient in data usage, but also accurately depicts the movement of the person. In addition, since the images themselves do not need to be saved, the privacy of the person is protected.

[0055] Figure 5is a flow chart illustrating an embodiment of a method performed by the intention determiner 1 for determining an intention to open a door.

[0056] In an obtain image step 40, the intention determiner 1 obtains an image of a physical space near the door. Here, the physical space near the door is interpreted as a physical space that a person must pass through in order to pass through the door and that is close enough to determine the position and orientation in the image of the nearby space.

[0057] In a determine position and orientation step 42, the intention determiner 1 determines the position and orientation of the person in the image. Optionally, a bounding box is also determined (see Figure 3 ). As mentioned above, a bounding box is a rectangle that just contains the person in the image. A bounding box can be used to indicate the orientation of the person (one of two possible orientations that can be used to determine the possible direction over time, assuming the person is walking forward).

[0058] Optionally, the center point of the person in the image is determined. The center point is then an indicator of position.

[0059] Optionally, the direction of the person in the image is determined. The direction indicates the direction of the person's torso. The shape of the head and / or head organs (such as eyes, ears, mouth, nose, etc.) can also be used to find the direction of the person. The direction of the person in the image is then an indicator of orientation.

[0060] This step includes providing the image to an image ML model, wherein the image ML model is configured to determine the position and orientation of a person in the image. The image ML model is configured to determine a stick figure of the person based on the image (see Figure 4 and described above). As described above, the image can be a 2D image or a 3D image. The image ML model can be any suitable type of machine learning model, such as a convolutional neural network (CNN), a hidden Markov model (HMM), etc.

[0061] In the adding to data structure step 44, the intention determiner 1 adds a data item including an indicator of a position and an indicator of an orientation to the data structure. Optionally, the data item includes the coordinates and / or orientation of (at least some) anatomical features represented by the stick figure 9. In this case, the data of the stick figure (e.g., the coordinates and identity of the anatomical features 20a to 20p, 21 of the person, and optionally the orientation of the selected anatomical features (e.g., face and / or torso) of the person) is also added to the data structure in the data item. The identity of the anatomical features 20a to 20p can be implied by the defined positions in the vector or array including the coordinates / orientation, that is, each position in the vector or array is predefined to be associated with a specific anatomical feature. Then, each position can contain the coordinates / orientation of the anatomical feature. Alternatively, each coordinate / orientation is coupled to an explicit identifier (within the data structure) of the anatomical feature to which the coordinate / orientation belongs. Based on the image type (2D / 3D) and optional processing, the coordinates and optional orientation can be stored as 3D coordinates or 2D coordinates.

[0062] Adding a data item can include adding the data item while maintaining the position of the data item in the sequence relative to any previous data item in the data structure. For example, an indicator of time (e.g., a timestamp) can be added to the data structure in association with the data item or as part of the data item. Alternatively or additionally, a sequence number can be added in association with the data item. Alternatively or additionally, the data structure itself maintains the sequence so that any newly added data item maintains the order in which the data item was added. This can be achieved, for example, by a data structure based on a linked list structure or a similar structure. Since the position of the data item is maintained in the sequence, the data structure can be used to assess how the position and orientation of a person evolves over time.

[0063] When a bounding box has been determined, details defining the bounding box (eg, appropriate coordinate and / or dimension data) are also stored in the data structure in association with the position and orientation of the person in the image.

[0064] Optionally, if items from the data structure are determined to be old (either by a timestamp or by a number of newer data items for the person in question in the data structure), then those items are removed from the data structure.

[0065] In a determine intention step 46, the intention determiner 1 determines whether there is an intention of a person to open the door based on the data structure. The determination may be based on a sequence of data items in the data structure, wherein the sequence is ordered according to when the data items were added to the data structure. In this way, the movement of position and orientation over time is represented by the data structure.

[0066] In one embodiment, determining whether intent is present may include providing a data structure to an intent ML model. The intent ML model is then configured to infer intent (or the absence of intent) based on the input data structure (i.e., a data structure including the position and orientation of a person as described above). The intent ML model may be any suitable type of machine learning model, such as a convolutional neural network (CNN), a hidden Markov model (HMM), an LSTM (long short-term memory), a machine learning transformer, and the like. The use of machine learning is well suited for such determinations because there are only a very limited number of possible outcomes, such as "a person with intent" and "a person without intent." It should be noted that there may be situations where the image ML model cannot determine the coordinates of all anatomical features due to inaccuracies in the image and / or partially occluded objects. In this case, such coordinates may be left blank, and the intent ML model still has many other coordinates on which to perform its inferences. Alternatively, coordinates that the image ML model did not determine are estimated before the data structure is provided to the intent ML model.

[0067] The intent ML model may be trained using manually annotated training data, e.g. video data (i.e., a sequence of 2D or 3D images) that an operator has manually annotated when intent for each person is present in the video data, in which case stick figures are determined corresponding to the same video data and form part of the input features to the intent ML model. Alternatively or additionally, the intent ML model may be trained on actual data of people entering or not entering through a door. A manual button (or gesture recognition, proximity sensor, light sensor, etc.) may then be required to allow the person to manually trigger the opening of the door when intent is erroneously not determined by the method based on the person's dynamics. Such errors and corrections through the button provide valuable training data for improving the performance of intent determination.

[0068] In one embodiment, the determination of whether intent exists may be based on traditional statistical models such as ARIMA (Autoregressive Integrated Moving Average).

[0069] In completing step 47, the intent determiner 1 determines whether an exit condition is true. In one embodiment, the exit condition is true when (and optionally only when) the person is no longer determined to be in the image. In one embodiment, the exit condition is true when intent is determined.

[0070] If the exit condition is true, the method proceeds to the optional send intent signal step 48 or ends. If the exit condition is not true, the method returns to the obtain image step 40 to analyze the new image and add the data to the data structure for the new intent estimate (as described above).

[0071] In an optional send intention signal step 48, the intention determiner 1 sends an intention signal to perform a door opening process. This can cause a door opener to open the door for the person. Optionally, the intention signal is used to trigger an electronic lock to perform access control for the person, for example using local wireless communication and / or biometrics.

[0072] Using the embodiments presented herein, a reliable determination of a person's intention to open a door is provided. It should be noted that the method can optionally be used to track multiple persons in an image simultaneously, in which case it is sufficient for the method to determine the intention that one person shows intention. By using a data structure containing several instances of position and orientation over time, rather than analysis of a single image, the effects of different lighting conditions throughout the day, shadows, etc. are mitigated or even eliminated.

[0073] Figure 6 It is shown FIG. 2A to FIG. 2D Schematic diagram of components of the intent determiner 1. It should be noted that when the intent determiner 1 is implemented in a host device, one or more of the mentioned components can be shared with the host device. The processor 60 is provided using any combination of one or more of a suitable central processing unit (CPU), a graphics processing unit (GPU), a multiprocessor, a microcontroller, a digital signal processor (DSP), etc. that are capable of executing software instructions 67 stored in the memory 64, and the memory 64 can therefore be a computer program product. Alternatively, the processor 60 can be implemented using an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. The processor 60 can be configured to perform the above reference Figure 5 Describe the method.

[0074] The memory 64 may be any combination of random access memory (RAM) and / or read-only memory (ROM). The memory 64 also includes a non-transitory permanent storage device, which may be any single memory or a combination of magnetic memory, optical memory, solid-state memory, or even remotely mounted memory, for example.

[0075] A data memory 66 is also provided for reading and / or storing data during execution of software instructions in the processor 60. The data memory 66 may be any combination of RAM and / or ROM.

[0076] The intention determiner 1 further comprises an I / O interface 62 for communicating with external and / or internal entities. Optionally, the I / O interface 62 further comprises a user interface.

[0077] In order not to obscure the concepts presented herein, other components of the intention determiner 1 are omitted.

[0078] Figure 7 An example of a computer program product 90 comprising a computer readable device is shown. On the computer readable device, a computer program 91 may be stored in a non-transitory memory. The computer program may cause a processor to perform a method according to the embodiments described herein. In this example, the computer program product is in the form of a removable solid-state memory, such as a Universal Serial Bus (USB) drive. As mentioned above, the computer program product may also be implemented in a memory of a device, such as Figure 6 Although the computer program 91 is schematically shown here as part of a removable solid-state memory, the computer program may be stored in any way suitable for a computer program product, such as another type of removable solid-state memory, or an optical disc such as a CD (Compact Disc), a DVD (Digital Versatile Disc) or a Blu-ray Disc.

[0079] Here now a list of embodiments listed with Roman numerals is given from another perspective.

[0080] i. A method for determining an intention to open a door, the method being performed by an intention determiner, the method comprising:

[0081] obtaining an image of a physical space proximate the door;

[0082] determining a position and orientation of a person in the image;

[0083] adding an indicator of the position and an indicator of the orientation to a data structure;

[0084] determining whether there is an intention of the person to open the door based on the data structure; and

[0085] The method is repeated until the exit condition is true.

[0086] ii. The method of embodiment i, wherein adding the indicator comprises adding the indicator while maintaining the position of the indicator in the sequence relative to any previous indicators in the data structure.

[0087] iii. A method according to embodiment i or ii, wherein the exit condition is true when the intention determiner determines that there is an intention of the person to open the door, and wherein the method further comprises:

[0088] Sends a signal for the door opening process to proceed.

[0089] iv. A method according to any one of embodiments i to iii, wherein the exit condition is true when the person is no longer determined to be in the image.

[0090] v. A method according to any one of embodiments i to iv, wherein determining the position and orientation includes: determining a bounding box, which is a rectangle that just contains only the person in the image; and storing the bounding box in the data structure.

[0091] vi. A method according to any one of embodiments i to v, wherein determining the position and orientation includes determining a center point of the person in the image, and wherein the center point is an indicator of the position.

[0092] vii. A method according to any one of embodiments i to vi, wherein determining the position and orientation includes determining a direction of a person in the image, the direction of the person in the image indicating a direction of a chest of the person, and wherein the direction of the person in the image is an indicator of the orientation.

[0093] viii. The method of any one of embodiments i to vii, wherein adding comprises adding an indicator of time to the data structure in association with the indicator of the location and the indicator of the orientation.

[0094] ix. A method according to any one of embodiments i to viii, wherein determining whether intent exists includes providing the data structure to an intent machine learning model, wherein the intent machine learning model is configured to infer intent or lack of intent based on the input data structure.

[0095] x. A method according to any one of embodiments i to ix, wherein determining the position and orientation includes providing the image to an image machine learning model, wherein the image machine learning model is configured to determine the position and orientation of a person in the image.

[0096] xi. A method according to embodiment x, wherein the image machine learning model is configured to determine a stick figure of the person based on the image.

[0097] xii. An intention determiner for determining an intention to open a door, the intention determiner comprising:

[0098] Processor; and

[0099] a memory storing instructions that, when executed by the processor, cause the intent determiner to:

[0100] obtaining an image of a physical space proximate the door;

[0101] determining a position and orientation of a person in the image;

[0102] adding an indicator of the position and an indicator of the orientation to a data structure;

[0103] determining whether there is an intention of the person to open the door based on the data structure; and

[0104] The instructions are repeated until the exit condition is true.

[0105] xiii. An intent determiner according to embodiment xii, wherein the instructions for adding the indicator include instructions that, when executed by the processor, cause the intent determiner to perform the following operations: send a signal to add the indicator while maintaining the position of the indicator in the sequence relative to any previous indicators in the data structure.

[0106] xiv. An intention determiner according to implementation xii or xiii, wherein the exit condition is true when the intention determiner determines that there is an intention of the person to open the door, and wherein the intention determiner also includes instructions that, when executed by the processor, cause the intention determiner to perform the following operations: send a signal to perform a door opening process.

[0107] xv. An intent determiner according to any one of embodiments xii to xiv, wherein the exit condition is true when the person is no longer determined to be in the image.

[0108] xvi. An intent determiner according to any one of embodiments xii to xv, wherein the instructions for determining the position and orientation include instructions that, when executed by the processor, cause the intent determiner to determine a bounding box and store the bounding box in the data structure, wherein the bounding box is a rectangle that just contains only the person in the image.

[0109] xvii. An intent determiner according to any one of embodiments xii to xvi, wherein the instructions for determining position and orientation include instructions that, when executed by the processor, cause the intent determiner to determine a center point of a person in the image, and wherein the center point is an indicator of the position.

[0110] xviii. An intention determiner according to any one of embodiments xii to xvii, wherein the instructions for determining the position and orientation include instructions that, when executed by the processor, cause the intention determiner to determine the direction of the person in the image, the direction of the person in the image indicating the direction of the person's chest, and wherein the direction of the person in the image is an indicator of the orientation.

[0111] xix. An intent determiner according to any one of embodiments xii to xviii, wherein the instructions for adding include instructions that, when executed by the processor, cause the intent determiner to add an indicator of time to the data structure in association with the indicator of the location and the indicator of the orientation.

[0112] xx. An intent determiner according to any one of embodiments xii to xix, wherein the instructions for determining whether intent exists include instructions that, when executed by the processor, cause the intent determiner to provide the data structure to an intent machine learning model, wherein the intent machine learning model is configured to infer intent or the lack of intent based on the input data structure.

[0113] xxi. An intent determiner according to any one of embodiments xii to xx, wherein the instructions for determining the position and orientation include instructions that, when executed by the processor, cause the intent determiner to provide the image to an image machine learning model, wherein the image machine learning model is configured to determine the position and orientation of a person in the image.

[0114] xxii. An intent determiner according to embodiment xxi, wherein the image machine learning model is configured to determine a stick figure of the person based on the image.

[0115] xxiii. A computer program for determining an intention to open a door, the computer program comprising computer program code which, when executed on the intention determiner, causes the intention determiner to:

[0116] obtaining an image of a physical space proximate to the door;

[0117] determining a position and orientation of a person in the image;

[0118] adding an indicator of the position and an indicator of the orientation to a data structure;

[0119] determining whether there is an intention of the person to open the door based on the data structure; and

[0120] The computer program code is repeated until an exit condition is true.

[0121] xxiv. A computer program product, comprising the computer program according to embodiment xxiii and a computer readable device, wherein the computer readable device comprises a non-transitory memory storing the computer program.

[0122] The above mainly describes various aspects of the present disclosure with reference to some embodiments. However, as will be readily appreciated by those skilled in the art, within the scope of the invention as defined by the appended patent claims, other embodiments other than the embodiments disclosed above are equally possible. Therefore, although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for illustrative purposes and are not intended to be limiting, with the true scope and spirit being indicated by the appended claims.

Claims

1. A method for determining an intention to open a door, the method being performed by an intention determiner (1), the method comprising: obtaining (40) an image of a physical space near the door; determining (42) a position and orientation of a person in the image by providing the image to an image machine learning model, wherein the image machine learning model is configured to determine the position and orientation of the person in the image, wherein the image machine learning model is configured to determine a stick figure of the person based on the image (9); adding (44) a data item to a data structure, the data item comprising an indicator of the position and an indicator of the orientation; determining (46) based on the data structure whether there is an intention by the person to open the door; and The method is repeated until the exit condition is true.

2. The method according to claim 1, wherein: In the step of adding (44) the data item, the data item comprises the coordinates of the anatomical feature represented by the stick figure (9).

3. The method according to claim 1 or 2, wherein: In the step of adding (44) the data item, the data item includes the orientation of at least one anatomical feature represented by the stick figure (9).

4. A method according to any one of the preceding claims, wherein: Adding (44) a data item includes adding the data item while maintaining the position of the data item in the sequence relative to any previous data item in the data structure.

5. A method according to any one of the preceding claims, wherein: When the intention determiner (1) determines that there is an intention of the person to open the door, the exit condition is true, and wherein the method further comprises: A signal (48) is sent to initiate the door opening process.

6. The method according to any one of claims 1 to 4, wherein: The exit condition is true when the person is no longer determined to be in the image.

7. The method according to any one of claims 1 to 6, wherein: Determining (42) the position and orientation includes determining a center point of the person in the image, and wherein the center point is an indicator of the position.

8. The method according to any one of claims 1 to 7, wherein: Determining (42) the position and orientation includes determining an orientation of the person in the image, the orientation of the person in the image indicating an orientation of the person's torso, and wherein the orientation of the person in the image is an indicator of the orientation.

9. The method according to any one of claims 1 to 8, wherein: Adding (44) the data item includes adding an indicator of a time to the data structure in association with the data item.

10. The method according to any one of claims 1 to 9, wherein: Determining (46) whether intent exists includes providing the data structure to an intent machine learning model, wherein the intent machine learning model is configured to infer intent, or the lack of intent, based on the input data structure.

11. An intention determiner (1) for determining an intention to open a door, the intention determiner (1) comprising: Processor (60); as well as A memory (64) storing instructions (67) which, when executed by the processor, cause the intent determiner (1) to: obtaining an image of a physical space proximate to the door; Determining a position and orientation of a person in the image by providing the image to an image machine learning model, wherein the image machine learning model is configured to determine the position and orientation of the person in the image, wherein the image machine learning model is configured to determine a stick figure of the person based on the image (9); adding a data item to a data structure, the data item comprising an indicator of the position and an indicator of the orientation; determining whether there is an intention of the person to open the door based on the data structure; and The instructions are repeated until the exit condition is true.

12. The intention determiner (1) according to claim 11, wherein: The data items include the coordinates of the anatomical features represented by the stick figure (9).

13. The intention determiner (1) according to claim 11 or 12, wherein: The data item includes the orientation of at least one anatomical feature represented by the stick figure (9).

14. The intention determiner (1) according to any one of claims 11 to 13, wherein: The instructions for adding the indicator include instructions (67) which, when executed by the processor, cause the intent determiner (1) to send a signal to add the data item while maintaining its position in the sequence relative to any previous indicators in the data structure.

15. The intention determiner (1) according to any one of claims 11 to 14, wherein: The exit condition is true when the intention determiner (1) determines that there is an intention of the person to open the door, and wherein the intention determiner (1) also includes instructions (67) that, when executed by the processor, cause the intention determiner (1) to perform the following operations: send a signal to perform a door opening process.

16. The intention determiner (1) according to any one of claims 11 to 14, wherein: The exit condition is true when the person is no longer determined to be in the image.

17. The intention determiner (1) according to any one of claims 11 to 16, wherein: The instructions for determining position and orientation include instructions (67) which, when executed by the processor, cause the intent determiner (1) to determine a center point of a person in the image, and wherein the center point is an indicator of the position.

18. The intention determiner (1) according to any one of claims 12 to 17, wherein: The instructions for determining position and orientation include instructions (67) that, when executed by the processor, cause the intention determiner (1) to determine a direction of a person in the image, the direction of the person in the image indicating a direction of the person's torso, and wherein the direction of the person in the image is an indicator of the orientation.

19. The intention determiner (1) according to any one of claims 12 to 18, wherein: The instructions for adding the data item include instructions (67) which, when executed by the processor, cause the intent determiner (1) to add an indicator of a time to the data structure in association with the data item.

20. The intention determiner (1) according to any one of claims 12 to 19, wherein: Instructions for determining whether intent exists include instructions (67) that, when executed by the processor, cause the intent determiner (1) to provide the data structure to an intent machine learning model, wherein the intent machine learning model is configured to infer intent, or the lack of intent, based on the input data structure.

21. A computer program (67, 91) for determining an intention to open a door, the computer program comprising computer program code which, when executed on an intention determiner (1), causes the intention determiner (1) to: obtaining an image of a physical space proximate to the door; Determining a position and orientation of a person in the image by providing the image to an image machine learning model, wherein: The image machine learning model is configured to determine a position and orientation of a person in the image, wherein the image machine learning model is configured to determine a stick figure of the person based on the image (9); adding a data item to a data structure, the data item comprising an indicator of the position and an indicator of the orientation; determining whether there is an intention of the person to open the door based on the data structure; and The computer program code is repeated until an exit condition is true.

22. A computer program product (64, 90) comprising a computer program according to claim 21 and a computer readable device comprising a non-transitory memory having the computer program stored therein.