Method and device for unlocking an opening element

Abstract

The invention relates to a method for unlocking an opening element (2), comprising the following steps: - observing a user authorized to open the opening element (2) and approaching the opening element (2) by means of a video image sensor (4), - detecting the skeleton of the forearm (7) of the user and its movements by analyzing the video image, - evaluating a first condition: the forearm (7) approaches in the direction of the position (9) in which the handle (6) of the opening element (2) is located, - evaluating a second condition: the hand (8) is at a distance from the position (9) in which the handle (6) of the opening element (2) is located which is less than a given distance threshold, - evaluating a third condition: biometric verification of the user, - if all evaluated conditions are met, the opening element (2) is unlocked.

Description

Technical Field

[0001] This invention relates to a method and apparatus for unlocking opening elements. The invention is applicable to any opening element where unlocking is intended to be restricted to at least one privileged user. More specifically, the invention applies to motor vehicles where access is intended to be restricted to the owner and / or persons authorized by the owner. In this context, opening elements refer to the vehicle's left front door, right front door, left rear door, right rear door, tailgate window, trunk lid, engine hood, and even charging port or fuel filler cap covers. Background Technology

[0002] As is well known, badges can be used to control the unlocking of opening elements. These badges may be integrated into smartphones and paired with the opening element or a system containing the opening element (such as a motor vehicle). The system detects the presence of the badge nearby without contact to control the unlocking of the opening element. This method has at least two drawbacks: first, badge detection cannot verify whether the badge holder is truly authorized; second, the system cannot detect the true intent to open the door. If someone obtains the badge fraudulently, the system will allow them to open the door. Similarly, even if the user merely approaches or lingers near the opening element, the proximity of the badge can control unlocking. This would cause the system to unlock the opening element immediately upon detecting the badge and then relock it if it remains unopened for a period of time. This could create a security vulnerability and, at the very least, result in unnecessary energy consumption.

[0003] To detect unlocking intent, it is also known to place hand-detecting sensors on or within the handle of the opening element. This method has at least two drawbacks. The first drawback is that hand detection typically cannot verify whether the hand belongs to an authorized person. The second drawback is that this method requires the addition of sensors. This additional cost limits its use to primary opening elements: in the case of a vehicle, namely the driver's and passenger's doors, excluding other opening elements such as rear doors, trunks, and covers.

[0004] Therefore, a method and apparatus for unlocking opening elements are sought to correct these shortcomings by detecting the true intent of unlocking and verifying its author. Summary of the Invention

[0005] Therefore, the present invention relates to a method for unlocking an opening element, comprising the following steps: - Observe the user who is authorized to open the opening element (2) and approach the opening element via a video image sensor. - Detect the user's arm skeleton and its movements by analyzing video images. - First condition for evaluation: The arm approaches the location of the handle of the opening element. - Evaluate the second condition: the distance between the hand and the handle of the opening element at the stated position is less than a given distance threshold. - Evaluation of the third condition: biometric verification of the user. - If all the evaluation conditions are met, the opening element is unlocked.

[0006] The specific features or implementation schemes used alone or in combination are as follows: - Detect the arm skeleton and its movements using a pose analysis algorithm. - If the arm skeletal segment points towards the handle with the hand forward, then the first condition is satisfied. - The distance threshold for the second condition is equal to 50 cm, preferably 30 cm. - The biometric verification is facial recognition or gait recognition, or less preferably voice recognition, fingerprint recognition, or iris scanning recognition, or at least two of the aforementioned recognition methods. The method also includes evaluating a fourth condition: preventing identity theft. - Implement measures to prevent identity theft in accordance with FIDO (for “rapid online identification”) Level B standard or ISO 30107 Level 2 standard. - The method further includes evaluating a fifth condition: detecting a nearby authorized badge, which is preferably associated with a user who has performed biometric verification through the third condition.

[0007] According to another aspect, the present invention covers an apparatus for unlocking an opening element, comprising a processing unit, at least one video image sensor connected to the processing unit, and at least one opening element unlocking actuator connected to the processing unit, the processing unit being capable of implementing the method described herein to control the unlocking actuator to unlock the opening element. Attached Figure Description

[0008] The invention will be better understood by reading the following description, which is given by way of example only, and by referring to the accompanying drawings in the appendix, wherein: [ Figure 1 The diagram illustrates the position of the user's arm and hand near the handle of the opening element. [ Figure 2 The conditions for verifying the first condition are illustrated in the diagram. [ Figure 3 The composition of the device according to the present invention is illustrated in a schematic diagram. Detailed Implementation

[0009] refer to Figure 1This invention relates to a method for unlocking an opening element 2. The method includes the following steps. In the first step, a user approaching the vehicle opening element 2 is observed by a video image sensor 4 integrated in the vehicle (see...). Figure 3 Based on the video images captured by sensor 4, image analysis is performed to detect the user's skeleton and its movement. More specifically, the analysis focuses on the user's posture, and even more so on detecting the position of the user's forearm 7 and hand 8 skeleton. This analysis detects the position and movement of said forearm 7.

[0010] Based on this detection, the first condition is evaluated. Is the forearm 7 close to the position 9 of the handle 6 of the opening element 2? This first condition is considered to indicate that the user intends to unlock.

[0011] Based on this detection, the second condition indicating proximity is evaluated. It is verified that the distance between the hand 8 detected by the skeleton detection algorithm in the previous step and the location 9 of the handle 6 of the opening 2 is less than a given distance threshold. If so, the condition is considered satisfied.

[0012] To ensure that the forearm 7 of the handle 6 approaching the opening element 2 is indeed the forearm of an authorized person, a third condition needs to be evaluated: biometric verification of the user.

[0013] As will be described later, other conditions may be added to the method. If all the evaluated conditions (intersection or logical AND) are satisfied, the method controls the unlocking of the opening element 2.

[0014] The video image sensor 4 can be of any type. It can be a single camera equipped with a wide-angle lens or a conical lens to provide a panoramic view around the opening element 2 or a system including at least one opening element 2, such as a vehicle.

[0015] In driver assistance or autonomous driving scenarios, vehicles are advantageously equipped with such video image sensors 4. Therefore, this may involve placing several (typically four) cameras around the vehicle to provide a panoramic view of the vehicle's surroundings.

[0016] Therefore, in the case of panoramic sensor 4 or multiple sensors forming an existing panoramic sensor 4, the latter can be advantageously reused. Thus, in this case, the present invention does not require a specific sensor.

[0017] Based on another feature, it is assumed that the detection of the skeleton and its motion is known. This can be performed, for example, using a POSE-type pose analysis algorithm (POSE being the English term for "position and pose"). Several implementations of such algorithms exist: YOLO, MediaPipe-Pose, OpenPose, etc. For a clearer explanation, you can visit the following website: https: / / viso.ai / deep-learning / pose-estimation-ultimate-overview / .

[0018] As mentioned earlier, known algorithms can be used to identify the user's skeleton approaching the opening element 2. When the user enters the field of view of the sensor 4, it is considered that they are approaching the opening element 2.

[0019] according to Figure 2 Another feature, shown more specifically in the diagram, is that the first condition is satisfied if the skeletal segment of the forearm 7 points towards the position 9 where the handle 6 is located. Further advantageously, it is confirmed that the hand 8 in front of the forearm 7 also points towards the handle 6. For this purpose, as... Figure 2 As shown, it can be verified that the forearm 7 points towards the area 9 around the handle 6 when extended. Therefore, as Figure 2 As shown, some configurations of the forearm 7 are effective, while others are not, because they do not clearly indicate the user's intention to bring their hand 8 close to the handle 6.

[0020] There are at least two reasons for using the concept of position 9 for handle 6 in this specification. The first reason is that position 9 is an expanded area around the effective handle 6 in order to create greater tolerance for verifying the first condition.

[0021] The second reason is that the handle 6 is virtual. In fact, due to the present invention, the handle 6 can be removed from the opening element 2, which is the opening member. Position 9 is an area that is not necessarily materialized but is known to the user, who reaches out to this area to indicate their intention to open the opening element 2.

[0022] According to the embodiment without handle 6, the present invention allows the opening element 2 to be unlocked, making it partially open. The user then places their hand behind the panel of the opening element 2. Grasping this panel serves as a handle to open the opening element 2. It can be noted that the engine compartment, air inlet cover, or fuel filler cap are not equipped with handle 6.

[0023] Based on another feature, a distance threshold for the second condition is selected such that it indicates the true approach of hand 8 and confirms the genuine intent to unlock. The threshold is a trade-off. A higher threshold advantageously allows for earlier detection, thus providing time for subsequent processing and ensuring good response time for device 1. Conversely, a high threshold tends to increase the number of false detections, which is disadvantageous. As an indication, this threshold is set to be equal to 50 cm. Preferably, it is set to be equal to 30 cm.

[0024] The first two conditions detect the user's intent to unlock. However, it is necessary to verify whether the user is authorized to request unlocking. Therefore, based on another feature, at least one biometric verification is performed.

[0025] This verification is best performed using facial recognition or gait recognition. In fact, both verifications can advantageously be performed using the same video image sensor 4. In this case, verification is performed during user approach, advantageously in parallel with skeleton detection, or even before skeleton detection.

[0026] Alternatively, biometric verification can also be performed using voice recognition, fingerprint recognition, or iris scanning. These methods are less popular because they require at least one additional sensor: a microphone for voice, a contact sensor for fingerprints, or an eye imaging sensor for iris scanning.

[0027] Based on another characteristic, and particularly to enhance the security of identification, two (or multiple) authentication methods can be implemented. To achieve this, at least two of the aforementioned identification steps must be performed.

[0028] Based on another feature, the security of the authentication method is further enhanced by adding a fourth condition to prevent identity theft.

[0029] This condition is designed to prevent attempts to steal identity, such as presenting a photo of an authorized person or using a mask instead of the face to deceive facial recognition.

[0030] This protection against identity theft is based on either Level B of the FIDO ("Rapid Online Identification") standard or Level 2 of the ISO 30107 standard.

[0031] According to another feature, the method may include evaluating a fifth condition. This condition includes detecting a nearby authorized badge. This badge is preferably associated with a user individual who has undergone biometric verification via a third condition. The combination of these two conditions provides enhanced security.

[0032] As described above, the method is advantageously applicable to all opening elements 2 of the vehicle. In fact, image and motion analysis of the skeleton enables the detection of the forearm 7 approaching any opening element 2 by identifying each opening element 2. Then, orderly unlocking is advantageously limited to the opening elements 2 in question.

[0033] Conversely, similar to centralized unlocking, unlocking can be extended to all unlocking elements 2 when an intention to unlock an unlocking element is detected.

[0034] The present invention also relates to a device 1 for unlocking the opening element 2. This device 1, more specifically, is as follows: Figure 3 As shown, the device includes a processing unit 3, at least one video image sensor 4 connected to the processing unit 3, and at least one unlocking actuator 5 for connecting to an opening element 2 of the processing unit 3. The video image sensor 4 is capable of capturing video images of a user approaching the device 1. The processing unit 3 is capable of performing various processing operations, and in particular, is capable of analyzing video images and analyzing skeletal motion.

[0035] According to a feature, the processing unit 3 can implement the above method to control the unlocking actuator 5 in order to unlock the opening element 2.

[0036] The present invention has been detailed and described in the accompanying drawings and the foregoing description. The latter should be considered illustrative and given by way of example, and should not be construed as limiting the invention to this specification. Many design variations are possible.

Claims

1. A method for unlocking an opening element (2), Its features This includes the following steps: - Observe the user who is authorized to open the opening element (2) and approach the opening element (2) via the video image sensor (4). - The user's forearm (7) skeleton and its movement are detected by analyzing video images. - Evaluation of the first condition: The forearm (7) approaches the location (9) of the handle (6) of the opening element (2). - Evaluation of the second condition: The distance between the hand (8) and the handle (6) of the opening element (2) at the location (9) is less than a given distance threshold. - Evaluation of the third condition: biometric verification of the user. - If all the evaluation conditions are met, the opening element (2) is unlocked.

2. The method according to claim 1, wherein the detection of the forearm (7) skeleton and its motion is performed by a posture analysis algorithm.

3. The method according to any one of claims 1 or 2, wherein the first condition is satisfied if the forearm (7) skeletal segment points to the position (9) where the handle (6) is located when the hand (8) is forward.

4. The method according to any one of claims 1 to 3, wherein the distance threshold of the second condition is equal to 50 cm, preferably 30 cm.

5. The method according to any one of claims 1 to 4, wherein the biometric verification is facial recognition or gait recognition, or less preferably voice recognition or fingerprint recognition or iris scanning recognition, or at least two of the aforementioned recognition methods.

6. The method according to any one of claims 1 to 5, further comprising evaluating a fourth condition: preventing identity theft.

7. The method according to any one of claims 1 to 6, further comprising evaluating a fifth condition: detecting a nearby authorization badge, preferably associated with a user who has performed biometric verification through the third condition.

8. An apparatus (1) for unlocking an opening element (2), the apparatus comprising a processing unit (3), at least one video image sensor (4) connected to the processing unit (3), and at least one unlocking actuator (5) for the opening element (2) connected to the processing unit (3). Its features The processing unit (3) is capable of implementing the method described in this document to control the unlocking actuator (5) to unlock the opening element (2).

Images