Pre-determined gesture recognition methods

Abstract

A method for recognizing a predetermined gesture, such as a kick (16) or a hand gesture, for the purpose of controlling the opening and closing and / or locking or unlocking of an opening element of an automobile, the method being performed by a computing unit (22). The method includes the steps of: obtaining an electrical signal corresponding to the gesture from a non-contact motion sensor (18); determining at least three time windows in the electrical signal, wherein the at least three time windows correspond to the approach, stop, and retreat stages of the gesture; determining a value for at least one characteristic associated with the electrical signal for each of the time windows; and using a boosting classification algorithm to determine, based on a vector, which of the following classes the gesture belongs to: positive events indicating that the gesture corresponds to a predetermined gesture and negative events indicating that the gesture does not correspond to a predetermined gesture.

Description

Technical Field

[0001] The present disclosure relates to the field of methods for recognizing predetermined gestures aimed at controlling the opening and / or closing and / or locking or unlocking of opening elements of a motor vehicle, and using motion sensors based on contactless detection, particularly sensors for detecting body movements such as kicks.

Background Art

[0002] A predetermined gesture sensor including a sensor for recognizing a kick gesture (known as a kick sensor) can enable a user to activate or control certain vehicle functions such as opening the trunk of a motor vehicle by moving the foot or performing a gesture. This is a form of hands-free interaction designed for convenience and ease of use.

[0003] Instead of using a physical button of the motor vehicle or its keyring, the user can activate the mechanism by simply kicking under a specific area where a sensor is arranged on the rear bumper near the trunk. The sensor detects the movement and triggers the unlocking or opening of the trunk.

[0004] The purpose of the kick sensor is to enable the user to access the loading area of the vehicle without having to put down the luggage or physically touch the vehicle, for example when both hands are occupied with shopping bags.

[0005] Although the kick sensor is convenient and can provide hands-free access, it also has some drawbacks, particularly in terms of sensitivity. The kick sensor may sometimes be too sensitive or not sensitive enough, misinterpret other movements or objects, and cause unintended activation. For example, if a user or passerby accidentally brushes the sensor area when passing by the vehicle, it may trigger the opening and closing of the trunk.

[0006] Therefore, having a more reliable kick sensor for detecting motion would be beneficial. [Overview of the project] [Problems that the invention aims to solve]

[0007] This disclosure is intended to improve the situation. [Means for solving the problem]

[0008] A method is proposed for recognizing pre-determined gestures, such as kicks or hand gestures, for the purpose of controlling the opening and closing and / or locking or unlocking of an opening element of an automobile. The method is performed by a computing unit and includes the steps of: acquiring an electrical signal corresponding to the gesture from a non-contact motion sensor; determining at least three time windows in the electrical signal, wherein the at least three time windows correspond to the approach, stop, and retreat stages of the gesture; determining the value of at least one characteristic associated with the electrical signal for each of the at least three time windows; forming a vector containing the values ​​of the at least one characteristic of the electrical signal for each of the at least three time windows; and using a boosting classification algorithm to determine, based on the vector, which of the following classes the gesture belongs to: positive events indicating that the gesture corresponds to a pre-determined gesture and negative events indicating that the gesture does not correspond to a pre-determined gesture.

[0009] Gesture sensors, particularly kick sensors, have traditionally been developed using capacitive technology, i.e., capacitive touch technology, to detect and interpret kick gestures or foot movements. Capacitive sensors operate on the principle of capacitance, that is, the ability of an object to store an electrical load. In the context of kick sensors, capacitive technology is used to create an invisible capacitance field in a specific area, usually near the rear bumper of a vehicle. When a person's foot enters or interacts with this field, the charge is disturbed, and the sensor can detect the change in capacitance.

[0010] More recently, new sensor technologies based on radar technology (24GHz or UWB) have been combined with simple algorithms. Radar-based kick sensors use the principles of radar (radio waves) to detect and interpret kick gestures or foot movements. The sensor emits radio waves and analyzes their reflection to detect the presence and movement of a person's foot. A simple algorithm is designed to extract the characteristics of the signal associated with the reflected radio waves and compare them to thresholds (minimum / maximum) that represent a gesture due to positive foot movement.

[0011] The main weakness of simple algorithms is their lack of robustness to misuse, particularly to unintended passersby (in the case of foot sensors). In fact, when viewed individually, disturbances and the events being detected often share the same characteristics.

[0012] The proposed method decomposes the signal into time windows and uses the characteristics of each time window by a supervised learning algorithm to enable further analysis of the signal, thereby classifying gestures resulting from human foot movements as positive or negative according to these characteristics.

[0013] The proposed method suggests using combinations of feature values ​​to solve classification problems. Instead of testing all features in parallel, as with simple algorithms, supervised learning allows for the design of a feature test sequence in which the results of each test are weighted and combined to maximize the distinction between the classes to be identified.

[0014] The characteristics described in the following paragraphs can be implemented selectively, independently of each other, or in combination.

[0015] -For each of the three time windows, the at least one characteristic of the electrical signal is the following characteristic, namely: - Duration of the approach time window, - Duration of the stop time window, - Duration of the withdrawal time window, - The sum of the durations of the approach time window, stop time window, and retreat time window. - Number of points in the approach time window with a positive phase shift, - Number of points in the withdrawal time window with a negative phase shift, - The ratio of the number of points in the approach time window having a positive phase shift to the duration of the approach time window. - The ratio of the number of points in a retreat time window with a negative phase shift to the duration of the retreat time window. - Whenever the absolute value of the amplitude difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, the absolute number of fluctuations in the signal is determined by incrementing the counter by the absolute value of the amplitude difference between the two consecutive points. - Whenever the absolute value of the period difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, the absolute number of fluctuations in the signal period is determined by increasing the counter by the absolute value of the period difference between the two consecutive points. - Whenever the absolute value of the phase difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, the absolute number of phase variations of the signal is determined by incrementing the counter by the absolute value of the phase difference between the two consecutive points. - The ratio of the average signal amplitude in the approach time window to the average signal amplitude in the retreat time window. It includes at least two of the following.

[0016] - The motion sensor is a foot motion sensor located under the trunk of the vehicle when in use, and the sensor includes two antennas, namely a first antenna pointed toward the rear of the vehicle when in use and a second antenna pointed toward the underside of the vehicle when in use, and at least one characteristic of the electrical signal for each of the at least three time windows is, - The ratio of the average signal amplitude in the approach time window of the first antenna to the average signal amplitude in the approach time window of the second antenna, and - The ratio of the average signal amplitude in the retreat time window of the first antenna to the average signal amplitude in the retreat time window of the second antenna. Includes.

[0017] - A vector-based determination of whether a gesture belongs to one of two classes includes determining a weight associated with the gesture based on the characteristic vector, and comparing that weight with a threshold that determines whether the gesture corresponds to a positive gesture event.

[0018] - The classification algorithm is a boosting algorithm, such as AdaBoost (adaptive boosting), Gradient Boost, or XGBoost (extreme gradient boosting).

[0019] - The method also includes the step of issuing a command to open and close and / or lock or unlock an opening element of an automobile in response to determining a positive event indicating that the gesture corresponds to a predetermined gesture.

[0020] According to another aspect, a computer program is proposed that includes instructions for performing all or part of the method according to the present invention when the program is executed by a processor.

[0021] According to another aspect, a non-transitory computer-readable recording medium on which such a program is recorded is proposed.

[0022] According to another aspect, a method for opening the trunk of a motor vehicle includes using the above method to determine a gesture and issuing an instruction to open and / or close the trunk and / or lock or unlock it in response to determining a positive event indicating that the gesture corresponds to a pre-determined control gesture. According to one example, the instruction is issued by a computing unit separate from the computing unit that executes the method for determining the gesture.

[0023] According to another aspect, a motor vehicle provided with the above non-transitory recording medium is proposed.

[0024] Other features, details and advantages will become apparent from reading the following detailed description and analyzing the accompanying drawings.

Brief Description of the Drawings

[0025] [Figure 1] It is a diagram showing a state where a user opens the trunk of a motor vehicle by a foot gesture. [Figure 2] It is a diagram showing an example of an electrical signal from a non-contact gesture sensor after an antenna detects a gesture, and the signal is decomposed into a real part (Q) and an imaginary part (I). [Figure 3] It shows a graph of the phase, amplitude and period of the signal shown in FIG. 2. [Figure 4] It is a diagram showing the decomposition of the phase shift between the imaginary part (I) and the real part (Q) of the signal shown in FIG. 2 into a time window according to an embodiment. [Figure 5]Figure 2 shows an example of the distribution of positive and negative events after applying a boosting classification algorithm to the signal shown. [Modes for carrying out the invention]

[0026] Referring to Figure 1, it is shown that user 10 opens the trunk 12 of a vehicle 14 in this case, an automobile, by moving their foot 16. For this purpose, a non-contact sensor 18 is positioned at the rear of the automobile, under the trunk 12, near the bumper. When the user moves their foot 16 in front of the sensor 18 following a kick gesture, the trunk opening system 20 detects the kick and controls the opening of the trunk 12. The system 20 includes the sensor 18 and a computing unit 22 which includes a boosting classification algorithm for determining whether the gesture detected by the sensor 18 is indeed a kick gesture.

[0027] For example, the non-contact sensor 18 can be of two types. It can have one detection antenna (such as a UWB sensor) or two detection antennas (such as a 24GHz sensor). If the sensor 18 has two detection antennas, it can have a first antenna 18a facing the rear of the vehicle 14 (axis A1) and a second antenna 18b facing the underside of the vehicle 14 (axis A2). These two antennas are used in combination to detect the movement of the foot 16. The sensor 18 receives electrical signals from each antenna 18a, 18b corresponding to the detected movement. This signal has a real part Q and an imaginary part I, with a temporal phase shift P between them.

[0028] Next, a method 30 for determining a gesture event is described. This method determines whether a gesture recorded by sensor 18 corresponds to a predetermined control gesture. The predetermined gesture is preferably a kick, i.e., swinging the leg forward and then returning it to its initial position. In some variations, the gesture may be more complex, but advantageously, it includes an approach phase (towards vehicle 14), a stopping phase, and a retreat phase (a movement backward relative to vehicle 14). The gesture may also be a hand gesture of similar characteristics, possibly associated with a function of vehicle 14 other than the trunk opening.

[0029] Method 30 uses the computing unit 22 to analyze the signal received by the non-contact sensor 18 and determine whether a predetermined gesture, namely a kick, was actually performed. The classification performed by Method 30 is designed to avoid false positives or false negatives.

[0030] In the first step, an electrical signal corresponding to the gesture detected by sensor 18 is acquired. Figure 2 shows an example of the signal obtained for antenna 18a or 18b. This signal includes a real part (Q) and an imaginary part (I). Figure 3 shows the phase, amplitude, and period variations of the signal.

[0031] In the second step, the received signal is processed. Based on the signal, at least three time windows T1, T2, and T3 are determined. These three time windows correspond to the approach, stop, and retreat phases of the gesture, i.e., the approach, stop, and retreat phases of the foot in front of the sensor 18. To determine these windows, the phase shift between the imaginary (I) and real (Q) parts of the signal is analyzed. During the gesture, the phase is stably positive, then undergoes a loss of value, an inflection point appears, the phase becomes significantly negative, and then descends and stabilizes. Thus, the approach time window T1 corresponds to the time window in which the phase is positive and stable. The stop time window T2 is the time window adjacent to the approach time window T1, and therefore corresponds to the time window in which the phase undergoes a loss of value, an inflection point appears, the phase becomes significantly negative, and then descends and returns. The retreat time window T3 is the time window adjacent to the stop time window T2, and therefore corresponds to the time window in which the phase is negative and stable. These three time windows can be determined using a signal processing algorithm.

[0032] It is possible to have more than three time windows. For example, this breakdown could include an additional window before the approach window T4 shown in Figure 4, and an additional window after the retreat window T5 shown in Figure 4.

[0033] In the third step, the value of at least one characteristic of the electrical signal is determined for each of the at least three time windows. The characteristic is, - Duration of the approach time window, - Duration of the stop time window, - Duration of the withdrawal time window, - The sum of the durations of the approach time window, stop time window, and retreat time window. - Number of points in the approach time window with a positive phase shift, - Number of points in the withdrawal time window with a negative phase shift, - The ratio of the number of points in the approach time window having a positive phase shift to the duration of the approach time window. - The ratio of the number of points in a retreat time window with a negative phase shift to the duration of the retreat time window. Whenever the absolute value of the amplitude difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, the absolute variation of the signal (as a whole, i.e., including the real and imaginary parts) is determined by increasing the counter by the absolute value of the amplitude difference between the two consecutive points. For example, this threshold is 30. - Whenever the absolute value of the period difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, this is the absolute variation in the period of the signal, determined by incrementing the counter by the absolute value of the period difference between two consecutive points. For example, this threshold is 2, - Whenever the absolute value of the phase difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, This is the absolute number of phase variations of a signal, determined by increasing the counter by the absolute value of the phase difference between two consecutive points. For example, this threshold is 2.

[0034] If sensor 18 has two antennas, - The ratio of the average amplitude of the signal in the approach time window (including the real and imaginary parts as a whole) to the average amplitude of the signal in the retreat time window, where, for example, the signal is modified to be centered at 0 by a high-pass filter, the ratio, - The ratio of the average signal amplitude in the approach time window of the first antenna to the average signal amplitude in the approach time window of the second antenna. - The ratio of the average amplitude of the signal in the retreat time window of the first antenna (including both the real and imaginary parts) to the average amplitude of the signal in the retreat time window of the second antenna, where, for example, the signal is modified to be centered at 0 by a high-pass filter. Other characteristics can also be considered.

[0035] In the fourth step, a vector is formed from the values ​​of the electrical signal characteristics determined for each time window. This vector is in the form of an N×1 vector of dimension 1 and has N characteristics of this signal.

[0036] In the fifth step, the boosting classification algorithm determines, based on the vector, which of the following two classes the pre-determined gesture belongs to: a positive event that includes the pre-determined gesture (if the gesture is an actual kick) or a negative event that does not correspond to the pre-determined gesture (i.e., if it is not a kick).

[0037] The classification algorithm is a boosting ensemble algorithm that classifies data into two classes.

[0038] Boosting algorithms are automated learning techniques that combine predictions from several weak or base models to create a stronger, more accurate predictive model. Boosting is a type of ensemble learning method in which several models are formed and combined to make better predictions than the individual models.

[0039] In boosting, the base model is generally simple and weak, such as a decision tree with a limited or shallow neural network. Each base model is trained on a subset of the training data, or by assigning different weights to data points. The models are formed iteratively, with each subsequent model focusing on instances that were not properly classified by the preceding model.

[0040] During the training process, reinforcement assigns higher weights to examples that were not properly classified, effectively increasing their importance. Subsequent models then train preferentially on these poorly classified examples by attempting to correct the errors made by the preceding models. This iterative process continues until predefined stopping criteria are met, such as the maximum number of models or the target accuracy level.

[0041] The final prediction is typically made by combining the predictions of all the base models, usually via a weighted voting system. The weights are determined based on the performance of each base model during training. The idea behind boosting is that by combining multiple weak models, each focusing on a different aspect of the data, the global model can capture complex relationships and improve predictive accuracy.

[0042] Common boosting algorithms include AdaBoost (adaptive boosting), Gradient Boost, and XGBoost (extreme gradient boosting).

[0043] Referring to Figure 5, the classification algorithm outputs weights. If these weights are greater than the threshold (shown by the dotted line), it is concluded that a positive kick event occurred. If these weights are less than the threshold, it is concluded that a negative event, not corresponding to a kick, occurred.

[0044] The threshold is determined upstream of method 30 during the training and calibration of the classification algorithm.

[0045] The classification algorithm may or may not use the N properties provided to it to determine the weights of the events.

[0046] Using a boosting classification algorithm reduces the size of the gesture detection problem. The problem would not have been solvable if each characteristic had been considered individually. Boosting allows for sequential testing, adding weights, resulting in only one dimension remaining, and ultimately performing the classification.

[0047] At the end of this process, if the gesture is determined to correspond to a pre-determined control gesture (positive event), a command can be issued to open or close the trunk, and / or lock or unlock the trunk. On the other hand, if the gesture is determined not to correspond to a pre-determined control gesture (negative event), no command is issued.

[0048] The issuance of instructions can be controlled by the computing unit 22, or by another computing unit connected to the computing unit 22.

Claims

1. A method for recognizing a predetermined gesture performed by a user, for the purpose of controlling the opening or closing and / or locking or unlocking of an opening element of an automobile when a predetermined gesture such as a kick (16) or a hand gesture is recognized, wherein the method is performed by a computing unit (22), a. A step of acquiring an electrical signal from a non-contact motion sensor (18), wherein the electrical signal corresponds to a gesture, b. A step of determining at least three time windows (T1, T2, T3) in the electrical signal, wherein the at least three time windows correspond to the approaches, stops, and retreats of the gesture, c. A step of determining the value of at least one characteristic associated with the electrical signal for each of the at least three time windows, d. The step of forming a vector that includes the value of the at least one characteristic of the electrical signal for each of the at least three time windows, e. Using a boosting classification algorithm, determine, based on the vector, which of the following two classes the gesture belongs to: a positive event indicating that the gesture corresponds to the pre-determined gesture, and a negative event indicating that the gesture does not correspond to the pre-determined gesture. A method that includes this.

2. For each of the three time windows, the at least one characteristic of the electrical signal is the following characteristic, namely, - Duration of the approach time window, - Duration of the stop time window, - Duration of the withdrawal time window, - The sum of the durations of the approach time window, stop time window and retreat time window, - The number of points in the approach time window having a positive phase shift, - The number of points in the withdrawal time window having a negative phase shift, - The ratio of the number of points in the approach time window having a positive phase shift to the duration of the approach time window, - The ratio of the number of points in the withdrawal time window having a negative phase shift to the duration of the withdrawal time window, - Whenever the absolute value of the amplitude difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, the absolute number of signal fluctuations is determined by increasing the counter by the absolute value of the amplitude difference between the two consecutive points. - Whenever the absolute value of the period difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, the absolute number of fluctuations in the signal period is determined by increasing the counter by the absolute value of the period difference between the two consecutive points. - Whenever the absolute value of the phase difference between two consecutive points is greater than a statistically defined threshold corresponding to an abnormal amplitude increase between two consecutive points, the absolute number of phase variations of the signal is determined by increasing the counter by the absolute value of the phase difference between the two consecutive points. - The ratio of the average signal amplitude in the approach time window to the average signal amplitude in the retreat time window. The method according to claim 1, comprising at least two of the above.

3. The motion sensor is a foot motion sensor that is positioned under the trunk of a car when in use, and the sensor includes two antennas, namely a first antenna directed toward the rear of the car when in use and a second antenna directed toward the underside of the car when in use, and at least one characteristic of the electrical signal for each of the at least three time windows is - The ratio of the average signal amplitude in the approach time window of the first antenna to the average signal amplitude in the approach time window of the second antenna, and - The ratio of the average signal amplitude in the withdrawal time window of the first antenna to the average signal amplitude in the withdrawal time window of the second antenna The method according to claim 1 or 2, including the method described in claim 1 or 2.

4. The method according to any one of claims 1 to 3, wherein the determination based on the vector of whether the gesture belongs to one of the two classes includes determining a weight associated with the gesture based on the characteristic vector, and comparing the weight to a threshold to determine whether the gesture corresponds to a positive gesture event.

5. The method according to any one of claims 1 to 4, wherein the classification algorithm is a boosting algorithm, such as adaptive boosting, gradient boosting, or extreme gradient boosting.

6. A computer program, which, when executed by a processor, includes instructions for performing the method described in any one of claims 1 to 5.

7. A non-temporary computer-readable recording medium, wherein, when the program is executed by a processor, the program is recorded on the recording medium in order to perform the method according to any one of claims 1 to 5.

8. A method for opening the trunk (12) of an automobile (14), comprising using the method for determining a gesture according to any one of claims 1 to 5, and issuing a command to open and close and / or lock or unlock the trunk in response to determining the positive event indicating that the gesture corresponds to the pre-determined control gesture.

9. The method for releasing according to claim 8, wherein the instruction is issued by a computing unit other than the computing unit that performs the method for determining the gesture.

10. An automobile (14) equipped with a non-temporary recording medium as described in claim 7.

Images