Method for determining a gesture
Patent Information
- Application Number
- PCT/EP2024/076185
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-09-22
- Filing Date
- 2024-09-19
- Publication Date
- 2025-03-27
Smart Images

Figure EP2024076185_27032025_PF_FP_ABST
Abstract
Description
Method of determining a gesture Description Technical field
[0001] The present disclosure relates to the field of methods for determining a gesture, such as a kick or a hand strike, using motion sensors, and in particular contactless motion sensors. Prior art
[0002] A kick sensor can allow a user to activate or control certain vehicle functions, such as opening a car trunk, by making a movement or gesture with their foot. It is a form of hands-free interaction designed to provide convenience and ease of use.
[0003] Instead of using a physical button on the car or its key fob, the user can activate the mechanism by simply making a kicking gesture under a specific area of the rear bumper near the trunk, where the sensor is located. The sensor detects the movement and triggers the unlocking and / or opening of the trunk.
[0004] The purpose of a kick sensor is to allow the user to access the vehicle's cargo area without requiring them to put their belongings down or physically interact with the car, such as when their hands are full with grocery bags.
[0005] While kick sensors can be convenient and provide hands-free access, they also have some drawbacks, particularly regarding sensitivity. Kick sensors can sometimes be too sensitive or not sensitive enough, and can then misinterpret other movements or objects and lead to unintentional activations. For example, if the user or a passerby accidentally brushes against the sensor area while walking past the vehicle, it can trigger the trunk to open or close. Summary
[0006] This disclosure improves the situation.
[0007] A method for determining a predetermined gesture, such as a kick or a hand strike, is proposed, the method being implemented by a computing unit connected to a first sensor and a second radiofrequency type motion sensor, arranged in use in proximity to each other on a motor vehicle, the first and a second sensor being adapted to produce respectively a first electrical signal and a second electrical signal upon the occurrence of a gesture in a detection zone, the method comprising: using a classification algorithm for each of the first and second electrical signals in order to obtain respectively first and second scores associated respectively with the first and second electrical signals, as well as, for each of the first and second scores, respective probability densities that the gesture belongs to at least two categories among a first category and a second category as a function of the value of the score, one among the first category and the second category corresponding to the determination that the gesture is the predetermined gesture,each classification algorithm having been trained individually for each of the first and second motion sensors so as to obtain: a probability density distribution that the gesture belongs to said first category as a function of the value of the score, and a probability density distribution that the gesture belongs to said second category as a function of the value of the score, and a first range of values of the score, associated with a gesture considered as belonging to the first category, a second range of values of the score, associated with a gesture considered as belonging to the second category, and a third range of values of the score, associated with a gesture considered as indeterminate; and determining that the gesture belongs to the first or second category, as a function of whether the first and second scores belong to the first, second and third ranges and the temporal order of determination of the first and second scores.,
[0008] According to the invention, determining that the gesture is of the first or second category comprises the following steps, with the first score being obtained first: - the first score belongs to the third range, starting a second clock; As long as the second clock has not reached the end of a second time window: When the second signal arrives at the calculation unit: obtaining the second score; Determining whether the second score belongs to the first range, second range or third range; If the second score belongs to the first range, determining that the gesture is of the first category; If the second score belongs to the second range, determining that the gesture is of the second category; If the second score belongs to the third range, the gesture is that of the category having the highest probability density between the first and second categories for the first and second scores.
[0009] According to another aspect, there is provided a computer program comprising instructions for implementing all or part of a method as defined herein when this program is executed by a processor. According to another aspect, there is provided a non-transitory, computer-readable recording medium on which such a program is recorded.
[0010] The features set forth in the following paragraphs may optionally be implemented, independently of one another or in combination with one another: - the first score is obtained first, and determining whether the gesture is of the first or second category includes: determining whether the first score belongs to the first range, second range, or third range; if the first score belongs to the first range, determining that the gesture is of the first category; and if the first score belongs to the second range, determining that the gesture is of the second category.
[0011] - once the gesture is determined to be of the first or second category, when a first clock reaches the end of a first time window, a reset of a memory containing information on the determination of the gesture, and a reset of the first clock.
[0012] - the second signal does not arrive at the computing unit before the end of the second time window: Comparison of the first score with a first threshold, the first score is lower than the first threshold, determination that the gesture is of the first category; If the first score is higher than the first threshold, determination that the gesture is of the second category.
[0013] - when the gesture is determined and the second clock reaches the end of the second time window, a reset of a memory containing information on the determination of the gesture, and a reset of the first clock.
[0014] - the predetermined gesture is a kick, and the detection zone is a rear part of a car vertically below a trunk of the car, the method further comprising sending a signal from the computing unit to a control unit, to actuate the opening or closing of the trunk when the gesture is of the second category.
[0015] - when the predetermined gesture is not a kick, a failure to send a signal from the computing unit to the control unit. Brief description of the drawings
[0016] Other features, details and advantages will become apparent upon reading the detailed description below, and upon analyzing the attached drawings, in which:
[0017] [Fig. 1] shows an illustration of a user opening a car trunk with a foot gesture, the car being equipped according to one embodiment with two contactless sensors arranged in a detection zone below the car trunk;
[0018] [Fig. 2] shows an example of score distribution for a first of the two sensors, each score giving a probability that the gesture is of a first category Y or of a second category X;
[0019] [Fig. 3] shows an example of score distribution for a second of the two sensors, each score giving a probability that the gesture is of a first category Y or of a second category X;
[0020] [Fig. 4] shows an example of signal processing for the graph in [Fig. 2];
[0021] [Fig. 5] shows an example of signal processing for the graph in [Fig. 3];
[0022] [Fig. 6] shows a flowchart representing a gesture determination process using the scores of the two contactless sensors. Description of the embodiments
[0023] Reference is now made to [Fig. 1] which shows a user 10 opening the trunk 12 of a vehicle 14, in this case a car, using the action of his foot 16. For this, two contactless sensors (first sensor 18, second sensor 19) are arranged at the rear of the car below the trunk 12 at the bumper, for example at two rear corners of the vehicle as illustrated in FIG. 1. It is by passing his foot 16 in a detection zone ZD in front of one or both sensors 18, 19, that a system 20 for opening the trunk 12 detects whether the gesture is a kick in order to control the opening and / or closing of the trunk 12. The system 20 includes a calculation unit 22 which determines whether the detected gesture is actually a kick gesture based on the electrical signals provided by the sensors 18, 19. The detection of this gesture can then be used to operate a control of the vehicle, such as the opening from trunk 12.
[0024] In the example of [Fig. 1], each of the sensors 18, 19 is arranged substantially at a lateral end of the vehicle body 14 below the bumper. They are positioned so as to capture a movement which would be located under the bumper, towards the middle laterally of the bumper, such as the gesture of the user's foot illustrated in [Fig. 1]. It could be that the sensors 18, 19 are arranged more or less close to each other laterally, as long as a minimum threshold distance between them is respected, so that they provide distinct signatures. The vehicle 14 may be equipped with more than two rear sensors 18, 19 that can be used for kick detection according to the method described below. The vehicle 14 could, for example, have three or four.
[0025] The non-contact sensors 18, 19 may be, according to one example, of two types. They may each have one detection antenna (such as a UWB sensor) or two detection antennas (for example, a 24 GHz sensor). In the case where the sensor 18 has two detection antennas, the sensor 18 may have a first antenna directed towards the rear of the vehicle 14, and a second antenna directed towards the underside of the vehicle 14. The sensor 18 receives from each antenna an electrical signal corresponding to the detected movement. This signal comprises a real part I and an imaginary part Q which have a time shift P between them. The sensors could also be CW (continuous wave) type sensors, based on radar technology and have two antennas.
[0026] The electrical signal from each sensor can then be used to determine whether what has been detected is a kicking gesture. For this purpose, a two-category classification algorithm is associated with each sensor to assist in determining the gesture based on the signal: a first category Y for a gesture categorized as not being a kick, and a second category X for a gesture being categorized as a kick. Although reference is made to a kick, the method and systems described herein apply to other body gestures, such as a hand clap and to the control of elements other than the trunk. Furthermore, although only two categories are discussed, it is possible that the classification could be done with more than two categories.
[0027] The sensors are autonomous and each have their own classification algorithm trained individually for each sensor. The classification algorithm can be a naive algorithm based on the SNR (signal to noise ratio) value or a supervised machine learning classifier (e.g. boosting).
[0028] We therefore understand that the score according to the invention designates a number which can be: - a signal-to-noise ratio value (or a value directly derived from the signal-to-noise ratio value); or - a value, returned by a supervised machine learning classifier, particularly of the "boosting" type, and used to determine membership in a class. In the particular case of a "boosting" type algorithm, the score designates a sum of elementary values each resulting from a predetermined test carried out on at least one parameter of the signal from the sensor. A test here designates a comparison with a predetermined threshold separating data associated with a class X and data associated with a class Y.
[0029] The same classification algorithm may be used for each of the two sensors 18, 19, or different classification algorithms may be used for each of the two sensors 18, 19. The classification algorithm may be executed by the computing unit 22. The classification algorithm could for each sensor be executed by a computing unit integrated with that sensor, and these computing units could be in contact with the computing unit 22 which determines whether the gesture is definitely a kick based on the classification obtained by each sensor, as will be explained below.
[0030] Referring now to [Fig. 2], an example of reference values obtained after training the classification algorithm for the first sensor 18 is illustrated. The reference values include the distribution of a score (abscissa) and the probability of occurrence (ordinate) of the first gesture category Y (gesture categorized as non-kick) and the second gesture category X (gesture categorized as kick). [Fig. 3] presents the same graph of reference values after training the classification algorithm but for the second sensor 19. For each sensor, the distribution of scores for each gesture category is Gaussian-shaped. The two Gaussians (gesture of category X and Y of the same sensor) are offset from each other along the abscissa. In the example of the figures, the scores are distributed over positive and negative values on the abscissa.It could be that the scores are distributed only over positive values or only over negative values.
[0031] It is understood that training the classification algorithm for the first sensor 18, respectively for the second sensor 19, refers to adjusting the parameters of the algorithm by using it with a plurality of gestures of known category X or Y. The adjustment of the parameters aims to minimize a classification error rate of an unknown gesture, during the subsequent use of the classification algorithm. In the case where the score is a signal-to-noise ratio (or a function of the signal-to-noise), the training consists of determining at least one threshold separating two classes X and Y. In the case where the score is a value returned by a supervised machine learning classifier, the training consists of determining a plurality of thresholds associated with a plurality of tests carried out on at least one parameter of the signal from the sensor, each threshold separating two classes X and Y within the framework of a given test. A classification algorithm trained individually for a given motion sensor, means that the training is carried out by taking into consideration the sensor data from this single motion sensor, independently of data from another motion sensor.
[0032] The Gaussians for the X and Y gesture distributions of the same sensor have vertices S1, S2 that are offset from each other. The Gaussians are a characterization of the individual performance of each of the sensors. The scoring method is considered to be sufficiently effective when the separation of the distributions is above a threshold value, for example less than 25% overlap between the two distributions. Thus, in one example, vertex S1 of the Gaussian representing the first gesture category Y has a lower score than vertex S2 of the Gaussian representing the second gesture category X, and this for each sensor. It could be the other way around. If we look at the example in [Fig. 2], vertex S1 of the Gaussian representing gesture category Y for sensor 18 has a score of 0.25, and vertex S2 of the Gaussian representing gesture category X for sensor 18 has a score of 6.Looking at the example in [Fig. 3], vertex S1 of the Gaussian representing gesture category Y for sensor 19 has a score of -0.6, and vertex S2 of the Gaussian representing gesture category X for sensor 18 has a score of 2.
[0033] The score distributions make it possible to determine a probability that the detected gesture is considered a kick by each of the sensors 18, 19. This being the case, the individual data of the sensors 18, 19 alone do not make it possible to effectively determine whether the gesture is actually a kick. For this and with reference to [Fig. 6], a gesture determination method 30 makes it possible to arbitrate according to the scores SC1, SC2, the probability of occurrence of the gesture categories X, Y and the temporal order of determination of the scores (which may also be the temporal order of arrival of the electrical signals from the sensors to the calculation unit 22), when two or more sensors are used, as in the example of the figures. When it is determined that the gesture is a kick, the calculation unit 22 can then communicate with a control unit to activate the opening and / or closing of the trunk 12.If, on the contrary, it is determined that the gesture is not a kick, no signal is sent to the control unit and the trunk 12 is not activated.
[0034] In a signal processing (or calibration) phase preliminary to the method 30, and with reference to [Fig. 4] and [Fig. 5], each distribution of scores of the algorithm trained for each of the two sensors 18, 19 is divided into three ranges of values Z1, Z2 and Z3 of scores.
[0035] The first value range Z1 is a window of scores for which it is almost certain that the detected gesture is of the first category Y (i.e. not a kick). This is a range of values referred to as "high confidence". The second value range Z2 is a window of scores for which it is almost certain that the detected gesture is of the second category X (i.e. kick). This is also a range of values referred to as "high confidence". The third value range Z3 is a window of scores for which there is no conclusion as to whether the detected gesture is of the first X or second Y category. This is a range of values referred to as "low confidence".
[0036] To delimit the value ranges Z1, Z2, Z3, thresholds A-TH-MIN and A-TH-MAX are chosen. The threshold A-TH-MIN delimits the first from the third value range, and the threshold A-TH-MAX delimits the third from the second value range. The first Z1 and second Z2 value ranges are therefore discontinuous, and the third value range Z3 lies between the value ranges Z1 and Z2, and is defined as the area between A-TH-MIN and A-TH-MAX. The value range Z3 is a low-confidence range because the relative probabilities that the gesture is category X and Y in this range do not allow one to directly infer whether it is a category X or Y gesture.
[0037] The A-TH-MIN threshold reflects a threshold below which there is a near-zero probability of occurrence that the gesture is of category X compared to a probability that the gesture is of category Y. According to an example, A-TH-MIN is defined by the threshold of the 2% probability of the Gaussian having the vertex with the highest score among the two Gaussians of the same sensor (i.e. Gaussian of gesture X for the examples of sensors 18, 19 of [Fig. 2] and [Fig. 3]). Indeed, below the 2% probability that the gesture is of category X, the probability that the detected gesture is indeed a gesture X is near-zero, but in this score window the probability that the gesture is of category Y is non-negligible. Setting the threshold at 2% is an example of a threshold value. The threshold could be 5%, 4%, 3%, 1% or less than 1% as long as it reflects a threshold below which there is almost no probability that the gesture is category X compared to category Y. In the example in [Fig.4], the A-TH-MIN threshold is defined for the score of -7. In the example of [Fig. 5], the A-TH-MIN threshold is defined for the score of -4.
[0038] The threshold A-TH-MAX reflects a threshold above which there is a quasi-zero probability of occurrence that the gesture is of category Y compared to a probability that the gesture is of category X. According to an example, A-TH-MAX is defined by the threshold of the 98% probability of the Gaussian having the vertex with the lowest score among the two Gaussians of the same sensor (i.e. Gaussian of gesture Y for the examples of sensors 18, 19 of [Fig. 2] and [Fig. 3]). Indeed, above the 98% probability that the gesture is of category Y, the probability that the detected gesture is of category Y is quasi- zero, but within this score window the probability that the gesture is category X is non-negligible. Setting the threshold at 98% is an example of a threshold value. The threshold could be 95%, 96%, 97%, 99%, or more than 99% as long as it reflects a threshold above which there is almost zero probability of occurrence of event Y compared to event X. In the example in [Fig. 4], the A-TH-MAX threshold is set for the score of 12.5. In the example in [Fig. 5], the A-TH-MAX threshold is set for the score of 6.5.
[0039] Finally, another threshold value is determined for each score distribution. The A-TH-CENTER threshold value corresponds to the intersection of the X and Y category score distributions. For the example in [Fig. 4], the A-TH-CENTER threshold is defined for a score of 0.2. For the example in [Fig. 5], the A-TH-CENTER threshold is defined for a score of -0.2.
[0040] The determination of the A-TH-MIN, A-TH-CENTER and A-TH-MAX thresholds for each of the scores of each of the sensors 18, 19 can be done in any order.
[0041] Once the signal processing has been carried out, the gesture determination method 30 can be implemented by the calculation unit 22 each time an electrical signal is provided by at least one of the two sensors 18, 19. The determination is a function of the scores and their temporal order of determination (or the temporal order of arrival of the electrical signals). The method 30 will now be described in relation to [Fig. 6],
[0042] In a first step 32, the signal from one of the two sensors 18, 19 is detected first.
[0043] In the next step 33, the signal is entered into the classification algorithm discussed above to obtain a score SC1 associated with the sensor 18 or 19 whose signal was detected.
[0044] In step 33, it is determined whether the score SC1 is in the range of values Z1, Z2, or Z3, the ranges of values being those determined for the score distributions of the algorithm trained by the sensor whose signal was detected.
[0045] If the SC1 score is in the first range of values Z1 (step 34), it is determined that the gesture is of category Y, and therefore that there is no kick (step 34-1). This determination can be stored in a cache (not shown) including a value associated with the gesture (example: gesture =Y).
[0046] If the SC1 score is in the second range of values Z2 (step 35), it is determined that the gesture is of category X, and therefore that there is a kick (step 35-1). This determination can be stored in the cache including a value associated with the gesture (example: gesture =X), then used by the calculation unit 22 to subsequently activate the opening or closing of the trunk 12.
[0047] Once the gesture is determined in steps 34-1 or 35-1, there is a reset in step 50, as soon as the first clock H1 reaches the end of the first time window T 1 (step 49). During the reset, the first clock H1 is restarted and a cache memory that contains the value of the determined gesture is emptied.
[0048] If the score SC1 is in the third range Z3 (step 48), there is no direct determination of the category of the gesture at this stage. In this case, in step 36 a second clock H2 is activated for a predetermined time window T2. This clock H2 will make it possible to wait for the possible signal coming from the other sensor. According to one embodiment, the values of the time windows T1 and T2 are such that an overlap of the two time windows is not possible. For example, the window T1 goes out before any start of the window T2.
[0049] In step 37, if during this time window T2, no signal from the other of the two sensors 18, 19 is received, then the gesture associated with the score SC1 will be determined with respect to the threshold value A-TH-CENTER determined previously during the calibration on the distribution of scores for the detected sensor 18 or 19 (step 38). If SC1 < A-TH-CENTER, it is determined that the gesture is of category Y, and therefore that there is no kick (step 39). If SC1 > A-TH-CENTER, it is determined that the gesture is of category X, and therefore that there is a kick (step 40). If SC1 = A-TH-CENTER, it is determined that the gesture is of category Y or X according to the configuration made upstream for this particular case.
[0050] If during the second time window T2, the signal from the other of the two sensors is received, it will then be analyzed to determine its SC2 score (step 41), then in which range of values Z1, Z2, or Z3 of the distribution of scores for the other of the sensors 18 or 19 detected the SC2 score belongs.
[0051] If the SC2 score is in the first range Z1 (step 42), it is determined that the gesture is of category Y, and therefore that there is no kick (step 42-1). This determination can be stored in the cache.
[0052] If the SC2 score is in the second range Z2 (step 43), it is determined that the gesture is of category X, and therefore that there is a kick (step 43-1). This determination can be stored in a cache comprising a value associated with the gesture (example: gesture =X), then used by the calculation unit 22 to subsequently activate the opening or closing of the trunk 12.
[0053] If the SC2 score is in the third range Z3 (step 44), there is no direct determination. In this case, in step 45, the probability values of each gesture are compared for the scores obtained SC1 and SC2 (4 values). The gesture retained will then be the one with the highest probability. For example, if the SC1 score for sensor 18 is 5 (the SC1 score is indeed in the Z3 zone), the probability density that the gesture is in category X is 0.095 and that for which the gesture is in category Y is 0.03. If the SC2 score for sensor 19 is 0 (the SC2 score is indeed in the Z3 range), the probability density for the gesture to be in category X is 0.12 and that for the gesture to be in category Y is 0.04. Thus, of the four probability densities, the highest is 0.12, which corresponds to a determination that the gesture is of category X (so the gesture is categorized as a kick).The determination can then be stored in the cache comprising a value associated with the gesture, and used by the calculation unit 22 to subsequently activate the opening and / or closing of the trunk 12 if the gesture is of category X.
[0054] Once the gesture has been determined in steps 39, 40, 42-1, 43-1, 45, there is a reset in step 50, as soon as the second clock H2 reaches the end of the time window T2 (step 46). The reset consists of a restart of the first clock H1 (which, according to one embodiment, if it is not already at zero, had been paused by the start of the second clock H2), a reset (without starting) of the clock H2, and a reset of the value associated with the gesture stored in memory.
Claims
Claims
1. Method (30) for determining a predetermined gesture, such as a kick (16) or a hand, the method being implemented by a computing unit (22) connected to a first sensor (18, 19) and a second radiofrequency type motion sensor, arranged in use in proximity to each other on a motor vehicle, the first and a second sensor being adapted to produce respectively a first electrical signal and a second electrical signal upon the occurrence of a gesture in a detection zone (ZD), the method comprising: a.Using a classification algorithm for each of the first and second electrical signals to obtain respectively first and second scores (SC1, SC2) associated respectively with the first and second electrical signals, as well as, for each of the first and second scores, respective probability densities that the gesture belongs to at least two categories among a first category (Y) and a second category (X) as a function of the value of the score, one of the first category and the second category corresponding to the determination that the gesture is the predetermined gesture, each classification algorithm having been trained individually for each of the first and second motion sensors so as to obtain: i.a probability density distribution that the gesture belongs to said first category (Y) as a function of the score value, and a probability density distribution that the gesture belongs to said second category (X) as a function of the score value, and ii. a first range (Z1) of score values, associated with a gesture considered to belong to the first category, a second range (Z2) of score values, associated with a gesture considered to belong to the second category, and a third range (Z3) of score values, associated with a gesture considered to be indeterminate; and b.Determining that the gesture belongs to the first or second category, depending on whether the first and second scores belong to the first, second, and third ranges and the temporal order of determining the first and second scores; characterized in that determining that the gesture is of the first or second category comprises the following steps, with the first score being obtained first:. a. If the first score belongs to the third range (48), the start (36) of a second clock (H2); b. As long as the second clock (H2) has not reached the end of a second time window (T2) (46): i. When the second signal arrives (37) at the calculation unit:
1. Obtaining (41) the second score; 2. Determining (42, 43) whether the second score belongs to the first range, second range or third range; 3. If the second score belongs to the first range (42), the determination (42-1) that the gesture is of the first category; 4. If the second score belongs to the second range (43), the determination (43-1) that the gesture is of the second category; 5. If the second score belongs to the third range (44), the gesture is that of the category with the highest probability density between the first and second categories for the first and second scores (45).
2. The method of claim 1, wherein the first score is obtained first, and determining that the gesture is of the first or second category comprises: a. Determining (34, 35) whether the first score is in the first range (Z1), second range (Z2), or third range (Z3); b. If the first score is in the first range, determining (34-1) that the gesture is of the first category; and c. If the first score is in the second range, determining (35-1) that the gesture is of the second category.
3. Method according to the preceding claim, in which once the gesture is determined to be of the first or second category, when a first clock (H1) reaches the end of a first time window (T1) (49), a reset (50) of a memory containing information on the determination of the gesture, and a reset of the first clock.
4. A method according to any one of claims 1 to 3, wherein: i. If the second signal does not arrive at the computing unit before the end of the second time window: ii. Comparison (38) of the first score with a first threshold (AT-TH-CENTER), 1. If the first score is lower than the first threshold, the determination (39) that the gesture is of the first category; 2. If the first score is higher than the first threshold, the determination (40) that the gesture is of the second category.
5. Method according to any one of claims 1 to 4, wherein: when the gesture is determined and the second clock reaches the end of the second time window (46), a reset (50) of a memory containing information on the determination of the gesture, and a reset of the first clock.
6. A method according to any preceding claim, wherein the predetermined gesture is a kick, and the detection area is a rear portion of a car vertically below a trunk of the car, the method further comprising sending a signal from the computing unit to a control unit, to actuate the opening or closing of the trunk when the gesture is of the second category.
7. Method according to the preceding claim, in which when the predetermined gesture is not a kick, an absence of sending of signal from the calculation unit to the control unit.
8. Computer program comprising instructions for implementing the method according to one of claims 1 to 7 when this program is executed by a processor.
9. Non-transitory recording medium readable by a computer on which is recorded a program for implementing the method according to one of claims 1 to 7 when this program is executed by a processor.
Citation Information
Patent Citations
Radar detection system for non-contact human activation of powered closure member
US20190162822A1
Method for opening a door of a motor vehicle by movement detection, using a doppler radar and associated detection device
US20220282553A1
Systems and methods for adaptive input thresholding
US20220308675A1