Locking system for a motor vehicle

Abstract

The invention relates to a locking system (1) for a motor vehicle (2), the locking system (1) comprising a control unit (3) and a key unit that can be carried by a user (4), which are designed to be in wireless communication with each other so as to control the motor vehicle (2), the locking system (1) also comprising an image generation unit (5) and an image processing unit (6) which is connected to the control unit (3) in terms of signal communication, wherein the control unit (3) and the image processing unit (6) are designed to identify a person as the user (4), to detect gestures and / or facial expressions and / or posture and / or body positioning of the user (4) and to assign these to predetermined control commands for controlling the motor vehicle (2).

Description

[0001] Marquardt GmbH

[0002] M 948 PCT

[0003] Locking system for a motor vehicle

[0004] Description:

[0005] The invention relates to a locking system for a motor vehicle, in particular for gesture-based control or access detection on a motor vehicle, and to an associated method.

[0006] In principle, locking systems based on a radio key that communicates with the vehicle are known in the prior art, for example from documents DE 10 2019 008 962 A1 or DE 43 40 260 A1. These locking systems can be identified by a radio key that communicates with the vehicle, allowing an authorized user of the vehicle to be identified and unlocking the vehicle, for example, by pressing a button on the radio key or on the vehicle. The present invention aims to expand and simplify the options for controlling the motor vehicle, and in particular for unlocking the motor vehicle, as well as to make them more secure. The invention is therefore based on the object of making the control of a motor vehicle, and in particular access recognition, more convenient, simpler, and more secure for the user.

[0007] This problem is solved by the combination of features according to patent claim 1.

[0008] According to the invention, a locking system for a motor vehicle is therefore proposed for controlling the motor vehicle. The control preferably also comprises the release and / or triggering of a function of the motor vehicle and in particular also the locking and / or unlocking of the motor vehicle. For this purpose, the locking system has a control unit and a key unit portable by a user, which are designed to communicate with each other wirelessly and preferably in an encrypted manner in order to control the motor vehicle. The key unit can be, for example, a radio key, a key ring (fob), a mobile phone or other objects suitable for this purpose. According to the invention, it is also provided that the locking system further has an image generation unit and an image processing unit connected to the control unit in terms of signal technology, i.e. so as to enable data exchange.The control unit and the image processing unit are designed to identify a person, in particular image-captured by the image generation unit, as the authorized user, to capture gestures and / or facial expressions and / or posture and / or body positioning of the user, and to assign these to predetermined control commands for controlling the motor vehicle.

[0009] The basic idea is to extend an access or locking system for a motor vehicle, which, for example, has a so-called "keyless" functionality, so that the vehicle can be controlled by the user's body or by movements on the user's body. Such movements or movement sequences can, for example, be the user's gestures and / or facial expressions and / or posture and / or body positioning or orientation.

[0010] In a simple case, the user can, for example, unlock the motor vehicle with a gesture, automatically open a door or trunk lid, or trigger another function of the motor vehicle.

[0011] It would also be conceivable for the vehicle's movement to be controlled by gestures. For example, the vehicle could be reversed out of a parking space using gestures, with the vehicle moving when beckoned and stopping when signaled by the hand.

[0012] The control does not have to be limited to a single gesture, for example. The image processing unit can store various gestures and / or facial expressions and / or postures and / or body positioning or orientations of the user, each of which is assigned a function or control signal for controlling the function of the motor vehicle.

[0013] The image generation unit may comprise at least one camera and / or a lidar sensor and / or a radar sensor for image generation.

[0014] If the vehicle is equipped with additional assistance systems that have an image generation unit, their image generation unit can also be used. Examples of this would be cameras used for all-round visibility, traffic sign recognition, or automated driving.

[0015] Regardless of which components the image generation unit comprises, it is preferably designed to detect the hands, body and face of a person or user, as well as to detect or detect their respective positioning and orientation as well as the movements taking place thereon.

[0016] The image processing unit and the control unit can also be formed integrally with each other.

[0017] To identify the user, the control unit can be configured, for example, to determine a position and / or distance of the key unit relative to the control unit or a predetermined point. The predetermined point can be, for example, the coordinate origin of a vehicle coordinate system that is fixed relative to the vehicle.

[0018] For this purpose, for example, a signal propagation time measurement between the key unit and the control unit can be used, whereby the control unit can, for example, have several antennas in order to be able to determine the position of the key unit from the differences in the signal propagation times between the key unit and the antennas.

[0019] Furthermore, it can be provided that the control unit is designed to identify a person as a user by evaluating the position and / or the distance of the key unit and in particular by comparing it with an access zone predetermined relative to the control unit or the predetermined point and / or to start image generation by the image generation unit.

[0020] If the key unit or the person carrying the key unit moves, in a simple case, into an area defined as an access zone, which could be an area adjacent to a door or trunk of the vehicle, for example, the person can be assumed to be the authorized user and their movements can be evaluated.

[0021] Furthermore, the image processing unit can be designed to determine a position and / or distance of a person relative to the control unit or a predetermined point.

[0022] Alternatively or additionally, the control unit and the image processing unit are configured to capture the gestures and / or facial expressions and / or posture and / or body positioning of a person imaged by the image generation unit, compare them with a predetermined identifier stored in the control unit or the image processing unit, i.e., a gesture and / or facial expression and / or posture and / or body positioning of the user defined as a key or identifier, and identify the person as the user if the captured information matches the stored information. For example, it can be provided that the person waves to the vehicle or performs another hand gesture when approaching the vehicle, thereby identifying themselves as the user.

[0023] This identification using an identifier can be combined with a position comparison of the person and the key unit. In particular, the control unit can be configured to determine the position of the key unit, and the image capture unit can be configured to determine the position of a person. If the positions match in combination with an identifier signaled by the person, the person is identified as the user.

[0024] Furthermore, the control unit and the image processing unit can also be configured to capture visible features, in particular of the face and / or the frontal head of a person imaged by the image generation unit, compare these features with features of previously known users stored in the control unit, and identify the person as the user if the features match. Accordingly, facial recognition can be implemented in this way, although the corresponding features must already be known to the control unit. To increase security, however, facial recognition can be combined with other variants for user identification, such as comparing the positions of the person and the key and / or identification by means of a gesture.

[0025] Preferably, the control unit is further configured to control the motor vehicle using the control commands, in particular to unlock the doors or lid of the motor vehicle and / or to start the motor vehicle's engine. In principle, however, a multitude of other functions of the motor vehicle can also be controlled.

[0026] A further aspect of the invention relates to a method for controlling a motor vehicle with a locking system according to the invention. The method provides that a person, in particular image-captured by the image generation unit, is identified as the user, wherein the user's gestures and / or facial expressions and / or posture and / or body positioning are captured by the image generation unit as a feature combination and transmitted to the image processing unit. The image processing unit then determines whether the feature combination is a static or a dynamic feature combination, wherein a dynamic feature combination is understood to be a sequence of movements and a static feature combination is understood to be a static posture. In particular with gestures, a distinction can be made between a static gesture, such as a single hand posture, and dynamic gestures as sequences of hand postures.The image processing unit then extracts predetermined features from the feature combination and compares the features of a static feature combination with a feature matrix of static features stored in the image processing unit, and compares the features of a dynamic feature combination with a feature matrix of dynamic features stored in the image processing unit. Based on the comparison, the image processing unit then determines a control command associated with the feature combination from the respective feature matrix.

[0027] The image processing unit can then transmit the control command to the control unit, which is designed to control a motor vehicle in accordance with the control command.

[0028] When comparing the features of a static feature combination with the feature matrix of static features stored in the image processing unit and the features of a dynamic feature combination with the feature matrix of dynamic features stored in the image processing unit to assign the control command, the image processing unit can also supplement the features with stored statistical data or determine a control command after the comparison based on statistical data. This statistical data can also be stored as a model, in particular as a neural network, in the control unit or the image processing unit, so that such a model can determine or predict the most probable gesture and thus the most probable control command based on the features extracted from the feature combination.

[0029] If features are extracted from a combination of features, e.g., a gesture, that cannot be fully assigned to a single or specific control command, the statistical data or the model can be used to determine which gesture or control command is most likely, and then select this or that. The features disclosed above can be combined in any way, as long as this is technically possible and they do not contradict each other.

[0030] Other advantageous developments of the invention are characterized in the subclaims or are presented in more detail below, together with the description of the preferred embodiment of the invention, with reference to the figures. They show:

[0031] Fig. 1 schematic view of a locking system;

[0032] Fig. 2 Process for detecting and assigning a gesture to a control command;

[0033] Fig. 3 Process for storing statistical data for static gestures;

[0034] Fig. 4 Procedure for storing statistical data for dynamic

[0035] gestures.

[0036] Figure 1 schematically illustrates, by way of example, a variant of a locking system 1 for a motor vehicle 2, by which the motor vehicle 2 can be controlled. The locking system 1 has a control unit 3 and a key unit 4 that can be carried by a user, which is not shown here but can be operated by the user 4. The control unit 3 and the key unit communicate wirelessly with each other to control the motor vehicle 2 in the sense of a keyless-go or keyless-entry system, which is indicated by a signal 7.

[0037] To expand the functionality of such a keyless-go or keyless entry system, the locking system 1 further comprises an image generation unit 5 and an image processing unit 6 connected to the control unit 3 and the image generation unit 5 via signal technology, so that the image generation unit 5 can generate an image and transmit it to the image processing unit 5, which evaluates the image and transmits the result of the evaluation to the control unit 3. In this case, an image can be understood as a single image, a sequence of single images, or a video stream. Accordingly, the control unit 3 and the image processing unit 6 are designed to identify a person, in particular from a plurality of persons, as a user or the authorized user 4, specifically here to record their gestures and to assign the recorded gestures to predetermined control commands for controlling the motor vehicle 2.

[0038] When approaching vehicle 2 on the driver's side of vehicle 2, user 4 can, for example, unlock the driver's door using a hand gesture. Gesture 10 can also simply be raising the hand to a door handle 8 of vehicle door 9. When user 4 approaches the passenger side of vehicle 2, user 4 can unlock the passenger door 9 using gesture 10. Furthermore, user 4 can also unlock all doors of vehicle 2 or activate other functions of vehicle 2 using an alternative gesture.

[0039] Figures 2 to 4 each show the proposed methods using a flowchart-like representation.

[0040] To maintain clarity, sections Y and Z of the method according to Fig. 2 are shown in Figs. 3 and 4, whereby the method according to Fig. 2 can also be used without the sections according to Figs. 3 and 4.

[0041] The method according to Figure 2 comprises, in particular, sections X (image-based gesture recognition process), Y (training process for static gestures), and Z (training process for dynamic gestures). In particular, the following steps are performed:

[0042] A Detection of an authorized user 4 in an access zone of the vehicle 2. B Start of the image generation unit 5.

[0043] C Transmitting an image to the image processing unit 6.

[0044] D Preprocessing to differentiate static or dynamic gestures, in particular capturing the gesture as a feature combination.

[0045] E Is the gesture or feature combination captured by the image a static gesture (then s) or is it a dynamic gesture (then d)? sF Feature extraction from the static gesture or feature combination. dF Feature extraction from the dynamic gesture or feature combination. sG Comparison with a feature matrix of static gestures in which features for a large number of static gestures are stored. dG Comparison with a feature matrix of dynamic gestures in which features for a large number of dynamic gestures are stored. sH Statistical prediction of a static gesture. dH Statistical prediction of a dynamic gesture. sJ Is this a first predetermined static gesture? If yes (1), then transmission of an associated control command sL to the control unit 3. If no (0), then on to the next static gesture.dJ Is this a first predetermined dynamic gesture? If yes (1 ), then transmission of an associated control command dL to the control unit 3. If no (0), then on to the next dynamic gesture. sK Is this a last or n-th predetermined static gesture? If yes (1 ), then transmission of an associated control command sM to the control unit 3. If no (0), then transmission of an associated control command sN to the control unit 3, which may also be an error message. dK Is this a last or n-th predetermined dynamic gesture? If yes (1 ), then transmission of an associated control command dM to the control unit 3. If no (0), then transmission of an associated control command dN to the control unit 3, which may also be an error message.

[0046] In steps sH and dH, statistical data is taken into account, which is collected through training a model or incorporated into the gesture selection. Since the exemplary processes for the training process of the dynamic gestures Z and the training process of the static gestures Y are identical in terms of their execution, Figures 3 and 4 are described together.

[0047] According to Figures 3 and 4, the following steps are essentially planned:

[0048] A Data acquisition of specific, static gestures (Figure 3) or dynamic gestures (Figure 4) of the authorized user 4, in particular as feature combinations.

[0049] B Compilation of training data.

[0050] C Extraction of features for individual gestures or feature combinations from the training data. D Generation of a feature matrix from the extracted features.

[0051] E Assigning gestures and / or control commands to the gestures or feature combinations stored in the feature matrix. F Training a model, particularly a neural network.

[0052] G Obtaining a trained model to predict a static gesture in step sH (Figure 3) or a dynamic gesture in step dH (Figure 4).

[0053] H Evaluation of the model. J Collection and consideration of metrics extracted during the evaluation.

[0054] The invention is not limited to the preferred embodiments described above. Rather, a number of variants are conceivable that utilize the presented solution even in fundamentally different embodiments.

Claims

Patent claims 1. Locking system (1) for a motor vehicle (2), wherein the locking system (1) comprises a control unit (3) and a key unit portable by a user (4), which are configured to communicate wirelessly with each other for the purpose of controlling the motor vehicle (2), wherein the locking system (1) further comprises an image generation unit (5) and an image processing unit (6) connected to the control unit (3) by means of a signal, wherein the control unit (3) and the image processing unit (6) are configured to identify a person as the user (4), to detect gestures and / or facial expressions and / or posture and / or body positioning of the user (4) and to assign these to predetermined control commands for controlling the motor vehicle (2).

2. Locking system according to claim 1, wherein the image generation unit (5) comprises at least one camera and / or a lidar sensor and / or a radar sensor for image generation.

3. Locking system according to claim 1 or 2, wherein the control unit (3) is configured to determine a position and / or distance of the key unit relative to the control unit (3) or a predetermined point.

4. Locking system according to the preceding claim, wherein the control unit (3) is configured to identify a person as a user (4) by evaluating the position and / or distance of the key unit and in particular by comparison with an access zone predetermined relative to the control unit (3) or the predetermined point, and / or to generate an image by To start the image generation unit. Locking system according to one of the preceding claims, wherein the image processing unit (6) is configured to determine the position and / or distance of a person relative to the control unit (3) or a predetermined point. Locking system according to one of the preceding claims, wherein the control unit (3) and the image processing unit (6) are configured to detect gestures and / or facial expressions and / or posture and / or body positioning of a person imaged by the image generation unit (5), to compare these with a predetermined identifier stored in the control unit (3) or the image processing unit (6), and to identify the person as a user (4) if there is a match.A locking system according to one of the preceding claims, wherein the control unit (3) and the image processing unit (6) are configured to detect visible features, in particular of the face and / or the frontal head, of a person imaged by the image generation unit (5), to compare these features with features of previously known users stored in the control unit, and to identify the person as user (4) if the features match. A method for controlling a motor vehicle (2) with a locking system (1) according to one of the preceding claims, wherein a person imaged, in particular by the image generation unit (5), is identified as user (4), wherein gestures and / or facial expressions and / or posture and / or body positioning of the user (4) are detected by the image generation unit (5) as a combination of features and transmitted to the image processing unit. The image processing unit (6) transmits the control command to the control unit (3), which determines whether the feature combination is static or dynamic, extracts predetermined features from the feature combination, compares the features of a static feature combination with a feature matrix of static features stored in the image processing unit (6) and the features of a dynamic feature combination with a feature matrix of dynamic features stored in the image processing unit (6), and determines a control command assigned to the feature combination from the respective feature matrix. The method according to the preceding claim, wherein the image processing unit (6) transmits the control command to the control unit (3), which is configured to control a motor vehicle (2) according to the control command.Method according to one of the two preceding claims, wherein the image processing unit (6), when comparing the features of a static feature combination with the feature matrix of static features stored in the image processing unit (6) and the features of a dynamic feature combination with the feature matrix of dynamic features stored in the image processing unit for the purpose of assigning the control command, supplements the features with stored statistical data or determines a control command after the comparison on the basis of statistical data.

Images