Method and system for user identification

Abstract

The invention relates to a method for user identification, wherein sensor data (10-x) are captured by means of at least one sensor (10-x), wherein an initial identification of a user is carried out on the basis of the captured sensor data (10-x), wherein at least in the event of a successful initial identification, a positive identification signal (20) is generated by means of a communication device (4), wherein, after a successful initial identification, at least one specified termination criterion (30-x) is checked, and wherein the positive identification signal is provided until the check indicates that at least one specified termination criterion (30-x) is present. The invention also relates to a system (1) for user identification.

Description

[0001] Description

[0002] Procedure and system for user identification

[0003] The invention relates to a method and a system for user identification.

[0004] With increasing digitalization and virtualization, authentication for the use of software, apps, or physical devices for everyday tasks is becoming more common and is increasingly reaching less sensitive areas of life (vehicles, media, entertainment, etc.). Public awareness and the need for data privacy and security are also on the rise, calling for secure (multi-factor) authentication (MFA), which was previously reserved only for sensitive banking or healthcare applications.

[0005] Many apps and digital services are linked to complex authorization systems in the background and secure authentication and thus activation in these authorization systems is of far-reaching importance. A prerequisite for secure authentication is secure proof of a user's identity. In many cases, ad hoc authentication requires knowledge (e.g. a password) and the use of a key (key-based authentication). Alternatively, identity-based authentication (IBA) can be used. Here, the identity of a person is first determined ad hoc on a device using biometric or other factors (e.g. unlocking a smartphone using facial recognition, also known as FaceID). This determined identity is then subsequently used for further, based authentication without the need to enter any further passwords or keys.

[0006] The invention is based on the object of improving a method and a system for user identification.

[0007] The object is achieved according to the invention by a method having the features of patent claim 1 and a system having the features of patent claim 16. Furthermore, the object is also achieved by a method having the features of patent claim 20 and a system having the features of patent claim 21. Advantageous embodiments of the invention emerge from the subclaims.

[0008] In particular, a method for user identification is provided, wherein sensor data are acquired by means of at least one sensor, wherein an initial identification of a user is carried out based on the acquired sensor data, wherein at least in the case of a successful initial identification a positive identification signal is provided by means of a communication device, wherein after a successful initial identification at least one termination criterion is checked, and wherein the positive identification signal is provided until the check shows that at least one termination criterion is present.

[0009] Furthermore, in particular, a system for user identification is created, comprising at least one sensor which is configured to capture sensor data, a data processing device and a communication device, wherein the data processing device is configured to receive sensor data captured by the at least one sensor and, based on the captured sensor data, to carry out an initial identification, to provide a positive identification signal via the communication device at least in the event of a successful initial identification, and to check at least one termination criterion after a successful initial identification, and to provide the positive identification signal until the check shows that at least one termination criterion is present.

[0010] Furthermore, in a further aspect, a method for user identification is provided, wherein sensor data are acquired by means of at least two sensors, wherein an initial identification of a user is carried out based on the acquired sensor data, and wherein at least in the case of a successful initial identification, a positive identification signal is provided by means of a communication device.

[0011] In a further aspect, a system for user identification is also created, comprising at least two sensors configured to acquire sensor data, a data processing device, and a communication device. The data processing device is configured to receive sensor data acquired by the at least two sensors and, based on the acquired sensor data, to perform an initial identification and, at least in the event of a successful initial identification, to provide a positive identification signal via the communication device. The method and the system make it possible to carry out an initial identification and subsequently, after successful identification, to provide a positive identification signal. This allows services to be activated without the identification having to be repeated for each service.According to the method, the positive identification signal is provided until at least one termination criterion is met. If the at least one termination criterion is met and the positive identification signal is no longer provided, the user is deemed not to be identified (or no longer to be identified). However, without the at least one termination criterion being met, the positive identification signal continues to be provided. After the initial identification, the at least one termination criterion is checked for this purpose. This is done repeatedly, in particular continuously and / or at regular intervals. In other words: the at least one termination criterion represents a trigger that terminates the provision of the positive identification signal; the checking can be implemented logically, in particular in the form of a "watchdog" circuit / logic.The idea behind the at least one termination criterion is that the positive identification signal is provided until it can no longer be ensured that the user identity established during the initial identification is still present.

[0012] Providing the positive identification signal can comprise transmitting the positive identification signal, in particular wirelessly, to at least one device and / or a service in the at least one device. Providing the positive identification signal can also comprise operating an interface for query-controlled provision of the positive identification signal. The positive identification signal comprises at least one piece of information that uniquely identifies a user, for example, an identification code and / or a unique data key. Furthermore, the information can also comprise information relating to biometric identification, for example, via facial recognition, a fingerprint, an iris scan, etc., which are used in conventional identification systems.

[0013] In an example scenario, a user identifies themselves using the method as part of the initial identification. The system then provides the positive identification signal. This allows devices, e.g. a vehicle, and / or services (in particular wirelessly) to be unlocked, or the user can authenticate themselves with the device and / or services using the provided positive identification signal. A sensor can, for example, be a microphone, a camera, a gyroscope, an acceleration sensor, a health sensor (e.g. in a smartwatch, e.g. a heart rate sensor, an acceleration sensor, an activity sensor, etc.). A sensor can in particular also be an operating device, for example a keyboard or a touch-sensitive display device (touchscreen), the recorded inputs of which can be used and / or evaluated as sensor data.Furthermore, a sensor can also be a temperature sensor or a vibration sensor. A sensor can also be an RFID reader. In general, a sensor can comprise one or more of the following modalities: optical, acoustic, mechanical, electromagnetic. Sensor data can be one-dimensional or multi-dimensional, for example, two-dimensional or three-dimensional. The sensor data can in particular comprise biometric data of a user. The sensor data is processed in particular as time-resolved sensor data or as sensor data acquired over time (i.e. as a time series). In addition to the sensor data, data from other data sources can also be taken into account during the initial identification and / or verification.

[0014] In particular, it is provided that the initial identification is carried out based on multimodal sensor data and / or other data sources. Furthermore, it can be provided that the verification is carried out based on multimodal sensor data and / or other data sources.

[0015] Parts of the system, in particular the data processing device, can be implemented individually or collectively as a combination of hardware and software, for example as program code executed on a microcontroller or microprocessor. However, it can also be provided that parts are implemented individually or collectively as an application-specific integrated circuit (ASIC) and / or a field-programmable gate array (FPGA). The system comprises, in particular, at least one computing device and at least one memory. The system can, in particular, comprise further devices, such as an operating device and / or a display device.

[0016] In particular, it is provided that a user, prior to initial identification, provides user data in the form of recorded sensor data, which is stored unprocessed or in processed form in a memory of the system and which is characteristic of the user. This user data, provided as reference data or reference signatures, includes, for example, the user's biometric data. This stored user data can be accessed during the initial identification and / or subsequent identifications and / or verification. The system can further comprise and / or provide suitable means with which the user data provided as reference data or reference signatures can be recorded and / or generated.

[0017] It can be provided that identification is also carried out after the initial identification. In particular, identification of a user can take place continuously, in particular repeatedly. This subsequent identification can generally take place in the same way as the initial identification. In particular, recorded sensor data from the at least one sensor and / or other data sources are evaluated for this purpose as described in this disclosure. A respective result can also be provided as part of the positive (or negative) identification signal. In this way, identification can be continuously re-established, in particular even after the initial identification.

[0018] It may be provided that a confidence value, which can also be referred to as an identity score, is determined and provided for the initial identification, for example, as part of the positive identification signal. The confidence value indicates, in particular, the confidence (certainty) or probability with which the initial identification was carried out. It may also be provided for an identification subsequent to the initial identification to provide a confidence value and / or to update the confidence value of the initial identification, so that a confidence level of the identification can be mapped and provided at any time.

[0019] It can be specified that several termination criteria are considered in a weighted manner. The respective weightings can be used to set a sensitivity for each termination criterion.

[0020] In one embodiment, the initial identification for recognizing a user identity comprises pattern recognition of patterns in the recorded sensor data. This makes it possible to recognize patterns characteristic of a user (which can also be referred to as fingerprints). For example, it can be provided to determine a distance to a stored pattern (fingerprint) of the user based on a suitable representation of the sensor data (e.g. by mapping to an abstract but representative (numerical) vector, hash value, etc.). The representation can be estimated or generated, for example, using a suitable trained machine learning method that maps the sensor data to a representative vector with reduced dimension. If the determined distance falls below a predetermined threshold, the user is considered identified.If there are several users who should or can be identified, several patterns (fingerprints) are stored with which a comparison is carried out. For example, an acoustic signature (as a pattern or fingerprint) of a user can be recognized in audio data recorded using a microphone. Other examples are the recognition of patterns in camera data (e.g. facial recognition), movement data (movement patterns, walking patterns, acceleration patterns, etc.), user-specific patterns when using devices, e.g. when using a keyboard or swiping behavior on a touch-sensitive display. The patterns mentioned are merely examples and can also include other types of sensor data. The patterns (orDepending on the type of sensor data, fingerprints can contain and / or be derived from amplitude values, a melspectrum, a change in image element values ​​around an observed image element, spatial frequencies, a frequency spectrum, features (keypoints) in images (e.g. using SIFT, SURF, etc.).

[0021] For example, the following methods can be used for pattern recognition:

[0022] - Abdullah Mueen et al., The Fastest Similarity Search Algorithm for Time Series Subsequences under Euclidean Distance, 2015, URL: http: / / www.cs.unm.edu / ~mueen / FastestSimilaritySearch.html;

[0023] - Chin-Chia Michael Yeh et al., Matrix Profile I: All Pairs Similarity Joins for Time Series: A Unifying View that Includes Motifs, Discords and Shapelets, 2016, IEEE ICDM 2016;

[0024] - Avery Li-Chun Wang, An Industrial-Strength Audio Search Algorithm, ISMIR 2003, 4th International Conference on Music Information Retrieval, Baltimore, Maryland, USA, October 27-30, 2003, Proceedings.

[0025] In particular, it can also be provided that a correlation between patterns in sensor data of different types is determined and also evaluated using pattern recognition. In this case, correlations in sensor data of different modalities are considered in particular.

[0026] It may be provided that, before applying the system and the method, pattern mining is carried out as an exploratory step in order to determine which features in the sensor data are suitable for identifying user-specific patterns (e.g.

[0027] Pattern recognition using Matrix Profile or Symbolic Aggregate Approximation (SAX) symbols for time series; patterns relevant for identification are, for example, those that do not repeat between different people.

[0028] In one embodiment, the initial identification for recognizing a user identity comprises applying a trained machine learning model to which at least a portion of the acquired sensor data is fed as input data. The model receives the most recently measured sensor data and sensor data previously recorded from the user that can identify the user (user data) as input data. The trained machine learning model compares the acquired sensor data with the user data and can estimate whether the most recently acquired user data originates from the same user.The model is trained using different pairings of sensor data segments from different users with sensor data segments from themselves and other users, where the former (identification segment and last acquired segment from the same user) serve as positive training data points and the latter (identification segment and last acquired sensor data from different people) serve as negative training data points.

[0029] With the help of the trained machine learning model, identification can be carried out using artificial intelligence and machine learning methods. In particular, it is provided that, in addition to the sensor data, the trained machine learning model also receives data characteristic of the user (as a data segment or data sample) as input data. Based on the recorded sensor data and the characteristic data, the trained machine learning model estimates as output whether the sensor data corresponds to the user, i.e., whether the user has been identified or not. In particular, the trained machine learning model provides a probability value that can be compared, for example, with a threshold value to decide whether the user has been identified or not. However, it can also be provided that the trained machine learning model directly provides a binary output value for this purpose.During a training phase, the machine learning model is trained to determine a user's identity based on sensor data supplied as input data. For this purpose, the machine learning model is additionally provided with a data segment of sensor data that corresponds to a user whose identity is to be determined or characterizes this user with respect to the respective sensor type (e.g., audio data, motion data, etc.).

[0030] In one embodiment, the initial identification uses a state estimator to detect the user's identity. This can be implemented using a filter (Kalman filter, particle filter, or similar) that aggregates consecutive estimates of the user's identity over time, allowing for a more reliable identity estimate. A unit function or a temporal decline in the confidence of a user's identity is modeled as the prediction step. In the correction step, the last estimate of the identification (fingerprint, machine learning model, etc.) and the current state of the filter are updated. This allows for higher confidence levels than are possible with estimates based on individual journals.

[0031] In one embodiment, it is provided that a termination criterion includes a loss of a coupling with at least one predetermined, previously signal-coupled device and / or reaching and / or exceeding a predetermined distance from the predetermined at least one device. This allows the presence or absence of an additional device to be checked as a termination criterion. Such a device can be, for example, a smartphone, a smartwatch, a health tracker, or wireless headphones. The device is coupled to the system (also referred to as "pairing") and generally remains coupled until the coupling is actively terminated by a user or the distance for wireless communication, which takes place via Bluetooth, for example, becomes too great and the connection is lost.Since the devices mentioned usually correspond to a very specific user, this can be used to determine the user's proximity to the system or the user's presence or absence. For example, if the system is designed as an electronic key that unlocks the operation of a vehicle, the initial identification can include checking whether pairing with the user's smartphone is possible. After successful initial identification, the positive identification signal is transmitted from the key to the vehicle and the vehicle can be used. If the key is put away at the end of the trip and the user moves away from the key with their smartphone, the connection is eventually broken and the pairing is canceled.The termination criterion is then met, the positive identification signal is no longer provided and operation of the vehicle is no longer possible until another initial identification is successfully carried out.

[0032] In one embodiment, it is provided that a termination criterion includes the occurrence or disappearance of at least one predetermined pattern in sensor data acquired after the successful initial identification of at least one sensor. In this way, it can be determined on the basis of the sensor data whether the user is still considered to be identified or not. For example, it can be provided to determine on the basis of movement data that a system designed as a key is no longer being moved. From this, it can be concluded that the key has been put away, i.e. is no longer with the user (e.g. in the trouser pocket). In a further example, the sensor data (e.g.

[0033] Patterns (e.g., a voice signature and / or a movement pattern) present in the audio data and / or movement data may change over time, so that these patterns no longer correspond to a previously identified user, but possibly to no user at all or to another user. Such a change is detected in this embodiment and used as the termination criterion.

[0034] In one embodiment, a termination criterion includes the condition that a position contradicts a location in a user's calendar entry corresponding to a current time. This allows the identification to be verified based on the location. For this purpose, it may be provided that the user's calendar data is queried and / or received, which can be compared with current position data (e.g., from a GPS).

[0035] In one embodiment, it is provided that the positive identification signal is provided until a predetermined number of termination criteria and / or a predetermined combination of termination criteria is met. This makes it possible to determine more reliably whether the user is still considered identified or not. For example, it can be provided that at least two termination criteria must be met for the positive identification signal to no longer be provided. For example, it can be provided that the loss of the connection to another device and a resting of the system (e.g. the key) must be present as termination criteria for the user to no longer be considered identified and the positive identification signal to no longer be provided.

[0036] In one embodiment, it is provided that sensor data from at least two sensors is recorded and taken into account for the initial identification. This allows the user to be recognized more reliably. In particular, this also allows a correlation between the sensor data of the at least two sensors to be evaluated. For example, correlations between recognized patterns in the sensor data can be evaluated. For example, it can be provided that sensor data from a microphone and sensor data from a gyroscope or from acceleration sensors are taken into account for the initial identification. Furthermore, it can additionally be provided that a connection to a device of the user (e.g. a smartphone and / or a smartwatch) is also taken into account. In one embodiment, it is provided that sensor data from at least two sensors is taken into account when checking the at least one termination criterion. This allows the checking orThe presence of at least one termination criterion can be made more robust. In particular, this can prevent the positive identification signal from being discontinued if a (single) termination criterion is present.

[0037] In one embodiment, the sensor data of the at least two sensors are merged before the initial identification and / or before checking the at least one termination criterion. This makes it possible to combine the sensor data early in the signal chain and to perform the initial identification and / or the verification based on the combined sensor data.

[0038] In one embodiment, the initial identification and / or the verification of the at least one termination criterion are performed individually on the sensor data of at least two sensors, with the obtained results subsequently being merged. This allows the sensor data to be evaluated individually and the results to be merged. This allows for more robust identification and verification of the identification.

[0039] In one embodiment, the merging comprises processing using at least one filter and / or a temporal estimator and / or using the Dempster-Shafer rule of combination. In particular, a temporal development or values ​​over time can also be taken into account during the initial identification and / or when checking the at least one termination criterion. This allows for a more robust user identification.

[0040] In one embodiment, it is provided that a confidence value is determined and provided for the initial identification and / or an identification following the initial identification. The confidence value indicates, in particular, the probability with which the initial identification was carried out. Even after the initial identification, the confidence value can be provided and / or reset, so that a confidence level of the identification is mapped and provided at any time. The confidence value can also be referred to as an identity score, which indicates the reliability or certainty with which the user's identity was and / or is currently being determined. When determining the confidence value, in particular, all modalities used for identification (sensors and / or other data sources) can be taken into account.The confidence value is estimated and / or determined during the initial identification and / or after the initial identification, in particular for each of the evaluation methods used (e.g., pattern recognition, trained machine learning model, etc.) of the sensor data and / or other data sources. During pattern recognition, for example, a degree of agreement between a pattern (fingerprint) in the recorded sensor data and a pattern (fingerprint) stored for a user can be used as a confidence measure. When using a trained machine learning model to evaluate the sensor data and / or other data sources, the confidence provided by the trained machine learning model during inference can be used as a confidence value when estimating the agreement.If multiple modalities and / or multiple methods are used to evaluate the sensor data and / or the other data sources, the respective confidence values ​​are aggregated (each weighted) into a common confidence value. The confidence value can be provided as part of the positive (or negative) identification signal.

[0041] It can be provided that a specific and / or aggregated confidence value is reduced taking into account a lapse of time, wherein the confidence value is compared with a predetermined threshold for the confidence, wherein an end criterion includes the falling below the predetermined threshold. In this way, the time criterion can be taken into account after an identification has taken place (initial or subsequent). In particular, it is provided that a (successful) identification following the identification under consideration can increase the confidence value again, so that although the confidence value decreases over time, it can always be kept above the threshold through successful subsequent identifications. However, if the confidence value decreases below the threshold over time, the end criterion is met and the positive identification signal is no longer provided.It may be provided that a gradient at which the confidence value decreases over time can be set as a sensitivity parameter. It may also be provided that the threshold value can be specified as a further sensitivity parameter.

[0042] In one embodiment, it is provided that the initial identification is started when at least one activity criterion is met. This means that the initial identification can only be started when identification of the user is expected. In this way, energy can be saved, in particular when identification of the user is unlikely. The activity criterion can, for example, contain a specific movement pattern in sensor data from a motion or acceleration sensor. If the system is designed as a key, an initial identification can, for example, always be started when movement above a predetermined threshold is detected, i.e. the key is moved (at all). If a positive identification signal is not currently provided, the at least one activity criterion is checked, in particular by evaluating recorded sensor data.This occurs repeatedly, especially continuously and / or at regular intervals. In other words: the at least one activity criterion represents a trigger that starts the initial identification; the check can be implemented logically, especially in the form of a "watchdog" circuit.

[0043] In one embodiment, if the initial identification fails, alternative identification is initiated and / or performed. This allows for the provision of a fallback option for identification. For example, if the initial identification fails, a user may be required to authenticate using a password. The password can be recorded, for example, in the form of a voice input or as text. In principle, however, a barcode, a QR code, or another key can also be recorded and evaluated for authentication.

[0044] In one embodiment, user data used to identify the user during initial identification and / or subsequent identification is updated if a data drift is detected. A data drift can be identified as a systematic, gradual decline in identification accuracy (measured as confidence in the identity estimate). A data drift can also be identified as a gradually increasing deviation of the data distribution from the stored data segment (stored user data) that the user enters in the initial setup, measured by simple statistical parameters (mean, variance, skew, kurtosis, ...) or a machine learning model that estimates whether a deviation exists. By detecting a data drift, data stored for the identification or authentication of a user (e.g., patterns / fingerprints) can be kept up to date at all times, e.g.by prompting the user to repeat the initial setup upon detection of a data drift, or by recording corresponding data without further user consultation. This allows, in particular, changes in user-specific patterns to be taken into account (e.g., changes in gait due to increasing age; changes in voice pitch; changes in movement patterns due to a change in training status or different footwear, etc.). In particular, it can be provided that the update takes place taking the confidence value into account. In particular, the user data can be updated whenever the confidence value is above the specified threshold. In this way, it can be ensured, in particular, that the updated user data actually corresponds to the user.

[0045] The system can, for example, be a portable system that can be worn on the body of a user and has, in particular, small dimensions of only a few centimeters.

[0046] In one embodiment of the system, at least one sensor is part of an electronic key. It can be provided that the data processing device is at least partially configured outside the key, for example, as an external computing device, central server, or cloud computer.

[0047] In one embodiment of the system, the system is designed as an electronic key. In this embodiment, the key comprises all components of the system, in particular the at least one sensor, in particular several sensors of different modalities, the data processing device, and the communication device.

[0048] In one embodiment of the system, at least part of the data processing device is configured as a cloud computer. Parts of the data processing device can be connected to one another via a communication link, in particular a wireless one.

[0049] Further features of the system's design are described in the various process configurations. The advantages of the system are the same as those of the process configurations.

[0050] Further features for the design of the system and method according to the further aspect will become apparent from the description of embodiments of the method and system. The advantages of the method and system according to the further aspect are the same as in the embodiments of the method and system. The invention is explained in more detail below using preferred embodiments with reference to the figures. Herein:

[0051] Fig. 1 is a schematic representation of an embodiment of the system for user identification;

[0052] Fig. 2 is a schematic flow diagram illustrating embodiments of the method and the system;

[0053] Fig. 3 is a schematic flow diagram illustrating embodiments of the method and system;

[0054] Fig. 4 is a schematic flow diagram to illustrate embodiments of the method.

[0055] Fig. 1 shows a schematic representation of an embodiment of the system 1 for user identification. The system 1 comprises at least one sensor 2-x, a data processing device 3, and a communication device 4. The system 1 is particularly configured to carry out the method described in this disclosure. The method is described in more detail below with reference to the system 1. The system 1 is, for example, part of a (physically formed) electronic key 60.

[0056] By way of example, several sensors 2-x are shown, each of which acquires sensor data 10-x. The acquired sensor data 10-x is fed to the data processing device 3. The sensor 2-1 is, for example, a microphone; the sensor 2-2 is, for example, a motion or acceleration sensor; and the sensor 2-3 is, for example, a position sensor that provides, for example, a position signal based on signals from a global navigation system (e.g., Global Positioning System, GPS). In principle, however, a different combination of sensors 2-x or additional data sources can also be provided.

[0057] The data processing device 3 comprises at least one computing device 3-1 and at least one memory 3-2. The data processing device 3 is configured to receive the sensor data 10-x acquired by the sensors 2-x and, based on the acquired sensor data 10-x, to perform an initial identification. At least in the case of a successful initial identification, the data processing device 3 provides a positive identification signal 20 via the communication device 4. It may be provided to provide a negative identification signal 21 in the case of a negative identification (no user detected or incorrect user detected) or in the case of a negative identification (no user detected or incorrect user detected).

[0058] The data processing device 3 is further configured to check at least one predefined termination criterion 30-x after a successful initial identification. The positive identification signal 20 is provided by the data processing device 3 until the check reveals that at least one predefined termination criterion 30-x is present. A definition of the at least one termination criterion 30-x is stored, for example, in the memory 3-2.

[0059] The positive identification signal 20 (or the negative identification signal 21) can be transmitted, for example, to a device 51 in a vehicle 50 to enable the operation of the vehicle 50 and / or other services. However, the device 51 can also be used in other areas, e.g., to authenticate a user on the Internet.

[0060] It can be provided that the initial identification for recognizing a user identity comprises pattern recognition of patterns in the acquired sensor data 10-x. It can be provided that, for this purpose, user-specific patterns 40 are stored in the memory 3-2, which are used in the pattern recognition. In particular, a suitable representation of the sensor data 10-x is provided by the data processing device 3, for example in the form of a representative (numerical) vector. The representation, for example the representative (numerical) vector, is then compared with the user-specific pattern 40 stored in the same representation form by determining a distance between the representations. It can be provided that the user-specific pattern 40 is or will be stored individually for each sensor type.With fused sensor data 10-f, a user-specific pattern 40 can also be stored for the fused sensor data 10-f. The distance is then compared with a predefined threshold. If the distance is smaller than the predefined threshold, the user for whom the user-specific pattern 40 is stored is considered to have been recognized. The positive identification signal 20 is then provided. Otherwise, no user or a given user was not recognized. In this case, the negative identification signal 21 can be provided. If the identification of multiple users is intended, a user-specific pattern 40 is stored for each user, and the comparison is made with each of the stored patterns 40.It can be provided that the initial identification for recognizing a user identity comprises applying a trained machine learning model 5, to which at least a portion of the acquired sensor data 10-x is supplied as input data. In particular, it is provided that a user-specific pattern 40 (or a user-specific data segment) is additionally supplied to the trained machine learning model 5 as input data. Based on the input data, the trained machine learning model 5 estimates whether a user identity is present or not. As output data, the trained machine learning model 5 can directly include a binary state value with the two characteristics "User X recognized" and "User X not recognized." It can also be provided that the trained machine learning model 5 estimates a probability for the presence of the user identity and outputs it as output data.The estimated probability value can then be compared to a predefined threshold. If the threshold is exceeded, the user is recognized; if the threshold is not met, the user is not recognized.

[0061] It can be provided that a termination criterion 30-x includes a loss of a coupling 24 with at least one predetermined, previously signal-coupled, device 25 and / or reaching and / or exceeding a predetermined distance to the predetermined at least one device 25. The device 25 can be, for example, a smartphone, a smartwatch, or wireless headphones. The coupling to several such devices 25 can also be monitored. The coupling (also referred to as "pairing") can take place, for example, via Bluetooth. In principle, the possibility of the coupling 24 or the presence of the at least one device 25 in the environment of the system 1, in particular of the key 60, can also be used for initial identification.If the loss of coupling is detected or the specified distance is reached and / or exceeded, the positive identification signal 20 is no longer provided. The distance can be estimated, for example, using a signal strength.

[0062] It can be provided that a termination criterion 30-x includes the occurrence or absence of at least one predetermined pattern in sensor data 10-x acquired from at least one sensor 2-x after the successful initial identification. If, for example, an audio signature is stored as a pattern for the user, based on which a voice of the user can be recognized in the acquired audio data, then after the initial identification, it can be checked based on the sensor data 10-1 of the sensor 2-1 (microphone) whether the stored audio signature is still contained. Analogously, patterns that correspond to the user can also be recognized in other sensor data. Furthermore, patterns that do not correspond to the user, for example, movement patterns, can also be recognized.If the system 1, in particular the key 60, is put away, it generally no longer moves in the same way as when the system 1, in particular the key 60, is worn on the user's body. This change can be detected in the sensor data 10-2 of the sensor 2-2 (motion sensor).

[0063] It can be provided that the positive identification signal 20 is provided until a predetermined number of termination criteria 30-x and / or a predetermined combination of termination criteria 30-x is met. This allows for a more robust detection of the loss of user identification. In particular, the probability of an unjustified loss of identification, i.e., of a misclassification, can be reduced.

[0064] It can be provided that for the initial identification, sensor data 10-x from at least two sensors 2-x are recorded and taken into account. In particular, it can be provided that a minimum set of sensors 2-x is defined or taken into account in order to carry out the initial identification and subsequently maintain the identification or check the at least one termination criterion. This allows the identification to take place in a multimodal way. For example, the following minimum set of sensors 2-x can be provided: a microphone for recording audio data and a motion sensor for recording movement data. In addition, it can also be provided to take into account a coupling to at least one other device 25 (e.g., a smartphone).

[0065] It can be provided that, when checking the at least one termination criterion 30-x, sensor data 10-x from at least two sensors 2-x are taken into account. In particular, it can be provided that a minimum set of sensors 2-x is defined or taken into account in order to maintain the identification or to check the at least one termination criterion 30-x. The minimum set can be the same as described above.

[0066] It can be provided that the sensor data 10-x of the at least two sensors 2-x are fused before the initial identification and / or before checking the at least one termination criterion 30-x. This is schematically illustrated in Fig. 2, which shows a flow chart to illustrate a data flow. A module 11 fuses the sensor data 10-x to form fused sensor data 10-f, which is then further processed by a module 12. The module 12 extracts patterns (e.g., as a fingerprint). Alternatively or additionally, the sensor data 10-x can also be evaluated using a trained machine learning model, as already described above. For pattern recognition, the module 12 can, for example, generate a representative (numerical) vector 13 from the fused sensor data 10-f or map the fused sensor data 10-f to this representative (numerical) vector 13.A module 14 compares the extracted pattern, in particular the representative (numerical) vector 13, with one or more stored (user-specific) patterns 40, which correspond to respective users with respect to the fused sensor data 10-f and which are also converted into the form of a representative (numerical) vector. In this case, a confidence measure is determined, for example in the form of a simple distance measure, which quantifies a distance between a stored pattern 40 and the extracted pattern so that the latter can be compared with a predetermined threshold value. An optional module 17 can provide for a temporal aggregation of comparison results determined over time.If the comparison reveals that the distance between the patterns is below the specified threshold, user identification for the respective user was successful, and an associated identification code is released. During the initial identification and / or during the verification of the identification, the positive identification signal 20, which may include the identification code, is then provided, for example, by means of a module 18. Otherwise, a negative identification signal 21 may be provided.

[0067] It can be provided that the initial identification and / or the checking of the at least one termination criterion is carried out individually on the sensor data 10-x of the at least two sensors 2-x, with the results obtained subsequently being merged with one another. This is illustrated schematically in Fig. 3, which shows a flow chart to clarify a data flow. From the sensor data 10-x of each of the sensors 2-x, patterns (e.g., as a respective fingerprint) are extracted in a module 12-1. Alternatively or additionally, the sensor data 10-x can also be evaluated using a trained machine learning model, as already described above. For pattern recognition, the respective module 12-x can, for example, generate a representative (numerical) vector 13-x from the sensor data 10-x or map the respective sensor data 10-x to an associated representative (numerical) vector 13-x.A respective module 14-x compares the respectively extracted pattern, in particular the respective representative (numerical) vector 13-x, with one or more stored patterns 40-x, which correspond to respective users with regard to the respective sensor data 10-x (e.g. voice signatures, movement patterns, etc.) and which are also brought into the form of a representative (numerical) vector. In particular, a confidence measure is determined in each case, for example in the form of a simple distance measure, which quantifies a distance between a respectively stored pattern 40-x and the extracted pattern so that this can be compared with a (respective) predetermined threshold value. The respective comparison results are then fused in a module 11. A filter can be used for this purpose, or, for example, the Dempster-Shafer rule of combination.An optional module 17 can be provided for performing a temporal aggregation of fused comparison results determined over time. If the fused comparison result shows that the distance between the patterns 40-x is below the specified threshold, the user identification for the respective user was successful, and an associated identification code is released. During the initial identification and / or during the verification of the identification, the positive identification signal 20, which may include the identification code, is then provided, for example, by means of a module 18. Otherwise, a negative identification signal 21 can be provided.

[0068] It may be provided that a confidence value is determined and provided for the initial identification and / or an identification following the initial identification.

[0069] It can be provided that a specific and / or aggregated confidence value is reduced taking into account an elapsed time, wherein the confidence value is compared with a predetermined threshold for the confidence, wherein a termination criterion includes falling below the predetermined threshold.

[0070] It can be provided that the initial identification is started when at least one activity criterion 31-x (Fig. 1) is met. The data processing device 3 is configured in particular to evaluate the acquired sensor data 10-x in order to determine the presence of the at least one activity criterion 31-x. The at least one activity criterion 31-x can, for example, comprise the recognition of a predetermined pattern 40 in the acquired sensor data 10-x, for example an audio signature stored for a user and / or the recognition of a (some kind of) movement of the system 1, in particular of the key 60. Furthermore, the presence of the at least one device 25 (e.g., a smartphone) in the vicinity of the system 1, in particular in the vicinity of the key 60, can also be an activity criterion 31-x.It can be provided that, if the initial identification fails, an alternative identification is initiated and / or performed. For example, it can then be provided that a user must enter a password. This can be done, for example, via voice input or text input. The system 1 comprises suitable input means for this purpose (shown). The password is compared with a stored password, and if there is a match, the initial identification is considered successful, and the positive identification signal 20 is provided.

[0071] It can be provided that user data used to identify the user during the initial identification and / or subsequent identification are updated if a data drift is detected. In particular, it can be provided that the update takes into account the confidence value. In particular, the user data can be updated whenever the confidence value is above the specified threshold.

[0072] It can be provided that at least part of the data processing device 3 is designed as a cloud computer 70 (or cloud service). For this purpose, the system 1, in particular the key 60, comprises a communication connection to the cloud computer 70.

[0073] The method and the system according to the further aspect result analogously from the description of the method and the system 1.

[0074] Fig. 4 shows a schematic flow diagram to illustrate embodiments of the method for user identification.

[0075] In a measure 100, sensor data is collected using at least one sensor. In particular, sensor data from multiple sensors is collected (particularly multimodally).

[0076] In a measure 101, an initial identification of a user is performed based on the recorded sensor data. The sensor data is evaluated in particular in the manner described above, for example, through pattern recognition and / or using a trained machine learning method. Other data sources can also be taken into account, e.g., the presence of a connectable device (e.g., a user's smartphone). In a measure 102, a positive identification signal is provided by a communication device, at least in the case of a successful initial identification.

[0077] In a measure 103, after a successful initial identification, at least one predefined termination criterion is checked. If the check shows that at least one termination criterion is not met, the process continues with measure 102, i.e., the positive identification signal continues to be provided, so that (positive) user identification is provided. If, however, the check shows that at least one termination criterion is met, the process returns to measure 100, and the positive identification signal is no longer provided until another successful initial identification is performed. It can be provided that a negative identification signal is provided in a measure 104, which in particular means that no user identity could be determined.

[0078] Measure 100a may stipulate that at least one activity criterion is checked, with the process continuing with measure 101 only if at least one activity criterion is met. Otherwise, the process returns to measure 100.

[0079] Further embodiments of the method have already been described above with reference to the system.

[0080] List of reference symbols

[0081] System -x Sensor

[0082] Data processing device -1 Computing device -2 Memory

[0083] Communication device trained machine learning model 0-x sensor data 0-f fused sensor data 1 module (fusion) 2, 12-x module (pattern extraction) 3, 13-x representative vector 4, 14-x model (comparison) 7 module (temporal aggregation) 8 module 0 positive identification signal 1 negative identification signal 4 coupling 5 (coupleable) device 0-x termination criterion 1 -x activity criterion 0 stored (user-specific) pattern 0-x stored (user-specific) pattern 0 vehicle 1 device (vehicle) 0 electronic key 0 cloud computer 9-104 measures of the procedure

Claims

Patent claims 1. A method for user identification, wherein sensor data (10-x) are acquired by means of at least one sensor (10-x), wherein an initial identification of a user is carried out on the basis of the acquired sensor data (10-x), wherein at least in the case of a successful initial identification a positive identification signal (20) is provided by means of a communication device (4), wherein after a successful initial identification at least one predetermined termination criterion (30-x) is checked, and wherein the positive identification signal is provided until the check shows that at least one predetermined termination criterion (30-x) is present.

2. Method according to claim 1, characterized in that the initial identification for recognizing a user identity comprises a pattern recognition of patterns in the acquired sensor data (10-x).

3. Method according to claim 1 or 2, characterized in that the initial identification for recognizing a user identity comprises applying a trained machine learning model (5) to which at least part of the acquired sensor data (10-x) is supplied as input data.

4. Method according to one of the preceding claims, characterized in that a termination criterion (30-x) includes a loss of a coupling (24) with at least one predetermined, previously signal-coupled, device (25) and / or reaching and / or exceeding a predetermined distance to the predetermined at least one device (25).

5. Method according to one of the preceding claims, characterized in that a termination criterion (30-x) includes an occurrence or absence of at least one predetermined pattern in sensor data (10-x) of at least one sensor (2-x) acquired after the successful initial identification.

6. Method according to one of the preceding claims, characterized in that the positive identification signal (20) is provided until a predetermined number of termination criteria (30-x) and / or a predetermined combination of termination criteria (30-x) is met.

7. Method according to one of the preceding claims, characterized in that for the initial identification, sensor data (10-x) from at least two sensors (2-x) are recorded and taken into account.

8. Method according to one of the preceding claims, characterized in that when checking the at least one termination criterion (30-x), sensor data (10-x) from at least two sensors (2-x) are taken into account.

9. The method according to claim 7 or 8, characterized in that the sensor data (10-x) of the at least two sensors (2-x) are fused before the initial identification and / or before checking the at least one termination criterion (30-x).

10. The method according to claim 7 or 8, characterized in that the initial identification and / or the checking of the at least one termination criterion (30-x) is carried out individually on the sensor data (10-x) of the at least two sensors (2-x), wherein the results obtained are subsequently merged with one another.

11. The method according to claim 9 or 10, characterized in that the fusion comprises processing by means of at least one filter and / or a temporal estimator and / or by means of the Dempster-Shafer rule of combination.

12. Method according to one of the preceding claims, characterized in that a confidence value is determined and provided for the initial identification and / or an identification subsequent to the initial identification.

13. Method according to one of the preceding claims, characterized in that the initial identification is started when at least one activity criterion (31-x) is fulfilled.

14. Method according to one of the preceding claims, characterized in that if the initial identification fails, an alternative identification is initiated and / or carried out.

15. Method according to one of the preceding claims, characterized in that user data used to recognize the user identity during the initial identification and / or a subsequent identification are updated if a data drift is detected.

16. System (1) for user identification, comprising: at least one sensor (2-x) which is configured to acquire sensor data (10-x), a data processing device (3), and a communication device (4), wherein the data processing device (3) is configured to receive sensor data (10-x) acquired by the at least one sensor (2-x) and to carry out an initial identification based on the acquired sensor data (10-x), at least in the case of a successful initial identification, a positive To provide an identification signal (20) via the communication device (4), and to check at least one predetermined termination criterion (30-x) after a successful initial identification, and to provide the positive identification signal (20) until the check shows that at least one predetermined termination criterion (30-x) is present.

17. System (1) according to claim 16, characterized in that at least the at least one sensor (2-x) is part of an electronic key (60).

18. System (1) according to claim 16, characterized in that the system (1) is designed as an electronic key (60).

19. System (1) according to one of claims 16 or 17, characterized in that at least part of the data processing device (3) is designed as a cloud computer (70).

20. A method for user identification, wherein sensor data (10-x) are recorded by means of at least two sensors (2-x), wherein an initial identification of a user is carried out based on the recorded Sensor data (10-x) is carried out, and wherein at least in the case of a successful initial identification, a positive identification signal (20) is provided by means of a communication device (4).

21. System (1) for user identification, comprising: at least two sensors (2-x) which are configured to acquire sensor data (10-x), a data processing device (3), and a communication device (4), wherein the data processing device (3) is configured to receive sensor data (10-x) acquired by the at least two sensors (2-x) and to carry out an initial identification based on the acquired sensor data (10-x), and to provide a positive identification signal (20) via the communication device (4) at least in the case of a successful initial identification.

Images