Sensor and method for generating training data for a machine learning classifier in a sensor

Abstract

A sensor and a method for generating training data for a machine learning classifier in the sensor are disclosed, wherein the method comprises: acquiring a first sensor signal and a second sensor signal by means of the sensor; determining whether the machine learning classifier classifies the first sensor signal into a pre-defined user motion pattern in response to an input of the first sensor signal; determining whether a classification model classifies the second sensor signal into a pre-defined user input for controlling a device in response to an input of the second sensor signal, wherein the user input is assigned to the user motion pattern; determining a training data vector for the machine learning classifier if the second sensor signal has been classified into the pre-defined user input and if the first sensor signal has not been classified into the user motion pattern, wherein the training data vector has the first sensor signal and the user motion pattern.

Description

Technical Field

[0001] Various embodiments generally relate to a sensor and a method for generating training data for a machine learning classifier in the sensor. Background Technology

[0002] Sensors are used in many devices, such as smartphones, various wearable devices (e.g., devices worn on the body and / or head), smart headphones, etc. For example, intelligent sensors (or smart sensors) can be used here, which can preprocess the detected sensor signals. For instance, the detected sensor signals can be preprocessed using a trained machine learning classifier (e.g., a neural network). The machine learning classifier can be trained using training data covering a wide range of users. For example, a machine learning classifier generalized in this way can identify motion patterns.

[0003] In Bourke et al.'s "Fall detection algorithms for real-world fallsharvested from lumbar sensors in the elderly population: A machine learning approach," 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2016, a method for using machine learning to identify fall risks in the elderly is described.

[0004] However, user-specific behaviors (such as movement patterns) may differ from general behaviors. Therefore, it may be necessary to train a general machine learning classifier specifically for the user. User-specific training data is essential for user-specific training of the general machine learning classifier. Therefore, it may also be necessary to generate user-specific training data. Summary of the Invention

[0005] The method and sensor having the features of independent claims 1 (first example) and 12 (thirtieth example) enable the generation of training data for a machine learning classifier in the sensor during use in a device.

[0006] Machine learning classifiers can be any type of classifier that can be adapted (e.g., trained) using machine learning. For example, machine learning classifiers can be decision trees, random forest classifiers, gradient boosting classifiers, support vector machines (SVM), k-nearest neighbor classifiers (KNN), neural networks, and so on. Neural networks can be any type of neural network, such as autoencoder networks, convolutional neural networks (CNNs), variational autoencoder networks (VAEs), sparse autoencoder networks (SAEs), recurrent neural networks (RNNs), deconvolutional neural networks (DNNs), generative adversarial networks (GANs), forward-thinking neural networks, sum-product neural networks, etc. Neural networks can have any number of layers. A pre-trained machine learning classifier can be pre-trained using any type of training principle, and can be further trained using any type of training principle. For example, a machine learning classifier can be a neural network, a pre-trained neural network can be trained using any type of training principle, and a pre-trained neural network can be further trained using any type of training principle, such as backpropagation.

[0007] The classification model can be any type of model that can classify sensor data based on user input. Specifically, the classification model can be designed to determine probability values ​​for all decision options (also known as "soft decision"). The classification model can clearly identify the presence of user input. For example, the classification model can classify sensor signals above a threshold as user input. For example, the classification model can classify sensor signals with a determined probability above a threshold for a single category (e.g., from one of multiple categories) as user input. According to various embodiments, the classification model can be a machine learning classifier as described in the previous paragraph, such as a neural network.

[0008] The generation of user-specific training data has the following effect: a general pre-trained machine learning classifier can be additionally trained using user-specific training data specifically targeting user behavior (e.g., movement patterns).

[0009] Furthermore, the method according to the first example has the effect of determining the training data during the operation of a device that interacts with the sensor.

[0010] The generated training data (e.g., the determined training data vector) can be used, for example, to retrain a machine learning classifier (e.g., a pre-trained machine learning classifier) ​​using methods described herein and / or known to the public. For example, this can improve future detections by leveraging the retrained machine learning classifier. For instance, the number of false positives and / or false negatives determined by the retrained machine learning classifier can be reduced.

[0011] Acquiring a first sensor signal and a second sensor signal by means of a sensor may include: acquiring the first sensor signal within a first time range; and acquiring the second sensor signal within a second time range, wherein the first time range begins before the second time range. The features described in this paragraph, combined with the first example, form the second example.

[0012] The sensor may have a first sensor element and a second sensor element. Acquiring the first sensor signal and the second sensor signal by means of the sensor may include: acquiring the first sensor signal by means of the first sensor element; and acquiring the second sensor signal by means of the first sensor element and / or by means of the second sensor element. The features described in this paragraph, combined with the first example or the second example, form a third example.

[0013] The first sensor element can detect a first physical variable, and the second sensor element can detect a second physical variable that is different from the first physical variable. The features described in this paragraph, combined with the third example, form the fourth example.

[0014] The first sensor element and / or the second sensor element may be micromechanical sensor elements. The combination of features described in this paragraph, either the third or fourth example, forms the fifth example.

[0015] The sensor may have one or more sensor elements selected from the group consisting of: acceleration sensor, rotational speed sensor, magnetic field sensor, pressure sensor, and contact sensor. The features described in this paragraph, combined with one or more of the first through fifth examples, form the sixth example.

[0016] A sensor may have multiple sensor elements; and acquiring a first sensor signal and a second sensor signal by means of the sensor may include: acquiring one or more first sensor signals by means of one or more of the multiple sensor elements; and acquiring one or more second sensor signals by means of one or more of the multiple sensor elements. Determining whether a machine learning classifier classifies a first sensor signal as a pre-given user motion pattern in response to an input of a first sensor signal may include: determining whether a machine learning classifier classifies one or more first sensor signals as a pre-given user motion pattern in response to an input of one or more first sensor signals. Determining whether a classification model classifies a second sensor signal as a pre-given user input for controlling a device in response to an input of a second sensor signal may include: determining whether a classification model classifies one or more second sensor signals as a pre-given user input in response to an input of one or more second sensor signals. The features described in this paragraph, combined with one or more of the first or sixth examples, form a seventh example.

[0017] The method may further include transmitting instructions to the device for controlling the device if the second sensor signal has been classified as a pre-given user input and / or if the first sensor signal has been classified as a user motion pattern. The features described in this paragraph, combined with one or more of the first through seventh examples, form an eighth example. For example, the determined training data vector can only be stored in the sensor, where the device cannot access the training data vector. For example, the training data vector can be processed within the sensor (e.g., at the sensor level, without sensor-device interaction). For example, this increases user data protection.

[0018] The method may further include: acquiring a third sensor signal using a sensor; determining whether a machine learning classifier classifies the third sensor signal as a pre-given user motion pattern in response to input of the third sensor signal; and if the machine learning classifier classifies the third sensor signal as a user motion pattern, transmitting instructions to the device for controlling the device. The features described in this paragraph, together with one or more combinations of the first to eighth examples, form the ninth example.

[0019] The method may further include storing the first sensor signal and the second sensor signal in a ring memory. The features described in this paragraph, combined with one or more of the first through ninth examples, form the tenth example.

[0020] Acquiring a first sensor signal and a second sensor signal using a sensor may include: acquiring a first measurement signal using a sensor; determining the first measurement signal as a first sensor signal if the signal strength of the first measurement signal is greater than a predefined signal threshold; acquiring a second measurement signal using a sensor; and determining the second measurement signal as a second sensor signal if the signal strength of the second measurement signal is greater than a predefined signal threshold. The features described in this paragraph, combined with one or more of the first to tenth examples, form an eleventh example.

[0021] Determining whether a machine learning classifier responds to an input of a first sensor signal and classifies the first sensor signal as a pre-given user motion pattern may include: determining whether a machine learning classifier responds to an input of the first sensor signal and classifies the first sensor signal as a pre-given user motion pattern among a plurality of pre-given user motion patterns; wherein determining whether a classification model responds to an input of a second sensor signal and classifies the second sensor signal as a pre-given user input for controlling the device includes: determining whether a classification model responds to an input of the second sensor signal and classifies the second sensor signal as a pre-given user input among a plurality of pre-given user inputs for controlling the device, wherein each of the plurality of user inputs is assigned to a user motion pattern among a plurality of user motion patterns, wherein if the second sensor signal has been classified as a user input among the plurality of pre-given user inputs and if the first sensor signal has not been classified as a user motion pattern assigned to the determined user input, then a training data vector is determined. The features described in this paragraph, combined with one or more of the first to eleventh examples, form the twelfth example.

[0022] Determining whether a machine learning classifier classifies a first sensor signal as a pre-given user motion pattern in response to input of the first sensor signal may include: determining the probability of each of a plurality of pre-given user motion patterns in response to inputting the first sensor signal into the machine learning classifier; and determining the first sensor signal as a pre-given user motion pattern if the probability determined for one of the plurality of user motion patterns is greater than a predefined first probability threshold. The features described in this paragraph, combined with one or more of the first through twelfth examples, form the thirteenth example.

[0023] The method may further include: acquiring a third sensor signal using a sensor; determining the probability of each of a plurality of pre-given user motion patterns in response to inputting the third sensor signal into a machine learning classifier; wherein a training data vector for the machine learning classifier is determined if, in response to inputting the third sensor signal into the machine learning classifier, the probability determined for one of the plurality of user motion patterns is greater than a predefined first probability threshold, and in response to inputting the first sensor signal into the machine learning classifier, the probability determined for said user motion pattern is less than the predefined first probability threshold and greater than a predefined second probability threshold. The features described in this paragraph, combined with the thirteenth example, form the fourteenth example.

[0024] The training data vector for the machine learning classifier can be determined under the following conditions: the second sensor signal is not classified as user input; in response to inputting the first sensor signal into the machine learning classifier, the probability determined for the user motion pattern assigned to the user input among multiple user motion patterns is less than a predefined first probability threshold and greater than a predefined second probability threshold; in response to inputting the third sensor signal into the machine learning classifier, the probability determined for the user motion pattern assigned to the user input among multiple user motion patterns is greater than the predefined first probability threshold. The features described in this paragraph, combined with the fourteenth example, form the fifteenth example.

[0025] The method may further include storing the training data vector in a FIFO memory. The features described in this paragraph are combined with one or more of the first through fifteenth examples to form the sixteenth example.

[0026] The machine learning classifier can be a pre-trained machine learning classifier. The features described in this paragraph are combined with one or more of the first through sixteenth examples to form the seventeenth example.

[0027] This method may also include: additionally training a machine learning classifier using the training data vectors. The features described in this section, combined with the seventeenth example, form the eighteenth example. The additional training can be performed internally by the sensor (e.g., through local learning). This has the effect of increasing user data protection.

[0028] Additional training of the machine learning classifier using the training data vector may include: determining the probability of each pre-given user motion pattern for each pre-given user motion pattern in response to a first sensor signal input to the machine learning classifier assigned to the training data vector; determining a loss value using the user motion patterns assigned to the training data vector and the probabilities of the user motion patterns determined by the machine learning classifier, with the aid of a loss function; the additional training of the machine learning classifier reduces the loss value determined using the training data vector with the aid of the loss function. The features described in this paragraph, combined with Example 18, constitute Example 19.

[0029] Additional training of the machine learning classifier using the training data vector may include: determining the probability of each of a plurality of pre-given user motion patterns in response to a first sensor signal input to the machine learning classifier assigned to the training data vector; determining a loss value using the user motion patterns assigned to the training data vector and the probabilities of the user motion patterns determined by the machine learning classifier, with the aid of a loss function; and further training the machine learning classifier using the determined loss value. The features described in this paragraph, combined with the eighteenth or nineteenth examples, constitute the twentieth example.

[0030] Acquiring a first sensor signal and a second sensor signal using sensors may include: acquiring the first sensor signal within a first time range and acquiring the second sensor signal within a second time range, wherein the first time range begins before the second time range. Additional training of the machine learning classifier using a training data vector may include: preprocessing the first sensor signal assigned to the training data vector such that a first time range of the first sensor signal is reduced to a time range smaller than the first time range; and further training the machine learning classifier using the preprocessed first sensor signal and a user motion pattern assigned to the training data vector. The features described in this paragraph, combined with one or more of Examples 18 through 20, form Example 21.

[0031] The training data vector can be a false negative training data vector. Determining whether a machine learning classifier classifies a first sensor signal as a pre-given user motion pattern in response to input of a first sensor signal may include: determining the probability of each of a plurality of pre-given user motion patterns in response to input of the first sensor signal into the machine learning classifier; determining the first sensor signal as a pre-given user motion pattern if the probability determined for one of the plurality of user motion patterns is greater than a predefined first probability threshold; wherein the method further includes: determining a false positive training data vector for the machine learning classifier if a second sensor signal is not classified as user input, and if the probability determined for the user motion pattern assigned to the user input among the plurality of user motion patterns is less than a predefined first probability threshold and greater than a predefined second probability threshold, wherein the false positive training data vector has the first sensor signal and the user motion pattern assigned to the user input. The features described in this paragraph, combined with one or more of the first to twenty-first examples, form the twenty-second example.

[0032] The method may further include: acquiring a third sensor signal using a sensor; determining the probability of each of a plurality of pre-given user motion patterns in response to the third sensor signal being input into a machine learning classifier; wherein a false positive training data vector is determined for the machine learning classifier if: a second sensor signal is not classified as user input; the probability of a user motion pattern assigned to the user input among the plurality of user motion patterns is less than a predefined first probability threshold and greater than a predefined second probability threshold in response to the first sensor signal being input into the machine learning classifier; and the probability of a user motion pattern assigned to the user input among the plurality of user motion patterns is less than the predefined first probability threshold in response to the third sensor signal being input into the machine learning classifier. The features described in this paragraph, combined with the twenty-second example, form the twenty-third example.

[0033] The method may further include storing the false positive training data vector in an additional FIFO memory. The features described in this paragraph are combined with the twenty-second or twenty-third example to form the twenty-fourth example.

[0034] The method may further include: additionally training the machine learning classifier using false positive training data vectors and / or false negative training data vectors. The features described in this paragraph, in combination with one or more of the examples from examples twenty-two through twenty-four, form example twenty-five.

[0035] The method may further include: determining whether the device is in sleep mode and / or battery charging mode; wherein if the device is in sleep mode and / or battery charging mode, a machine learning classifier is additionally trained using false positive training data vectors and / or false negative training data vectors. The features described in this paragraph, combined with Example Twenty-Five, form Example Twenty-Six.

[0036] A computer program product may store program instructions that, when executed, perform one or more methods according to the first to the twenty-sixth examples. A computer program product having the features described in this paragraph forms the twenty-seventh example.

[0037] The non-volatile storage medium can store program instructions that, when executed, perform one or more methods from the first to the twenty-sixth examples. A non-volatile storage medium having the features described in this paragraph forms the twenty-eighth example.

[0038] The non-volatile storage medium can store program instructions that, when executed, perform one or more methods from the first to the twenty-sixth examples. A non-volatile storage medium having the features described in this paragraph forms the twenty-ninth example.

[0039] The sensor may have: one or more sensor elements configured to acquire a first sensor signal and a second sensor signal; a computer configured to determine, in response to inputting the first sensor signal into a machine learning classifier, whether the machine learning classifier classifies the first sensor signal as a pre-given user motion pattern; in response to inputting the second sensor signal into a classification model, whether the classification model classifies the second sensor signal as a pre-given user input for controlling the device, wherein the user input is assigned to the user motion pattern; and if the second sensor signal has been classified as the pre-given user input, and if the first sensor signal has not been classified as the user motion pattern, determine a training data vector for the machine learning classifier, wherein the training data vector has the first sensor signal and the user motion pattern. A sensor having the features described in this paragraph forms the thirty-first example.

[0040] The computer can also be configured to train a machine learning classifier using training data vectors. The features described in this paragraph are combined with the thirtieth or thirty-first example to form the thirty-second example.

[0041] An interface is provided, configured to transmit instructions for controlling the device to the execution means of the device. The features described in this paragraph, in combination with one or more of the thirty to thirty-second examples, form the thirty-third example.

[0042] The computer can also be configured to determine whether the device is in sleep mode and / or battery charging mode; and if the device is in sleep mode and / or battery charging mode, the computer can be configured to train a machine learning classifier using training data vectors. The features described in this paragraph, combined with the thirty-third example, form the thirty-fourth example.

[0043] The computer may be further configured to transmit instructions to the execution device via the interface if the first sensor signal has been classified as a user motion pattern, wherein the instruction is assigned to the user motion pattern; and / or if the second sensor signal has been classified as user input, wherein the instruction is assigned to the user input. The features described in this paragraph, combined with the thirty-third or thirty-fourth example, form the thirty-fifth example.

[0044] A system may include one or more sensors according to Examples 33 to 35. The system may also have an actuator configured to receive instructions from the sensors and execute one or more control actions assigned to the received instructions. A system having the features described in this paragraph forms Example 36. Attached Figure Description

[0045] Embodiments of the present invention are shown in the accompanying drawings and explained in more detail in the following description.

[0046] Figures 1A to 1M Sensors according to various embodiments are shown; Figures 2A to 2E Exemplary detected signals according to various embodiments are shown; Figure 3A and 3B A system having sensors according to various embodiments is shown; Figure 4 Methods for generating training data for machine learning classifiers in sensors according to various embodiments are illustrated; Figure 5 Methods for generating training data for machine learning classifiers in sensors according to various embodiments are shown. Detailed Implementation

[0047] In one embodiment, "computer" can be understood as any type of logical implementation entity, which can be hardware, software, firmware, or a combination thereof. Thus, in one embodiment, "computer" can be hardwired logic circuitry or programmable logic circuitry, such as a programmable processor, such as a microprocessor (e.g., CISC (Big Data Set Processor) or RISC (Reduced Instruction Set Processor)). "Computer" can also be software implemented or executed by a processor, such as any type of computer program, such as a computer program using virtual machine code like Java. Consistent with the alternative embodiments, any other type of implementation of the corresponding functionality described in more detail below can be understood as "computer".

[0048] Smart sensors can identify a user's motion patterns using a machine learning classifier generalized to many users. However, user-specific motion patterns can differ from generalized motion patterns. Therefore, it may be necessary to train the general machine learning classifier specifically for the user. User-specific training data is necessary for user-specific training of the general machine learning classifier. Therefore, it may also be necessary to generate user-specific training data. Various embodiments relate to apparatus and methods for generating training data for a machine learning classifier in a sensor during use in a device. The device can obviously have a sensor with a pre-trained machine learning classifier, and the machine learning classifier can also be additionally trained specifically for that user during use of the device.

[0049] Figures 1A to 1M A sensor 100 according to various embodiments is shown. The sensor 100 may be, for example, a smart sensor (e.g., a Smart Sensor).

[0050] Figure 1A A sensor 100 according to various embodiments is illustrated. The sensor 100 may include one or more sensor elements 102. For example, the sensor 100 may have one or more sensor elements 102 selected from the group consisting of: an acceleration sensor, a rotational speed sensor, a magnetic field sensor, a pressure sensor, a touch sensor, etc. For example, an acceleration sensor may be configured to determine acceleration (e.g., an acceleration vector) in each of three spatial directions. For example, a rotational speed sensor may be configured to determine rotational speed (e.g., a rotational speed vector) in each of three spatial directions. For example, a magnetic field sensor may be configured to determine magnetic fields (e.g., magnetic field changes) in each of three spatial directions (e.g., magnetic field vectors). For example, a pressure sensor may be configured to detect pressure (e.g., a scalar pressure value). According to various embodiments, at least one of the one or more sensor elements 102 may be a micromechanical sensor element.

[0051] One or more sensor elements 102 may be configured to acquire a first sensor signal 104. One or more sensor elements 102 may be configured to acquire a second sensor signal 104. One or more sensor elements 102 may be configured to acquire the first sensor signal 104 within a first time range. One or more sensor elements 102 may be configured to acquire a second sensor signal 106 within a second time range. The first time range may begin before the second time range. For example, the start of the second sensor signal 106 may be after the start of the first sensor signal 104. According to various embodiments, the second sensor signal 106 may be detected after the first sensor signal 104.

[0052] Sensor 100 may also include computer 108. As described above, computer 108 can be any type of logical implementation entity. For example, computer 108 may include one or more processors. Computer 108 may be configured to process sensor signals (e.g., first sensor signal 104, e.g., second sensor signal 106) detected by one or more sensor elements 102. According to various embodiments, one or more sensor elements 102 may be configured to provide digital sensor signals and computer 108 may be configured to process digital sensor signals.

[0053] refer to Figure 1B One or more sensor elements 102 may have a first sensor element 102a and a second sensor element 102b. The first sensor element 102a may be configured to acquire a first sensor signal 104. The second sensor element 102b may be configured to acquire a second sensor signal 106. According to various embodiments, the first sensor element 102a may be configured to detect a first physical variable (e.g., acceleration, rotational speed, magnetic field). The second sensor element 102b may be configured to detect a second physical variable (e.g., acceleration, pressure, temperature change). The second physical variable may be different from the first physical variable.

[0054] According to various embodiments, one or more sensor elements 102 may have multiple sensor elements. For example, one or more sensor elements 102 may have multiple first sensor elements and multiple second sensor elements. Each of the multiple first sensor elements can acquire a first sensor signal 104. One or more of the multiple first sensor elements can detect physical variables that are different from each other. Each of the multiple second sensor elements can acquire a second sensor signal 106. One or more of the multiple second sensor elements can detect physical variables that are different from each other.

[0055] The embodiments are described below based on the first sensor signal 104 and the second sensor signal 106. However, it should be noted that multiple first sensor signals (acquired, for example, by means of multiple first sensor elements) and / or multiple second sensor signals (acquired, for example, by means of multiple second sensor elements) can be used. For example, the processing described below for the first sensor signal 104 can be performed on the multiple first sensor signals (e.g., together, for example, by means of a machine learning classifier, all the multiple first sensor signals are processed, for example, sequentially, wherein each of the multiple first sensor signals can be processed accordingly into the first sensor signal 104). For example, the processing described below for the second sensor signal 106 can be performed on the multiple second sensor signals (e.g., together, for example, by means of a classification model, all the multiple second sensor signals are processed, for example, sequentially, wherein each of the multiple second sensor signals can be processed accordingly into the second sensor signal 106). It should also be noted that this also applies to the third sensor signal described herein.

[0056] The first sensor signal 104 can be assigned to a user motion pattern (e.g., a gesture). For example, the first sensor element 102a can have an accelerometer, a rotation speed sensor, a magnetic field sensor, or a pressure sensor. For example, the device can have a sensor 100 and the device 100 can be connected to a user (e.g., worn by the user, such as a smartwatch), such that the first sensor signal 104 acquired by means of the first sensor element 102a (e.g., an activation gesture of a smartwatch) is generated by the user's movement. The second sensor signal 106 can be assigned to user input (e.g., input from the user detected by means of a sensor element). For example, user input can be acquired by means of an accelerometer. For example, the device can have an input device, such as a touchscreen, one or more buttons, one or more buttons, etc., and the user can provide input by means of the input device. For example, user input can be acquired as the second sensor signal 106 by means of the second sensor element 102b. According to various embodiments, the second sensor element 102b can acquire user input (e.g., input detected by means of an accelerometer, such as pressing a button, such as touching a touchscreen). For example, the acquisition of the second sensor signal 106 by means of the second sensor element 102b can be reliably (e.g., explicitly) assigned to the user input (e.g., since the second sensor signal 106 acquired by means of the user input by means of the accelerometer has a greater signal strength than the motion pattern, for example, if the touch screen is touched, the second sensor signal 106 is acquired by means of the touch screen, and if the touch screen is not touched, the second sensor signal 106 is not acquired).

[0057] According to various embodiments, sensor 100 may have a first storage device 110. The first storage device 110 may be configured to store sensor signals (e.g., first sensor signal 104, e.g., second sensor signal 106). The first storage device 110 may have at least one memory. The memory used in the embodiments may be a volatile memory such as DRAM (Dynamic Random Access Memory), or a non-volatile memory such as PROM (Programmable Read-Only Memory), EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or flash memory, such as a memory device with a floating gate, a charge-capture memory device, MRAM (Magnetoresistive Random Access Memory), or PCRAM (Phase Change Direct Access Memory). The first storage device 110 may be configured to store sensor signals and provide the sensor signals to computer 108. According to various embodiments, the first storage device 110 may be a ring memory. The ring memory may be configured to continuously store signals acquired by one or more sensor elements 102. The ring memory may be configured to overwrite signals acquired by one or more sensor elements 102 after a predetermined time period and / or according to a predetermined maximum memory occupancy of the ring memory. For example, a ring memory can be configured to store acquired signals for up to approximately 20 seconds (e.g., up to approximately 10 seconds).

[0058] According to various embodiments, one or more sensor elements 102 can acquire continuous signals. For example, sensor 100 can be configured to continuously acquire and process signals (e.g., to identify user motion patterns and / or user input). For example, continuous signals can be acquired at frequencies from approximately 10 kHz to approximately 1 kHz. For example, continuous signals can be provided as digital signals. For example, digital values ​​(e.g., with a resolution of approximately 8 bits to approximately 16 bits) can be provided for physical variables detected by means of one or more sensor elements 102.

[0059] According to various embodiments, each of the one or more sensor elements 102 can acquire continuous signals. For example, sensor 100 can be configured to continuously acquire and process signals (e.g., to identify user motion patterns and / or user input) by means of each of the one or more sensor elements 102. For example, each of the one or more sensor elements 102 can detect a physical quantity that is different from one or more other sensor elements of the one or more sensor elements 102.

[0060] Computer 108 can be configured to determine signals of continuous signals within a predefined prior time range as sensor signals at pre-given time intervals (e.g., a pre-given clock). Computer 108 can be configured to determine signals at a pre-given clock (e.g., at a clock of 100 Hz, or 1 kHz) selected in the range of approximately 10 Hz to approximately 10 kHz. For example, a prior time range can be selected from approximately 0.5 s to approximately 10 s (e.g., at a clock of 10 Hz, or 1 kHz). According to various embodiments, a prior time range can be selected based on the average duration of the user motion pattern to be acquired (e.g., in the range of approximately 1 second to approximately 3 seconds). According to various embodiments, a prior time range can be selected based on the average duration of the user input to be acquired. For example, a first prior time range can be selected for a first sensor element 102a based on the average duration of the user motion pattern to be acquired. For example, a second prior time range can be selected for a second sensor element 102b based on the average duration of the user input to be acquired. According to various embodiments, the determination of the sensor signal can be a sliding window method. For example, the pre-given clock may be less than a pre-given previous time range, so that the successively determined sensor signals may have at least partially overlapping time ranges.

[0061] Exemplary sensor signals in Figures 2A to 2E As shown in the figure. These exemplary sensor signals are referenced in the description of sensor 100 for the purpose of illustrating the sensor 100. It should be noted that one or more sensor elements 102 can provide a first sensor signal 104 of any type assigned to a user's motion pattern and a second sensor signal 106 of any type assigned to user input.

[0062] Figure 2A A graph 200A is shown showing the signal acquired by one or more sensor elements 102 as a function of time. For example, the signal shown in graph 200A may be a superposition of multiple signals acquired by multiple sensor elements of one or more sensor elements 102. For example, the displayed signals may be assigned to different physical quantities. According to various embodiments, one of the one or more sensor elements 102 may be configured to acquire a signal (e.g., measure a signal) and determine the acquired signal as a sensor signal if the signal strength is greater than a predefined signal threshold. For example, if the signal strength detected by means of a first sensor element 102a is greater than a predefined first signal threshold 210, then the first sensor element 102a may determine a first sensor signal 104. For example, the first sensor element 102a may acquire the detected signal as the first sensor signal 104 independently of the signal strength (e.g., by means of a sensor element as referenced herein). Figure 1C The feature extraction determines the features of the acquired first sensor signal 104. For example, the first sensor element 102a may acquire the first sensor signal 104 within a first time range 202. For example, if the signal strength of a signal detected by means of the second sensor element 102b is greater than a predefined second signal threshold 212, the second sensor element 102b may determine the second sensor signal 106. According to various embodiments, a predefined signal threshold may be assigned to each of the one or more sensor elements 102 (e.g., according to the sensor type of the respective sensor element). For example, the second sensor element 102b may acquire the second sensor signal 106 within a second time range 204. According to various embodiments, the computer 108 may be configured to determine whether to acquire the second sensor signal 106 within a predefined time range 206 after the start of the first time range 202. For example, the computer 108 may be configured to determine whether to acquire the second sensor signal 106 after the first sensor signal 104. As described above, the first sensor signal 104 may be assigned to a user motion pattern and the second sensor signal 106 may be assigned to user input. Computer 108 can clearly determine whether user input follows a user motion pattern (e.g., whether user input responds to a user motion pattern).

[0063] refer to Figure 1C Computer 108 can be configured to implement a pre-trained machine learning classifier. According to various embodiments, the machine learning classifier has been pre-trained such that it classifies sensor signals in response to input (e.g., first sensor signal 104). As described herein, the machine learning classifier can be any type of classifier that can be adapted (e.g., trained) by means of machine learning. According to various embodiments, inputting the sensor signal into the machine learning classifier can include feature extraction for determining features of the sensor signal and processing the determined features of the sensor signal by means of the machine learning classifier. For example, the machine learning classifier can be a decision tree, random forest classifier, gradient boosting classifier, support vector machine, nearest neighbor classifier, etc., and inputting the sensor signal into the machine learning classifier can have feature extraction, and the machine learning classifier can use the determined features to perform classification. For example, features of the sensor signal determined by means of feature extraction can be output in a feature table. Features of the sensor signal determined by means of feature extraction can have, for example, the average value of the sensor signal, the minimum value of the sensor signal, the maximum value of the sensor signal, the mean square value of the sensor signal, amplitude, cross-correlation with a test function, etc. For example, a machine learning classifier can be a neural network, and sensor signals can be directly input into the neural network (e.g., without prior feature extraction).

[0064] The following describes an embodiment using a neural network 114 as a machine learning classifier. However, it should be noted that the machine learning classifier, as described in the previous paragraph, can also be another type of classifier that can be trained using machine learning, and feature extraction can be performed on different types of classifiers.

[0065] According to various embodiments, the neural network 114 implemented by means of computer 108 has been pre-trained, i.e., the pre-trained neural network 114 classifies sensor signals in response to input of sensor signals (e.g., first sensor signal 104). For example, the pre-trained neural network 114 can classify sensor signals into user motion patterns or into non-user motion patterns. For example, the pre-trained neural network 114 can classify sensor signals into a pre-given user motion pattern among a plurality of pre-given user motion patterns or into a pre-given user motion pattern other than a plurality of pre-given user motion patterns in response to input of sensor signals. According to various embodiments, the pre-trained neural network can be a classification network and can be configured to determine the probability of each of a plurality of categories in response to input of sensor signals. Each of the plurality of categories can be assigned to one of a plurality of pre-given user motion patterns. For example, computer 108 can be configured to determine the pre-given user motion pattern if the probability of the category assigned to the pre-given user motion pattern among the plurality of categories determined by the pre-trained neural network 114 is greater than a predefined probability threshold. For example, if a pre-trained neural network 114 determines a probability less than a predefined probability threshold for each of multiple categories, then computer 108 can determine that a pre-given user motion pattern has not been acquired.

[0066] According to various embodiments, a neural network 114 can be pre-trained for sensor signals (e.g., sensor signals from an accelerometer, speed sensor, pressure sensor, or magnetic field sensor) describing multiple user motion patterns. For example, the neural network 114 may have been pre-trained for user motion patterns from various users. For example, the training dataset may have multiple sensor signals. Each of the multiple sensor signals may be assigned to one of the multiple user motion patterns. The training dataset may have sensor signals labeled (e.g., characterized) with user motion patterns (e.g., exactly one user motion pattern is assigned to each of the multiple sensor signals).

[0067] For example, multiple sensor signals can be assigned to each user's motion pattern among multiple user motion patterns. The sensor signals among the multiple sensor signals can have motion patterns of multiple users (e.g., users of different ages, users from different geographical locations, users of different genders, etc.).

[0068] It appears that sensor signals can describe a user's motion patterns, and the training dataset can have multiple sensor signals for each of multiple motion patterns, and multiple user sensor signals for each motion pattern. The neural network 114 can be pre-trained to classify a sensor signal as either a user motion pattern among multiple user motion patterns or a user motion pattern not among multiple user motion patterns in response to input from a sensor signal among the multiple sensor signals. The user motion pattern determined by the neural network 114 is compared with the user motion pattern assigned to that sensor signal, and an error is determined. This error is then used to train the neural network 114 (e.g., to reduce errors). This can be performed for each of the multiple sensor signals in the training dataset.

[0069] As described above, the neural network 114 can be configured to determine the probability of each of a plurality of categories. A category (e.g., a category assigned to a user's motion pattern) can be assigned to each of the plurality of sensor signals in the training dataset. According to various embodiments, the neural network 114 can be pre-trained such that, in response to an input of a sensor signal among the plurality of sensor signals, the neural network 114 determines the probability of each of the plurality of categories for that sensor signal, determines a loss value between the category assigned to the sensor signal and the probability determined for that category using a loss function, and trains the neural network 114 using the loss value to reduce the loss value. This can be performed for each of the plurality of sensor signals in the training dataset.

[0070] As described above, sensor 100 may have multiple first sensor elements capable of acquiring multiple first sensor signals. The neural network 114 described herein may be configured to classify the multiple first sensor signals into a pre-given user motion pattern among multiple user motion patterns in response to input from the multiple first sensor signals. For example, the pre-trained neural network 114 may be configured to process sensor signals from an accelerometer, a rotation speed sensor, a pressure sensor, and / or a magnetic field sensor. The pre-training of the neural network 114 may substantially correspond to the pre-training described above, wherein each of the multiple sensor signals in the training dataset has multiple sensor signals.

[0071] The pre-trained neural network 114 can be configured to, in response to an input to the first sensor signal 104, classify the first sensor signal 104 as a pre-given user motion pattern (e.g., a gesture) or as a non-pre-given user motion pattern. The pre-trained neural network 114 can also be configured to, in response to an input to the first sensor signal 104, classify the first sensor signal 104 as a pre-given user motion pattern among a plurality of pre-given user motion patterns or as a pre-given user motion pattern not among a plurality of pre-given user motion patterns. The computer 108 can be configured to determine whether the pre-trained neural network 114 classifies the first sensor signal 104 as a pre-given user motion pattern (e.g., a pre-given user motion pattern among a plurality of pre-given user motion patterns) in response to an input to the first sensor signal 104.

[0072] Computer 108 may also be configured to implement classification model 116. Classification model 116 may be configured to process second sensor signal 106. Classification model 116 may be configured to classify second sensor signal 106 as a pre-given user input or as a non-pre-given user input. User input may be user input for controlling a device (e.g., a device having sensor 100, such as user input for performing control actions of said device). Classification model 116 may be configured to classify second sensor signal 106 as a pre-given user input among a plurality of pre-given user inputs or as a pre-given user input not among a plurality of pre-given user inputs. Each of the plurality of pre-given user inputs may be (e.g., bijectively) assigned to (e.g., exactly one) a plurality of pre-given user motion patterns. Computer 108 may be configured to determine whether classification model 116 classifies second sensor signal 106 as a pre-given user input (e.g., a pre-given user input among a plurality of pre-given user inputs) in response to input from second sensor signal 106.

[0073] According to various embodiments, computer 108 may be configured to determine training data vector 122 for neural network 114 if second sensor signal 106 has been classified as a pre-given user input and if first sensor signal 104 has not been classified as a user motion pattern assigned to the pre-given user input. Figure 2A An exemplary second sensor signal 106 that has been classified as a pre-given user input is shown, and an exemplary first sensor signal 104 that has not been classified as a user motion pattern assigned to a pre-given user input is shown.

[0074] As described above, the training dataset used to train the neural network 114 may have an assigned user motion pattern for each sensor signal. The training data vector 122 may include a first sensor signal 104 and a user motion pattern assigned to the first sensor signal (e.g., a user motion pattern assigned to user input). According to various embodiments, the pre-trained neural network 114 may be additionally (e.g., user-specifically) trained using the training data vector 122.

[0075] It appears that a first sensor signal 104 can be acquired in response to a user's movement (e.g., a gesture), where the neural network 114 does not recognize the movement as a pre-given movement pattern. A second sensor signal 106 can be an input from the user in response to a user movement not recognized as a pre-given movement pattern, to, for example, execute a control action assigned by the device to the pre-given movement pattern and assigned to the user input. It appears that the computer 108 can determine that the user's movement should elicit a desired control action, and the computer 108 can be configured to determine the first sensor signal 104 and the user movement pattern assigned to the desired control action as a training data vector 122. It appears that the training data vector 122 represents a user-specific training data vector used to specifically train the neural network 114.

[0076] refer to Figure 1D The computer 108 can be configured to transmit instructions 124 for controlling the device to the device. The computer 108 can be configured to transmit a first instruction 124a for controlling the device to the device if the first sensor signal 104 has been classified as a pre-given user motion pattern (e.g., a pre-given user motion pattern of multiple pre-given user motion patterns). The first instruction 124a can be assigned to a user motion pattern. It appears that the user intends to perform a desired control action on the device by means of the pre-given user motion pattern, and the first instruction 124a can be an instruction for performing the desired control action. Figure 2B A graph 200B is shown, showing an exemplary first sensor signal 104 that has been classified as a pre-given user motion pattern.

[0077] Computer 108 can be configured to transmit a second instruction 124b for controlling the device if the second sensor signal 106 has been classified as a pre-given user input (e.g., a pre-given user input among multiple pre-given user inputs). The second instruction 124b can be assigned to the user input. It appears that the user intends to perform a desired control action on the device based on the user input, and the second instruction 124b can be an instruction for performing the desired control action.

[0078] According to various embodiments, as described above, a sliding window method can be performed, in which computer 108 can determine a plurality of sensor signals using continuous signals acquired by means of one or more sensor elements 102. The time ranges of successive sensor signals of the plurality of sensor signals may at least partially overlap. For example, computer 108 can determine a first plurality of sensor signals using continuous signals acquired by means of a first sensor element 102a. For example, computer 108 can determine a second plurality of sensor signals using continuous signals acquired by means of a second sensor element 102b. Computer 108 can be configured to determine, for each of the first plurality of sensor signals, whether neural network 114 classifies the sensor signal as a pre-given user motion pattern in response to the input of the corresponding sensor signal. Computer 108 can be configured to determine, for each of the second plurality of sensor signals, whether classification model 116 classifies the sensor signal as a pre-given user input in response to the input of the corresponding sensor signal.

[0079] According to various embodiments, computer 108 may be configured to determine a first number of sensor signals from a first plurality of sensor signals that have been classified by means of neural network 114 into a pre-given user motion pattern (e.g., the same user motion pattern among a plurality of user motion patterns) within a predetermined time range (e.g., within approximately three seconds, approximately five seconds, or approximately ten seconds). Computer 108 may be configured to send a first instruction 124a for controlling the device only once if the first number of pre-given user motion patterns is greater than a first number threshold (e.g., greater than one). It appears that for overlapping time ranges, multiple sensor signals can be identified as user-controlled device motion, and computer 108 can send only the first instruction 124a to the device, thereby, for example, not performing the control action multiple times.

[0080] According to various embodiments, computer 108 can be configured to determine a second number of sensor signals from a second plurality of sensor signals that have been classified as a pre-given user input (e.g., the same user input among multiple user inputs) by means of classification model 116 within a predefined time range (e.g., within approximately three seconds, approximately five seconds, or approximately ten seconds). Computer 108 can be configured to transmit a second instruction 124b for controlling the device only once if the second number for the pre-given user input is greater than a second number threshold (e.g., greater than one). It appears that for overlapping time ranges, multiple sensor signals can be identified as user inputs for controlling the device, and computer 108 can transmit only the second instruction 124b to the device, thereby, for example, not performing the control action multiple times.

[0081] According to various embodiments, computer 108 can be configured to determine the sum of a first quantity and a second quantity within the same predefined time range. Computer 108 can be configured to send only one instruction 120 for controlling the device if the sum of the first and second quantities is greater than a third quantity threshold (e.g., greater than one). In this way, it appears that multiple activations of the control action can be prevented.

[0082] For example, a timestamp can be assigned to each of the multiple sensor signals, and the computer 108 can be configured to use the assigned timestamps to determine the corresponding number of sensor signals within a predefined time range.

[0083] Similarly, computer 108 can be configured to determine the number of training data vectors identified as a pre-given user motion pattern (e.g., the same user motion pattern among multiple user motion patterns) within a predefined time range (e.g., within approximately three seconds, approximately five seconds, or approximately ten seconds). Computer 108 can be configured to store the training data vectors only once if multiple training data vectors have already been identified for the same user motion pattern among multiple user motion patterns within the predefined time range (e.g., stored in a second storage device 140 in the case of false negative training data vectors, or stored in a third storage device 142 in the case of false positive training data vectors; see also). Figure 1I According to various embodiments, the predefined time range may be approximately one day, and if the number of training data vectors determined within a day is greater than a predefined error threshold (e.g., greater than 100), the computer 108 may determine a fault.

[0084] refer to Figure 1E One or more sensor elements 102 can be configured to acquire a third sensor signal 126. For example, a first sensor element 102a can also be configured to acquire a third sensor signal 126. The third sensor signal 126 can be assigned to a user motion pattern (e.g., gesture).

[0085] According to various embodiments, a pre-trained neural network 114 can be configured to process a third sensor signal 126. The pre-trained neural network 114 can be configured, in response to an input to the third sensor signal 126, to classify the third sensor signal 126 as a pre-given user motion pattern or to classify the third sensor signal 126 as a non-pre-given user motion pattern. The pre-trained neural network 114 can also be configured, in response to an input to the third sensor signal 126, to classify the third sensor signal 126 as a pre-given user motion pattern among multiple pre-given user motion patterns or to classify the third sensor signal 126 as a pre-given user motion pattern other than multiple pre-given user motion patterns. A computer 108 can be configured to determine whether the pre-trained neural network 114 classifies the third sensor signal 126 as a pre-given user motion pattern (e.g., a pre-given user motion pattern among multiple pre-given user motion patterns) in response to an input to the third sensor signal 126.

[0086] According to various embodiments, the pre-trained neural network 114, as described above, can be configured to determine the probability of each of a plurality of pre-given user motion patterns in response to input of a sensor signal. For example, the pre-trained neural network 114 can be configured to determine the probability of each of a plurality of first categories in response to input of a sensor signal. Each of the plurality of first categories can be assigned (e.g., bijective assignment) to a pre-given user motion pattern among the plurality of pre-given user motion patterns. The computer 108 can be configured to determine the first sensor signal 104 as a pre-given user motion pattern among the plurality of user motion patterns if the probability of the pre-trained neural network 114 determining the pre-given user motion pattern is greater than a predefined first probability threshold. The computer 108 can be configured to determine whether the pre-trained neural network 114, in response to input of the first sensor signal 104, has determined that the probability of the pre-given user motion pattern among the plurality of pre-given user motion patterns determined by means of the neural network 114 is greater than a predefined first probability threshold. The predefined first probability threshold may, for example, have a probability value selected from a range of approximately 0.5 to approximately 1 (e.g., a probability value of 0.9, a probability value of 0.8, a probability value of 0.7, a probability value of 0.6).

[0087] Computer 108 can be configured to determine whether the pre-trained neural network 114, in response to the input of the first sensor signal 104, determines a given user motion pattern among a plurality of given user motion patterns. The probability determined by the neural network 114 is less than a predefined first probability threshold and greater than a predefined second probability threshold. The predefined second probability threshold can be less than the predefined first probability threshold. The predefined second threshold can, for example, have a probability value selected in the range of approximately 0.25 to approximately 0.8 (e.g., a probability value of 0.8, a probability value of 0.7, a probability value of 0.6, a probability value of 0.5, a probability value of 0.3).

[0088] The computer 108 can be configured to determine the training data vector 122 if, in response to the input of the third sensor signal 126 into the neural network 114, the probability of determining a user motion pattern among multiple user motion patterns is greater than a predefined first probability threshold, and if, in response to the input of the first sensor signal 104 into the neural network 114, the probability of determining a user motion pattern is less than the predefined first probability threshold and greater than a predefined second probability threshold. Figure 2C A graph 200C is shown with an exemplary first sensor signal 104 and an exemplary third sensor signal 126. For the exemplary first sensor signal 104, the probability of determining a user motion pattern is less than a first probability threshold and greater than a second probability threshold. For the exemplary third sensor signal 126, the probability of determining a user motion pattern is greater than the first probability threshold. The third sensor signal 126 can be detected within a third time range 214. The first time range 202 can begin before the third time range 214. For example, the start of the third sensor signal 126 can occur after the start of the first sensor signal 104. According to various embodiments, the third sensor signal 126 can be detected after the first sensor signal 104 (e.g., within a predefined time range 206).

[0089] It appears that a first sensor signal 104 can be acquired in response to a user's first movement (e.g., a gesture), wherein the neural network 114 does not recognize the first movement as a pre-given movement pattern because the probability of the predetermined movement pattern determined by the neural network 114 is less than a predefined first probability threshold. A third sensor signal 126 can also be acquired in response to a user's second movement, wherein the neural network 114 recognizes the second movement as a pre-given movement pattern because the probability of the pre-given movement pattern determined by the neural network 114 is greater than the predefined first probability threshold. For example, the second movement might occur in response to a user's first movement that is not recognized as a pre-given movement pattern because, for example, no control action is performed for the movement pattern assigned to the first movement by the device. It appears that the computer 108 can determine that if the probability determined by the first sensor signal 104 is greater than a second probability threshold, the user's first movement should trigger a desired control action, and the computer 108 can be configured to determine the first sensor signal 104 and the assigned user movement pattern as training data vector 122.

[0090] like Figure 1F As shown, the computer 108 can be configured to transmit a first instruction 124a assigned to the user's motion pattern to the device if the probability of the neural network 114 determining the user's motion pattern is greater than a first probability threshold in response to the input of the third sensor signal 126 into the neural network 114.

[0091] Training data vector 122 could be a false negative training data vector. (See reference.) Figure 1GThe computer 108 can be configured to determine a first false negative training data vector 122a if the probability (e.g., probability value) of a user motion pattern among multiple user motion patterns determined by the neural network 114 for the first sensor signal 104 is less than a predefined first probability threshold ("No" in 118), and if the classification model 116 does not determine the second sensor signal 106 as a user input assigned to the user motion pattern in response to the input of the second sensor signal 106 ("No" in 120). The first false negative training data vector 122a may include the first sensor signal 104 and the assigned user motion pattern. Computer 108 can be configured to determine a second false negative training data vector 122b if the probability (e.g., probability value) of a user motion pattern determined by neural network 114 for a first sensor signal 104 is less than a predefined first probability threshold ("No" in 118) and greater than a predefined second probability threshold ("Yes" in 130), if the probability (e.g., probability value) of a user motion pattern determined by neural network 114 for a third sensor signal 126 is greater than the predefined first probability threshold ("Yes" in 128), and if classification model 116 does not determine the second sensor signal 106 as a user input assigned to the user motion pattern in response to the input of the second sensor signal 106 ("No" in 120). The second false negative training data vector 122b may include the first sensor signal 104 and the assigned user motion pattern.

[0092] refer to Figure 1H Computer 108 can be configured to determine false positive training data vector 132. Computer 108 can be configured to determine false positive training data vector 132 if the probability (e.g., probability value) of a user motion pattern determined by neural network 114 for a first sensor signal 104 is less than a predefined first probability threshold ("No" in 118) and greater than a predefined second probability threshold ("Yes" in 130), if the probability (e.g., probability value) of a user motion pattern determined by neural network 114 for a third sensor signal 126 is less than a predefined first probability threshold ("Yes" in 128), and if classification model 116, in response to input of second sensor signal 106, does not identify second sensor signal 106 as a user input assigned to the user motion pattern ("No" in 120). False positive training data vector 132 may have the first sensor signal 104 and the assigned user motion pattern. Figure 2DA graph 200D is shown with an exemplary first sensor signal 104, for which the probability of determining a user motion pattern is less than a first probability threshold and greater than a second probability threshold, and wherein within a predefined time range 206 following a first time range 202 of the first sensor signal 104, neither a second sensor signal 106, which has been classified as user input assigned to the user motion pattern, nor a third sensor signal 126, which has been classified as the user motion pattern, is acquired.

[0093] It appears that a first sensor signal 104 can be acquired in response to a user's movement (e.g., a gesture), wherein the neural network 114 does not identify the movement as a pre-given user movement pattern, but still determines that the probability of the pre-given user movement pattern is greater than a predefined second probability threshold. For example, one or more sensor elements 102 can determine sensor signals higher than their respective assigned signal thresholds within a predefined time range after the acquisition of the first sensor signal 104 (e.g., within a predefined time range 206 after the start of the first time range 202 assigned to the first sensor signal 104). For example, one or more sensor elements 102 can acquire a second sensor signal 106 and a third sensor signal 126 within a predefined time range. It appears that the user does not want to perform the movement used to perform the control action, and therefore neither performs additional movement nor provides user input, so the computer 108 determines that the movement assigned to the first sensor signal 104 does not represent the user's desired movement pattern (e.g., a false positive signal).

[0094] refer to Figure 1I Sensor 100 may include a second storage device 140. For example, the second storage device 140 may be a first FIFO ("First-In-First-Out") memory. The second storage device 140 may be configured to store false negative training data vectors (e.g., first false negative training data vector 122a, or second false negative training data vector 122b). Sensor 100 may include a third storage device 142. The third storage device 142 may be, for example, a second FIFO memory. The third storage device 142 may be configured to store false positive training data vectors (e.g., false positive training data vector 132). Computer 108 may be configured to provide the first false negative training data vector 122a and / or the second false negative training data vector 122b to the second storage device 140. Computer 108 may be configured to provide the false positive training data vector 132 to the third storage device 142.

[0095] According to various embodiments, computer 108 may be configured to, if a false negative training data vector has been determined, copy the first sensor signal 104 assigned to the false negative training data vector from a first storage device 110 (e.g., a ring memory) to a second storage device 140 (e.g., a first FIFO memory) (e.g., copying to the current list position of the first FIFO memory, and then incrementing the index pointer of the first FIFO memory). According to various embodiments, computer 108 may be configured to, if a false positive training data vector has been determined, copy the first sensor signal 104 assigned to the false positive training data vector from the first storage device 110 to a third storage device 142 (e.g., a second FIFO memory) (e.g., copying to the current list position of the second FIFO memory, and then incrementing the index pointer of the second FIFO memory).

[0096] refer to Figures 1J to 1M Computer 108 can be configured to additionally train a pre-trained neural network 114 using a determined training data vector (e.g., a first false negative training data vector 122a, a second false negative training data vector 122b, a false positive training data vector 132). This additional training of neural network 114 can substantially correspond to the pre-training of neural network 114 described herein, where the training dataset has the determined training data vector. For example, in response to input from the determined training data vector, the pre-trained neural network 114 can determine the probability of each pre-given user motion pattern among a plurality of user motion patterns. Computer 108 can be configured to compute a loss value using a loss function (e.g., maximum likelihood loss function, cross-entropy loss function, mean squared error, etc.) with the user motion pattern assigned to the corresponding training data vector and the probability of that user motion pattern determined by neural network 114. The pre-trained neural network 114 can be additionally trained in a manner that reduces (e.g., minimizes) the loss value.

[0097] For example, when the training data vector is a false negative training data vector (e.g., a first false negative training data vector 122a, or a second false negative training data vector 122b), the greater the deviation of the probability of the assigned user motion pattern from "1", the larger the loss value determined by the loss function. It appears that the first sensor signal 104 of the false negative training data vector represents a motion pattern performed by the user to execute the desired control action, where, as described above, the neural network 114 does not classify the first sensor signal 104 as a user motion pattern. It appears that additional training can lead to the fact that the probability of the user motion pattern assigned to the first sensor signal 104 of the false negative training data vector by the neural network 114 increases, enabling the additionally trained neural network 114 to recognize user-specific behavior as the desired control action.

[0098] For example, when the training data vector is a false positive training data vector (e.g., false positive training data vector 132), the greater the deviation of the probability of the assigned user motion pattern from "0", the larger the loss value determined by the loss function. It appears that the first sensor signal 104 of the false positive training data vector represents a motion pattern in which the user should not perform a control action, where, as described above, the neural network 114 of the first sensor signal 104 has a probability greater than a second probability threshold for determining the user motion pattern. It appears that additional training can lead to the fact that the probability of the user motion pattern assigned to the first sensor signal 104 of the false positive training data vector by the neural network 114 is reduced, such that the additionally trained neural network 114 is able to not classify user-specific behaviors that should not perform control actions as user motion patterns used to perform control actions. This causes the additionally trained neural network 114 to not perform any control actions that the user does not want.

[0099] According to various embodiments, the computer 108 may also be configured to preprocess the corresponding training data vectors and use the preprocessed training data vectors to train the pre-trained neural network 114. Figure 1K The determination of the preprocessed training data vector 144 of the false positive training data vector 132 is illustrated by way of example. Figure 2E An exemplary first sensor signal 104 acquired within a first time range 202 of the false positive training data vector 132 is shown. The computer 108 can be configured to preprocess the first sensor signal 104, reducing the first time range 202 to a third time range 216 smaller than the first time range 202. It appears that portions of the first sensor signal 104 with, for example, low signal strength can be removed. This, for example, has the effect of reducing the memory requirements of the training data vector. Furthermore, this can reduce the computational cost in additional training of the neural network 114.

[0100] According to various embodiments, one or more sensor elements 102 can acquire multiple first sensor signals, multiple second sensor signals, and multiple third sensor signals. Sensor 100 can be configured to use the multiple first sensor signals, multiple second sensor signals, and multiple third sensor signals (e.g., under the conditions described herein) to determine multiple false negative training data vectors and multiple false positive training data vectors. A second storage device 140 can be configured to store each of the multiple false negative training data vectors. A third storage device 142 can be configured to store each of the multiple false positive training data vectors. Reference Figure 1LComputer 108 can be configured to determine whether the memory (e.g., a first FIFO memory) of second storage device 140 is full. For example, computer 108 can be configured to determine whether more than a predefined number (e.g., more than ten, more than fifteen, more than twenty) of training data vectors are stored in second storage device 140. Computer 108 can be configured to determine that second storage device 140 is full if more than a predefined number of training data vectors are stored in second storage device 140 ("yes" in 150). Computer 108 can be configured to, if computer 108 determines that the memory of second storage device 140 is full, additionally train neural network 114 using multiple false negative training data vectors. Computer 108 can be configured to determine whether the memory (e.g., a second FIFO memory) of third storage device 142 is full. For example, computer 108 can be configured to determine whether more than a predefined number (e.g., more than ten, more than fifteen, more than twenty) of training data vectors are stored in third storage device 142. Computer 108 can be configured to determine that third storage device 142 is full if more than a predefined number of training data vectors are stored in third storage device 142 ("Yes" in 152). Computer 108 can also be configured to train neural network 114 154 additionally using multiple false positive training data vectors if computer 108 determines that the memory of third storage device 142 is full.

[0101] According to various embodiments, computer 108 may be configured to determine whether a signal with a signal strength greater than a predefined signal threshold has been determined by means of one or more sensor elements 102. Computer 108 may be configured to, if computer 108 determines that no signal has a signal strength greater than the predefined signal threshold, additionally train a pre-trained neural network 114 using multiple false negative training data vectors and / or multiple false positive training data vectors. For example, computer 108 may determine whether no signal with a signal strength greater than the predefined signal threshold has been determined within a predefined time range. Computer 108 may be configured to, if no signal with a signal strength greater than the predefined signal threshold has been determined within a predefined time range, determine that sensor 100 is in a sleep mode. According to various embodiments, computer 108 may be configured to, if sensor 100 is in a sleep mode, additionally train neural network 114.

[0102] Figure 3A and Figure 3BA system 300 having a sensor 100 according to various embodiments is shown. The system 300 may have an execution device 302. The system 300 may have a communication device 304. The sensor 100 may also have an interface 306. The interface 306 and the communication device 304 of the sensor 100 may be configured to allow data (e.g., instructions) to be transmitted from the sensor 100 to the communication device 304 and from the communication device 304 to the sensor. For example, the interface 306 may be a wired interface (e.g., a serial interface, such as a serial peripheral interface, SPI) or a wireless interface (e.g., a WLAN, such as BLUETOOTH, etc.) for communicating with the communication device 304. The communication device 304 may also be configured to receive data from the execution device 302 and transmit data to the execution device 302. The computer 108 of the sensor 100 may be configured to determine an instruction 120. The computer 108 may be configured to transmit the instruction 120 to the communication device 304 via the interface 306. The communication device 304 may be configured to transmit the instruction 120 to the execution device 302. The actuator 302 can be configured to perform one or more control actions assigned to the instruction 120 received from the sensor 100.

[0103] According to various embodiments, the device may be, for example, a smartphone, a wearable device (e.g., a device worn on the body and / or head, such as smart clothing, smartwatches, smart glasses, etc.), a smart headset, etc.

[0104] The device may have system 300. For example, the device may be a smartwatch, and the smartwatch may have sensor 100. For example, sensor 100 may send instructions to the actuator 302 of the smartwatch to perform control actions of the smartwatch.

[0105] According to various embodiments, the device may include sensor 100, while the other device may not have this device. For example, the device may be a smartphone, while the other device may be a smartwatch. The smartwatch may include sensor 100. Interface 306 may be a wireless interface. For example, sensor 100 may send instructions to the actuator 302 of the smartphone via the wireless interface to perform control actions on the smartphone.

[0106] refer to Figure 3BThe computer 108 of sensor 100 can be configured to receive information via interface 306 regarding whether the device is in sleep mode (e.g., computer 108 can determine that the user is sleeping based on signals from one or more determined sensor elements 102). The computer 108 of sensor 100 can also be configured to receive information via interface 306 regarding whether the device is in battery charging mode. According to various embodiments, computer 108 can be configured to additionally train neural network 114 154 if computer 108 determines that the device is in sleep mode and / or battery charging mode ("yes" in 156).

[0107] Figure 4 A method 400 for generating training data for a machine learning classifier in a sensor according to various embodiments is shown.

[0108] Method 400 may include acquiring a first sensor signal and a second sensor signal by means of a sensor (in 402).

[0109] Method 400 may include determining whether a machine learning classifier classifies the first sensor signal into a pre-given user motion pattern in response to input of the first sensor signal (in 404).

[0110] Method 400 may include determining whether a classification model classifies a second sensor signal as a pre-given user input in response to the input of the second sensor signal (in 406). The user input may be assigned to a user motion pattern.

[0111] Method 400 may include: if the second sensor signal has been classified as a pre-given user input and if the first sensor signal has not been classified as a pre-given user motion pattern, then determining a training data vector for a machine learning classifier (in 408). The training data vector may include the first sensor signal and the user motion pattern.

[0112] Figure 5 A method 500 for generating training data for a machine learning classifier in a sensor according to various embodiments is shown.

[0113] Method 500 may include acquiring sensor signals and additional sensor signals by means of a sensor (in 502).

[0114] Method 500 may include determining whether a machine learning classifier determines a pre-given user motion pattern in response to an input of a sensor signal if the probability is less than a predefined first probability threshold and greater than a predefined second probability threshold (in 504).

[0115] Method 500 may include determining whether a machine learning classifier determines a pre-given user motion pattern with a probability greater than a predefined first probability threshold in response to input of an additional sensor signal (in 506).

[0116] Method 500 may include: determining a training data vector for the machine learning classifier (in 508) if the probability of a pre-given user motion pattern determined in response to inputting sensor signals into the machine learning classifier is less than a predefined first probability threshold and greater than a predefined second probability threshold, and if the probability of the pre-given user motion pattern determined in response to inputting additional sensor signals into the machine learning classifier is greater than the predefined first probability threshold. The training data vector may include sensor signals and user motion patterns.

Claims

1. A method for generating training data for a machine learning classifier in a sensor, comprising: The first sensor signal and the second sensor signal are acquired by means of the sensor; Determine whether the machine learning classifier classifies the first sensor signal into a pre-given user motion pattern in response to the input of the first sensor signal; Determine whether the classification model responds to the input of the second sensor signal by classifying the second sensor signal as a pre-given user input for controlling the device, wherein the user input is assigned to the user's motion pattern; If the second sensor signal has been classified as the pre-given user input, and if the first sensor signal has not been classified as the user motion pattern, then a training data vector is determined for the machine learning classifier, wherein the training data vector contains the first sensor signal and the user motion pattern. The training data vector mentioned above is a false negative training data vector; Determining whether the machine learning classifier classifies the first sensor signal into a pre-given user motion pattern in response to the input of the first sensor signal includes: In response to the input of the first sensor signal into the machine learning classifier, the probability of each of the multiple pre-given user motion patterns is determined; If the probability of a user motion pattern determined for one of the plurality of user motion patterns is greater than a predefined first probability threshold, then the first sensor signal is determined to be the pre-given user motion pattern. The method further includes: If the second sensor signal is not classified as the user input, and if the probability determined for the user motion pattern assigned to the user input among the plurality of user motion patterns is less than a predefined first probability threshold and greater than a predefined second probability threshold, then a false positive training data vector is determined for the machine learning classifier, wherein the false positive training data vector has the first sensor signal and the user motion pattern assigned to the user input.

2. The method according to claim 1, The sensor described herein has multiple sensor elements; as well as The acquisition of the first sensor signal and the second sensor signal by means of the aforementioned sensor includes: One or more first sensor signals are acquired by means of one or more of the plurality of sensor elements; and One or more second sensor signals are acquired by means of one or more of the plurality of sensor elements; Determining whether the machine learning classifier classifies the first sensor signal into a pre-given user motion pattern in response to the input of the first sensor signal includes: Determine whether the machine learning classifier classifies the one or more first sensor signals into the pre-given user motion pattern in response to the input of the one or more first sensor signals; Determining whether the classification model responds to the input of the second sensor signal by classifying the second sensor signal as a pre-given user input for controlling the device includes: Determine whether the classification model classifies the one or more second sensor signals as the pre-given user input in response to the input of the one or more second sensor signals.

3. The method according to any one of claims 1 or 2, further comprising: If the second sensor signal has been classified as the pre-given user input; and / or If the first sensor signal has been classified as the user's motion pattern Then, instructions for controlling the device are transmitted to the device.

4. The method according to any one of claims 1 to 3, further comprising: The third sensor signal is acquired using the aforementioned sensor; Determine whether the machine learning classifier classifies the third sensor signal into the pre-given user motion pattern in response to the input of the third sensor signal; as well as If the machine learning classifier classifies the third sensor signal as the user's motion pattern, it transmits instructions to the device for controlling the device.

5. The method according to any one of claims 1 to 4, Determining whether the machine learning classifier classifies the first sensor signal into a pre-given user motion pattern in response to the input of the first sensor signal includes: In response to inputting the first sensor signal into the machine learning classifier, the probability of each of a plurality of pre-given user motion patterns is determined; If the probability of determining one of the multiple user motion patterns is greater than a predefined first probability threshold, then the first sensor signal is determined to be the pre-given user motion pattern.

6. The method according to any one of claims 1 to 5, The machine learning classifier mentioned therein is a pre-trained machine learning classifier.

7. The method according to claim 6, further comprising: The machine learning classifier is further trained using the training data vector.

8. The method according to claim 7, The acquisition of the first sensor signal and the second sensor signal by means of the aforementioned sensor includes: Acquire the first sensor signal within the first time range, and Acquire the second sensor signal within a second time range. The additional training of the machine learning classifier using the training data vector includes: The first sensor signal assigned to the training data vector is preprocessed such that the first time range of the first sensor signal is reduced to a time range smaller than the first time range. The machine learning classifier is further trained using pre-processed first sensor signals and user motion patterns assigned to the training data vector.

9. The method according to claim 8, further comprising: The third sensor signal is acquired using the aforementioned sensor; In response to the input of the third sensor signal into the machine learning classifier, the probability of each of the multiple pre-given user motion patterns is determined; The false positive training data vector is determined for the machine learning classifier in the following cases: The second sensor signal was not classified as the user input. In response to the first sensor signal being input to the machine learning classifier, the probability determined for the user motion pattern assigned to the user input among the plurality of user motion patterns is less than a predefined first probability threshold and greater than a predefined second probability threshold, and In response to inputting the third sensor signal into the machine learning classifier, the probability determined for the user motion pattern assigned to the user input among the plurality of user motion patterns is less than a predefined first probability threshold.

10. A non-volatile storage medium storing program instructions that, when executed, perform the method according to any one of claims 1 to 9.

11. A sensor configured to perform the method according to any one of claims 1 to 9.

Images