Blink detection method and apparatus based on dynamic threshold, device, and storage medium
By dynamically updating the blink detection threshold based on blink feature parameters from recent electromyography signal sets, the problem of insufficient threshold adaptability in existing technologies is solved, thereby improving the accuracy and stability of blink detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HENGXUAN TECH (BEIJING) CO LTD
- Filing Date
- 2026-04-17
- Publication Date
- 2026-06-09
Smart Images
Figure CN122163235A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of wearable devices, and more specifically, provides a blink detection method, apparatus, device, and storage medium based on dynamic thresholds. Background Technology
[0002] Blink detection has a wide range of applications in wearable devices, human-computer interaction, fatigue monitoring, and other scenarios.
[0003] Existing blink detection methods can monitor blinks based on EMG (Electromyography). The detection method based on EMG signals identifies blinking behavior by collecting bioelectric signals generated by the activity of muscles around the eyes. This method has the advantages of fast response and strong resistance to ambient light interference.
[0004] In blink detection methods based on electromyography (EMG) signals, blink-related feature parameters are typically extracted from the EMG signal and compared with a preset detection threshold to determine whether a blink has occurred. However, the detection threshold is usually a fixed threshold or a threshold calibrated manually once. Since the amplitude and distribution of EMG signals vary significantly among different users, and the characteristics of EMG signals for the same user also change at different times and under different conditions, using a fixed threshold or a one-time calibrated threshold can easily lead to a decrease in the accuracy of blink detection, or even result in false positives or false negatives. Summary of the Invention
[0005] In view of this, this application aims to provide a blink detection method, apparatus, device and storage medium based on dynamic threshold, so as to improve the accuracy of blink detection.
[0006] In a first aspect, embodiments of this application provide a blink detection method based on a dynamic threshold, comprising: S1, acquiring an electromyographic (EMG) signal set; the EMG signal set includes EMG signals of the user's eye region corresponding to the current time window and / or at least one time window prior to the current time window; S2, extracting blink feature parameters of each EMG signal in the EMG signal set; S3, updating a blink detection threshold based on the blink feature parameters; and S4, performing blink detection using the updated blink detection threshold.
[0007] Secondly, embodiments of this application provide a blink detection device based on a dynamic threshold, comprising: an acquisition module for acquiring an electromyographic (EMG) signal set; the EMG signal set including EMG signals of a user's eye region corresponding to the current time window and / or at least one time window prior to the current time window; a feature extraction module for extracting blink feature parameters of each EMG signal in the EMG signal set; an update module for updating a blink detection threshold based on the blink feature parameters; and a detection module for performing blink detection using the updated blink detection threshold.
[0008] Thirdly, embodiments of this application provide a wearable device, including: a data acquisition unit, a memory, and a processor; the memory is connected to the data acquisition unit and the processor; the data acquisition unit is used to acquire electromyographic signals and store them in the memory; the memory stores a program, which, when executed by the processor, causes the processor to perform the blink detection method as described in any of the first aspects.
[0009] Fourthly, embodiments of this application provide a readable storage medium storing a program that, when run on a processor, causes the processor to implement the blink detection method based on dynamic thresholds as described in any of the first aspects.
[0010] In blink detection, a blink detection threshold is used to determine whether a user blinks. An unreasonable blink detection threshold can lead to missed or false blinks. Fixed or one-time calibrated thresholds may become unsuitable for different users or users in different states, affecting the accuracy of blink detection. In this embodiment, the blink detection threshold can be dynamically updated based on the blink feature parameters extracted from the electromyography (EMG) signal. The EMG signal is the recently acquired EMG signal of the user's eye region, i.e., the EMG signal in the current time window and / or before the current time window. This allows the blink detection threshold to be dynamically adjusted according to the user's recent state or according to the usage of different users. Thus, the blink detection threshold can be adaptively adjusted, making it suitable for the current state of the user, improving the adaptability of blink detection to different users, and adapting to the changes in the characteristics of the EMG signal of the user at different times and in different states. This reduces the problems of misjudgment and missed judgment when detecting blinks due to unreasonable blink detection threshold settings, thereby improving the accuracy and stability of blink detection and assisting the device in the reliability of subsequent work based on the blink detection results. Attached Figure Description
[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 A flowchart illustrating a blink detection method based on a dynamic threshold, provided as an embodiment of this application; Figure 2 A schematic diagram of a blink detection device based on a dynamic threshold provided in an embodiment of this application; Figure 3 This is a schematic diagram of a wearable device provided in an embodiment of this application.
[0013] Icon: Blink detection device based on dynamic threshold 200; acquisition module 210; feature extraction module 220; update module 230; detection module 240; wearable device 300; acquisition unit 310; memory 320; processor 330. Detailed Implementation
[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.
[0015] First, this application provides a blink detection method based on dynamic threshold. This method is used to detect blinks and can be applied to various blink detection scenarios such as wearable devices, human-computer interaction, and fatigue monitoring. No limitation is made here.
[0016] In embodiments of this application, the blink detection method based on a dynamic threshold can be executed by the processor of a device. This device can be a wearable device, such as a head-mounted device, smart glasses, or other eye-wearable devices.
[0017] Next, the blink detection method based on dynamic thresholds provided in the embodiments of this application will be described. Please refer to... Figure 1 , Figure 1 A flowchart illustrating a blink detection method based on a dynamic threshold, provided in an embodiment of this application, is included. The blink detection method based on a dynamic threshold includes: S1, acquire electromyographic signal set.
[0018] In this embodiment, electromyography (EMG) signals are collected from the user's eye area. The specific collection method can be found in existing technologies and will not be elaborated here.
[0019] In this embodiment of the application, the electromyographic signal set includes electromyographic signals, and each electromyographic signal is a segment of electromyographic signal of a certain time length. This time length can also be referred to as a time window.
[0020] In this embodiment, the time window is used to limit the time range of the electromyographic signals involved in the statistical calculation of blink feature parameters. By analyzing the electromyographic signals within the time window, the characteristic changes of the electromyographic signals within a certain time scale can be reflected, thereby providing a basis for the dynamic updating of the blink detection threshold.
[0021] In this embodiment of the application, the electromyography (EMG) signal set includes at least one EMG signal, wherein the EMG signal set may include EMG signals of the user's eye region corresponding to the current time window and / or at least one time window prior to the current time window.
[0022] In other words, the electromyographic (EMG) signals collected within and before the current time window represent the user's EMG signals over a past period. These EMG signals may be collected from different user states and can reflect changes in the user's blink-related characteristics. Therefore, in the embodiments of this application, EMG signals can be collected to update the blink detection threshold using the blink feature parameters of the EMG signals, thereby adapting the device to the current state of the current user.
[0023] In some embodiments of this application, the device may store the electromyographic signals extracted during a previous blink detection process and use them as electromyographic signals.
[0024] In one embodiment of this application, acquiring an electromyographic signal set may also include: acquiring electromyographic signals of a preset duration; and extracting each electromyographic signal from the electromyographic signals of the preset duration based on a preset time window.
[0025] In this embodiment, the preset duration electromyographic signal is the electromyographic signal collected by the device from the current moment to a past period of time, and the time length is the preset duration. The preset duration may include the duration corresponding to one or more time windows. The preset duration can be set according to specific scenarios and needs, and is not limited here.
[0026] In the embodiments of this application, there are multiple types of time windows, and any type of time window can be selected as the preset time window.
[0027] For example, in this application embodiment, a sliding time window and a fixed time window are provided.
[0028] The duration of a fixed-duration window is fixed, and two adjacent fixed-duration windows do not overlap in time, and two adjacent time windows may be discontinuous. For example, taking a preset duration of 100ms for electromyography (EMG) signals as an example, the duration of each fixed-duration window is 20ms. Then, the 0-20ms interval is the first EMG signal, the 20ms-40ms interval and the 60ms-80ms interval are the EMG signals corresponding to different time windows between the current time windows, and the 80ms-100ms interval is the EMG signal corresponding to the current time window.
[0029] The sliding time window updates continuously on the time axis according to a preset step size. This can be understood as a fixed duration for each sliding time window, with some overlap between adjacent windows. For example, assuming a preset duration of 100ms for an electromyographic (EMG) signal, each sliding time window lasts 20ms, with a 10ms overlap. The 0-20ms interval represents the first EMG signal; the 10ms-30ms, 20ms-40ms, 30ms-50ms…70ms-90ms intervals represent different EMG signals within the current time window; and the 80ms-100ms interval represents the EMG signal within the current time window. There is a 10ms overlap between these intervals.
[0030] The above is merely an example of the preset time window provided in this application. The preset time window may also be of other types or in other ways, and is not limited here.
[0031] The preset duration electromyographic (EMG) signal is an EMG signal collected by the device. The device continuously collects EMG signals from the user's eye area, which may include EMG signals corresponding to blinking and non-blinking. The correlation between non-blinking signals and blinking is weak, and may even affect the accuracy of blink detection. Therefore, in some embodiments of this application, after acquiring the preset duration EMG signal, the preset duration EMG signal can also be preprocessed.
[0032] The preprocessing step is used to extract the effective electromyographic signals from the electromyographic signals of a preset duration, thereby reducing invalid and interference signals in the electromyographic signals, improving the accuracy of subsequent blink feature extraction and threshold update, and thus improving the accuracy of blink detection.
[0033] Accordingly, both the electromyographic (EMG) signal and the EMG signal to be measured are extracted from the preprocessed EMG signal of a preset duration. That is, extracting the EMG signal set from the preprocessed EMG signal of a preset duration based on a preset time window includes: extracting the EMG signal from the preprocessed EMG signal of a preset duration based on the preset time window.
[0034] There are various preprocessing methods. In some embodiments of this application, the provided preprocessing includes at least one of the following: filtering the electromyography signal of a preset duration based on a first preset frequency band; removing power frequency interference of a second preset frequency from the electromyography signal of a preset duration; rectifying the electromyography signal of a preset duration; smoothing the electromyography signal of a preset duration; and baseline correction of the electromyography signal of a preset duration.
[0035] The electromyographic signals of blinking are usually within a certain target frequency band. Therefore, in order to suppress the influence of non-target frequency band signals on blinking event detection, in the embodiments of this application, a first preset frequency band is set based on the frequency band range, and the electromyographic signals of preset duration are filtered based on the first preset frequency band to retain the effective electromyographic components within the target frequency band.
[0036] For example, the target frequency band of the electromyographic signal during blinking is usually 20Hz to 250Hz. Therefore, the first preset frequency band can be set to 20Hz to 250Hz to characterize the electromyographic activity corresponding to the contraction of the orbicularis oculi muscle during blinking. Low-frequency signals below the first preset frequency band may originate from baseline drift or slow movement, while high-frequency signals above the first preset frequency band may originate from electronic noise or electromagnetic interference. Both of these are non-target frequency band signals.
[0037] The power supply system or surrounding electronic equipment may cause some power frequency interference to the collected electromyographic signals. Power frequency interference usually manifests as a periodic signal with a fixed frequency, such as 50Hz or 60Hz. To reduce the impact of power frequency interference on blink event detection, in this embodiment, power frequency interference of a second preset frequency in the electromyographic signal of a preset duration can be removed to achieve power frequency suppression processing. This can be achieved by methods such as notch filtering, and the specific method is not limited here.
[0038] Since electromyographic (EMG) signals typically exhibit alternating positive and negative alternating AC signals, to facilitate subsequent calculations of amplitude and energy characteristics and accurately extract blink feature parameters, this embodiment of the application may perform rectification processing on the filtered EMG signal. For example, full-wave rectification can be used to convert the EMG signal into a unipolar signal.
[0039] During the acquisition of electromyographic (EMG) signals, some abnormal data points may exist. To reduce the interference of these abnormal data points, smoothing processing can be performed. Specific processing methods include averaging multiple data points or filtering abnormal data points. This can improve the smoothness of the EMG signal, thereby helping to improve the accuracy of the extracted blink feature parameters.
[0040] The baseline of the electromyographic (EMG) signal is the average or reference level of the EMG signal when the user does not exhibit significant active muscle contraction or blinking. In practical applications, blink detection requires determining whether the EMG signal exceeds a preset threshold relative to the baseline; therefore, the stability of the baseline directly affects the detection accuracy. In the embodiments of this application, the baseline parameters currently used for signal processing can also be updated or corrected based on the average or reference level of the EMG signal during a preset duration of EMG signal processing when the user does not exhibit significant active muscle contraction or blinking.
[0041] The above examples are merely illustrations. You may choose to use one or a combination of the above preprocessing methods, or use other types of preprocessing methods. No restrictions are imposed here.
[0042] S2, extract blink feature parameters from each electromyographic signal in the electromyographic signal set.
[0043] In this embodiment of the application, the blink feature parameters are used to characterize the characteristics of the electromyographic signal during blinking, and can also be used to characterize whether blinking occurs. The blink feature parameters extracted from the electromyographic signal can be called blink feature parameters, and the blink feature parameters extracted from the electromyographic signal to be tested can be called target blink feature parameters.
[0044] In this embodiment, the extracted blink feature parameters include, but are not limited to, one or more of the following: peak amplitude of the electromyographic (EMG) signal, peak-to-peak value of the EMG signal, maximum absolute value or average absolute value of the EMG signal, root mean square value of the EMG signal, short-time energy or signal power of the EMG signal, blink feature duration of the EMG signal, rising edge duration of the EMG signal, rising edge slope of the EMG signal, and peak occurrence time of the EMG signal. The peak occurrence time is the time when the peak value in the EMG signal appears within the corresponding time window. Specific extraction methods for blink feature parameters can refer to existing technologies and will not be elaborated here. Furthermore, the blink feature parameters and the target blink feature parameters can also be a weighted sum of any of the aforementioned feature parameters.
[0045] The short-time energy represents the cumulative sum of the squares of the signal amplitude within a time window, and the formula for calculating the short-time energy can be expressed as:
[0046] Here, m represents the short-term energy level, and m is the sequence number of the electromyographic signal. This is an electromyographic signal. For window functions (such as Hamming window, rectangular window). This refers to the frame length or the length of the time window.
[0047] For example, signal power is used to characterize the energy transmitted by a signal per unit time, measuring the average strength of the signal. Signal power can be calculated as follows:
[0048] Where P is the signal power, T is the time length for calculating the signal power, and X(t) is the electromyographic signal.
[0049] Alternatively, signal power can also be calculated as follows:
[0050] Where P represents the signal power, -N to N represent the number of electromyographic (EMG) signal samples used to calculate the signal power, and x(n) represents the EMG signal. The above calculation method is only an example; other methods may also be used. The specific calculation methods for other parameters can also refer to existing technologies and are not limited here.
[0051] In the embodiments of this application, the extracted blink feature parameters and the target blink feature parameters are blink feature parameters of the same type. When extracting the blink feature parameters of the electromyographic signal, the target blink feature parameters corresponding to the electromyographic signal to be tested can also be extracted at the same time.
[0052] S3, update the blink detection threshold based on blink feature parameters.
[0053] In this embodiment of the application, there can be multiple blink detection thresholds. For example, the blink detection threshold can be updated based on at least one of the following blink feature parameters: peak amplitude of electromyography (EMG) signal; peak-to-peak value of EMG signal; root mean square value of EMG signal; short-time energy of EMG signal; characteristic duration of EMG signal.
[0054] The extracted blink feature parameters need to match the features corresponding to the blink detection threshold. For example, if the blink detection threshold is also extracted from the threshold corresponding to short-time energy, the extracted blink feature parameters must include at least the short-time energy of the electromyographic signal.
[0055] With a constant blink detection threshold, blink accuracy can vary depending on the user's state and the current user on the device. For example, the frequency, duration, and force of blinking will differ between a user who is alert and one who is fatigued, resulting in different electromyographic (EMG) signals during blinking. Similarly, the blinking actions of user A and user B will differ, leading to different EMG signals generated during blinking.
[0056] Fixed or one-time calibrated blink detection thresholds are difficult to adapt to the user's actual state in all scenarios, and thus cannot achieve high detection accuracy in all scenarios. Therefore, in the embodiments of this application, the blink detection threshold can be updated to adapt to the user's current state, thereby achieving high blink detection accuracy. Specifically, blink feature parameters of the blink signal are used to update the blink detection threshold to adapt to the user's current state or the actual user on the device.
[0057] In the embodiments of this application, there are various update methods. For example, in some embodiments, the blink detection threshold can be updated directly using the extracted blink feature parameters. That is, the extracted blink feature parameters can be used as the new blink detection threshold.
[0058] In some embodiments of this application, blink feature parameters extracted based on the most recent or specified time of electromyography signal can be directly used as new blink detection thresholds.
[0059] In some embodiments of this application, the blink detection threshold can also be updated using the current blink detection threshold, blink feature parameters, and a preset update method. The preset update method includes, but is not limited to: weighting the blink feature parameters and the blink detection threshold according to preset weights; adjusting the current blink detection threshold by incrementing or decrementing it based on the difference between the blink feature parameters and the current blink detection threshold; and iteratively updating the blink detection threshold step-by-step within multiple time windows. Specific update methods are not limited here.
[0060] In some embodiments of this application, updating the blink detection threshold based on blink feature parameters may further include: calculating statistical feature parameters of each blink feature parameter; determining whether to trigger an update of the blink detection threshold based on the statistical feature parameters; and updating the blink detection threshold based on the blink feature parameters if it is determined that an update of the blink detection threshold is triggered.
[0061] In this embodiment of the application, the statistical characteristic parameter can be a statistical parameter in mathematics, such as at least one of the mean, median, variance, and root mean square value. The statistical characteristic parameter can also be other types of parameters such as standard deviation, quantile, interquartile range, and time-weighted statistics. The specific calculation method of each type of statistical characteristic parameter can refer to the corresponding existing method, and is not limited here.
[0062] For example, the root mean square (RMS) value is the square root of the average of squares, and the formula for its calculation can be expressed as:
[0063] Where RMS is the root mean square value, n is the number of samples for calculating the square root of x, and x i This is an electromyographic signal.
[0064] In the embodiments of this application, statistical feature parameters can also be parameters with specific meanings obtained by statistically analyzing some parameters of the blink feature parameters according to certain rules. For example, in this embodiment, statistical feature parameters may also include: the variation range of the blink feature parameters relative to historical statistical values, the duration of the blink feature parameters, the time interval between adjacent blink events, the baseline fluctuation of the electromyographic signal, noise level, stability index, etc. Such statistical feature parameters need to be obtained by statistically analyzing the blink feature parameters according to certain rules or methods, and can also be regarded as a type of statistical feature parameter.
[0065] In the embodiments of this application, statistical feature parameters are used to determine whether to trigger an update of the blink detection threshold. That is, after obtaining the blink feature parameters, the blink detection threshold is not directly updated using the blink feature parameters, but the statistical feature parameters of the blink feature parameters are used to determine whether to trigger an update.
[0066] In this embodiment, the blink detection threshold may be updated when the statistical feature parameters meet a preset threshold update condition. Therefore, determining whether to trigger an update of the blink detection threshold based on the statistical feature parameters may include: determining that the statistical feature parameters meet the preset threshold update condition.
[0067] In this application embodiment, the preset threshold update conditions can be of various types. For example, in some embodiments of this application, the preset threshold update conditions include, but are not limited to, at least one of the following: the change in blink feature parameters relative to statistical values exceeds a preset range; the statistical feature parameters change; the duration of blink feature parameters or the time interval between adjacent blink events changes; the baseline fluctuation, noise level, or stability index of electromyographic signals changes. When the above preset threshold update conditions are detected, a dynamic update of the blink detection threshold is triggered. The number and type of preset threshold update conditions can be set according to specific circumstances, and an update can be triggered when one or more of the above conditions are met, without limitation.
[0068] In this embodiment, the statistical feature parameters are not directly used to update the blink detection threshold. Instead, they are used to trigger whether the blink detection threshold needs adjustment. The blink feature parameters are the actual parameters involved in threshold adjustment; that is, the blink detection threshold is updated using the blink feature parameters, while the statistical feature parameters are only used to determine whether an update is triggered. Compared to directly updating the blink detection threshold based on the blink feature parameters, this method first determines whether an update to the blink detection threshold is triggered based on the statistical feature parameters, and then adjusts the blink detection threshold based on the blink feature parameters if an update is triggered. By using the statistical feature parameters for change perception and the blink feature parameters for threshold adjustment, frequent changes in the blink detection threshold due to short-term fluctuations in statistics or abnormal data can be avoided, improving the stability and robustness of the threshold update process. Simultaneously, it reduces unnecessary threshold update calculations, lowering device power consumption and resource usage. Furthermore, it makes the updated blink detection threshold more closely reflect the current user's actual state, thereby reducing false positives and false negatives and improving the accuracy and reliability of blink detection.
[0069] In other embodiments of this application, there are various ways to update the blink detection threshold. For example, in addition to directly updating the blink detection threshold using blink feature parameters, in other embodiments, updating the blink detection threshold based on blink feature parameters may also include: performing a weighted calculation based on the blink feature parameters and the current blink detection threshold to obtain the updated blink detection threshold. The specific weights of the blink feature parameters and the current blink detection threshold can be reasonably configured according to specific scenarios and needs. For example, the weights of the statistical feature parameters and the current blink detection threshold can each be 50%, statistical feature parameters: current blink detection threshold = 5%: 95%, statistical feature parameters: current blink detection threshold = 10%: 90%, or more weight ratios; the specific values are not limited here.
[0070] For example, in some other embodiments of this application, updating the blink detection threshold based on blink feature parameters may include: calculating the change between the blink feature parameters and the current blink detection threshold, that is, the absolute value of the difference between the blink feature parameters and the current blink detection threshold; adjusting the current blink detection threshold according to the change, wherein if the blink feature parameters are greater than the current blink detection threshold, the adjustment is incremented, and if they are less than the current blink detection threshold, the adjustment is decremented.
[0071] For example, in some other embodiments of this application, after the blink feature parameters are determined in a certain time window and the statistical feature parameters are triggered to update, the current blink detection threshold can be gradually adjusted to match the blink feature parameters in the next multiple time windows according to a preset step size, so as to avoid detection abnormalities caused by excessive threshold changes.
[0072] Other methods may be used for updating. The above are just some examples provided in this application and should not be construed as limiting this application.
[0073] In some embodiments of this application, the blink detection threshold can be updated at the end of each time window, that is, the execution of S1-S3 is triggered after each time window ends. If blink feature parameters have been extracted from the electromyographic signal beforehand, these parameters can be saved for direct retrieval in subsequent processes without needing to be extracted again.
[0074] Frequent threshold updates can lead to power consumption and resource consumption. However, when users use the device for a long time, their state usually does not change significantly and tends to be stable. Therefore, in some other embodiments of this application, the blink detection threshold can be updated only when a change in the user or the user's state is detected.
[0075] S4 performs blink detection using the updated blink detection threshold.
[0076] In the embodiments of this application, the electromyographic signal to be tested can be compared with the updated blink detection threshold to determine whether it is a blinking event. The electromyographic signal to be tested for blink detection can be the electromyographic signal corresponding to the current time window or the electromyographic signal corresponding to the next time window; the setting method can vary depending on the scenario and is not limited here.
[0077] In the embodiments of this application, it can be determined whether the user blinks by judging whether the target blink feature parameter of the electromyography signal to be tested is continuously greater than the updated blink detection threshold.
[0078] For example, in some embodiments of this application, blink detection based on the updated blink detection threshold may include: acquiring the electromyographic signal to be tested; extracting the target blink feature parameters corresponding to the electromyographic signal to be tested; determining the blink duration based on the target blink feature parameters and the updated blink detection threshold; and determining that a blink has been detected if the blink duration is within a preset blink duration range.
[0079] The blink duration is the duration during which the target blink feature parameter is greater than the updated blink detection threshold.
[0080] The duration of a blink is typically within a certain range. A blink that is too short or too long may indicate a non-blinking action. For example, a blink that is too long might be due to the user closing their eyes to rest or meditate, while a blink that is too short might be caused by eye muscle twitching. Therefore, in the embodiments of this application, when using a blink detection threshold to determine whether a blink has occurred, it is also possible to determine that the blink duration is within a preset blink duration range. This helps to exclude some obviously non-blinking events, which to some extent helps to improve the accuracy of blink detection and reduce power consumption and resource usage caused by subsequent judgment processes.
[0081] In addition to whether the user blinks, the specific type of blinking event can also be determined in the embodiments of this application.
[0082] Therefore, in one embodiment, after performing blink detection on the electromyographic signal to be tested using the updated blink detection threshold, the method may further include: acquiring the electromyographic signal to be tested; determining the corresponding blink event based on the target blink feature parameters of the electromyographic signal to be tested; and determining the blink event type based on the blink event and the time corresponding to the blink event.
[0083] A blink event is used to characterize the occurrence of a blink. For example, based on the aforementioned embodiments, if the target blink feature parameter has a blink duration, then a blink event is characterized.
[0084] Blinking events can be categorized into single blinks, double blinks, or multiple blinks. The temporal relationship between each blink event can be determined by analyzing the time corresponding to each blink event, thus identifying the blink event type. For example, in the embodiments of this application, the determination of a double blink event or a multiple blink event also requires meeting at least one of the following conditions: the time interval between adjacent blink events is within a preset range; and the number of blink events detected within a preset time window meets a preset requirement.
[0085] In some embodiments of this application, the blink event type may also include unilateral blink events and bilateral blink events. In this embodiment, the blink location may be determined based on the characteristics of the second sub-electromyography signal of the electromyography signal to be measured; and the blink event type corresponding to the electromyography signal to be measured may be determined based on the blink location.
[0086] In the embodiments of this application, the device can collect electromyographic (EMG) signals from different eye regions of the user, namely the EMG signals from the left eye and the right eye. These EMG signals can be referred to as second sub-EMG signals. Based on this, the second sub-EMG signals can be compared with a blink detection threshold to determine whether the left and right eyes have blinked.
[0087] Monocular blinking events and binocular blinking events can also be distinguished based on differences in electromyographic signal characteristics collected from different eye regions or differences in amplitude, energy, or temporal characteristics of corresponding electromyographic signals from the left and right eyes. For example, based on various parameters representing blinking moments, such as duration, peak time, and characteristic time intervals representing blinking, it can be determined whether the left and right eyes blink simultaneously.
[0088] Once information including the number of blinks and the location of blinks is determined, the corresponding blink event type can be determined based on the combination of the two. For example, blink event types include, but are not limited to, single blink events, double blink events, multiple blink events, unilateral blink events, and bilateral blink events.
[0089] In some devices, blinking events generate control commands during human-computer interaction, which in turn control the device to perform certain operations. In such scenarios, it is necessary to distinguish between the user's voluntary blinking and natural blinking.
[0090] Therefore, in some embodiments of this application, after performing blink detection on the electromyographic signal to be tested using the updated blink detection threshold, the method may further include: acquiring target blink feature parameters and / or time structure of the electromyographic signal to be tested; and determining whether the electromyographic signal to be tested includes an active blinking event based on the target blink feature parameters and / or time structure.
[0091] Active blinking refers to a user's conscious blinking. Compared to natural blinking, conscious blinking usually exhibits statistical differences in terms of the amplitude, energy, duration, or occurrence pattern of electromyographic (EMG) signals. For example, the amplitude and energy of the EMG signal from voluntary blinking are greater, and the duration is longer. Therefore, in the embodiments of this application, blinking feature parameters and / or time structure of the EMG signal corresponding to natural blinking can be collected, and a threshold can be set based on the blinking feature parameters and / or time structure of the EMG signal corresponding to natural blinking. It can then be determined whether the target blinking feature parameters and / or time structure of the EMG signal to be tested are greater than the threshold, thereby determining whether it is an active blinking.
[0092] In the above embodiments, this application provides an adaptive mechanism for blink detection threshold, which enables the blink detection threshold to better adapt to the current state of the user, improves the adaptability of blink detection to different users, adapts to the changes in electromyographic signal characteristics of users at different times and in different states, thereby reducing the problems of misjudgment and missed judgment when performing blink detection on users due to unreasonable blink detection threshold settings, and thus improving the accuracy and stability of blink detection on users, so as to assist the device in the reliability of subsequent work based on blink detection results.
[0093] Based on the same inventive concept, this application also provides a blink detection device based on a dynamic threshold. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 A blink detection device based on dynamic threshold is provided in one embodiment of this application. The blink detection device 200 based on dynamic threshold includes: an acquisition module 210, a feature extraction module 220, an update module 230 and a detection module 240.
[0094] The acquisition module 210 is used to acquire an electromyographic signal set; the electromyographic signal set includes electromyographic signals of the user's eye region corresponding to the current time window and / or at least one time window before the current time window.
[0095] The feature extraction module 220 is used to extract blink feature parameters of each electromyographic signal in the electromyographic signal set.
[0096] The update module 230 is used to update the blink detection threshold based on the blink feature parameters.
[0097] The detection module 240 is used to perform blink detection using the updated blink detection threshold.
[0098] In one embodiment, the update module 230 is used to update the blink detection threshold based on at least one of the following blink feature parameters: the peak amplitude of the electromyographic signal; the peak-to-peak value of the electromyographic signal; the root mean square value of the electromyographic signal; the short-time energy of the electromyographic signal; and the characteristic duration of the electromyographic signal.
[0099] In one embodiment, the update module 230 is configured to: calculate statistical feature parameters of each of the blink feature parameters; the statistical feature parameters include at least one of mean, median, variance, and root mean square value; determine whether to trigger an update of the blink detection threshold based on the statistical feature parameters; and update the blink detection threshold based on the blink feature parameters if it is determined that an update of the blink detection threshold is triggered.
[0100] In one embodiment, the update module 230 is used to determine, before updating the blink detection threshold based on the statistical feature parameters, that the blink feature parameters and / or the statistical feature parameters meet a preset threshold update condition.
[0101] In one embodiment, the update module 230 is used to perform a weighted calculation based on the blink feature parameters and the current blink detection threshold to obtain the updated blink detection threshold.
[0102] In one embodiment, after each time window ends, the acquisition module 210, feature extraction module 220, and update module 230 trigger the execution of their tasks.
[0103] In one embodiment, the acquisition module 210 is used to acquire electromyography (EMG) signals of a preset duration; extract the EMG signal set from the EMG signals of the preset duration based on a preset time window; the preset time window is a sliding time window or a fixed duration window; the duration of the fixed duration window is fixed and two adjacent fixed duration windows do not overlap in time; the duration of the sliding time window is fixed and two adjacent sliding time windows partially overlap in time.
[0104] In one embodiment, the acquisition module 210 is further configured to preprocess the preset duration electromyographic (EMG) signal, wherein the preprocessing is used to extract valid EMG signals from the preset duration EMG signal. The extraction module is configured to: extract each EMG signal from the preprocessed preset duration EMG signal based on the preset time window.
[0105] In one embodiment, the acquisition module 210 is further configured to perform at least one of the following preprocessing steps: filtering the preset duration electromyographic signal based on a first preset frequency band; removing power frequency interference of a second preset frequency from the preset duration electromyographic signal; rectifying the preset duration electromyographic signal; smoothing the preset duration electromyographic signal; and performing baseline correction on the preset duration electromyographic signal, wherein the baseline is the average level or reference level of the electromyographic signal when the user does not have obvious muscle active contraction or blinking.
[0106] In one embodiment, the extraction module is further configured to: acquire the electromyography signal to be tested; extract the target blink feature parameters corresponding to the electromyography signal to be tested; determine the blink duration based on the target blink feature parameters and the updated blink detection threshold, wherein the blink duration is the duration during which the target blink feature parameters are greater than the updated blink detection threshold; and determine that a blink has been detected if the blink duration is determined to be within a preset blink duration range.
[0107] In one embodiment, the detection module 240 is further configured to: acquire a target electromyographic signal after blink detection using the updated blink detection threshold; determine a corresponding blink event based on the target blink feature parameters of the target electromyographic signal; and determine the blink event type based on the blink event and the time corresponding to the blink event.
[0108] In one embodiment, the detection module 240 is further configured to: after performing blink detection on the electromyographic signal to be tested through the updated blink detection threshold, acquire the electromyographic signal to be tested; acquire the target blink feature parameters and / or time structure of the electromyographic signal to be tested; and determine whether the electromyographic signal to be tested includes an active blinking event based on the target blink feature parameters and / or time structure.
[0109] Based on the same inventive concept, this application also provides a wearable device 300. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of the structure of a wearable device 300 provided in an embodiment of this application. The wearable device 300 includes: a data acquisition unit 310, a memory 320, and a processor 330. The memory 320 is connected to the data acquisition unit 310 and is also connected to the processor 330.
[0110] In this embodiment, the acquisition unit 310 is used to acquire electromyographic signals and store them in the memory 320. The memory 320 stores a program, which, when executed by the processor 330, causes the processor 330 to perform the blink detection method based on dynamic threshold as provided in any of the foregoing embodiments.
[0111] In the embodiments of this application, the wearable device 300 includes, but is not limited to, a head-mounted device, smart glasses, or other eye-wearable devices.
[0112] In some embodiments of this application, in the wearable device 300, the acquisition unit 310, the memory 320, and the processor 330 can be separate devices. For example, the acquisition unit 310 is disposed on device A, and the memory 320 and the processor 330 are disposed on device B. The acquisition unit 310 includes an electromyography (EMG) signal acquisition circuit or sensor. Device B receives the EMG signals acquired by the acquisition unit 310 of device A and executes the blink detection method based on dynamic threshold provided in the foregoing embodiments.
[0113] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium having a program stored thereon, which, when run on a processor, causes the processor to execute the blink detection method based on dynamic thresholds provided in the above embodiments.
[0114] The readable storage medium can be any available medium that the processor can access, or a data storage device such as a server or data center that integrates one or more available media. The readable storage medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs (digital video discs)), or semiconductor media (e.g., SSDs (solid state disks)).
[0115] If the blink detection method based on dynamic thresholds is implemented as a software functional module and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause the communication module to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disks, or optical disks.
[0116] Based on the same inventive concept, this application also provides a computer program product, which includes a computer program that, when executed by a communication module, implements the aforementioned blink detection method based on a dynamic threshold. The computer program product can be a software installation package, a program script, etc.
[0117] In the embodiments provided in this application, it should be understood that the disclosed methods and devices can also be implemented in other ways. The device embodiments described above are merely illustrative. The functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0118] The above embodiments can be freely combined without conflict, and the resulting embodiments are covered within the protection scope of this application.
[0119] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A blink detection method based on dynamic threshold, characterized in that, include: S1, acquire an electromyographic signal set; the electromyographic signal set includes electromyographic signals of the user's eye region corresponding to the current time window and / or at least one time window before the current time window; S2, extract the blink feature parameters of each electromyographic signal in the electromyographic signal set; S3, Update the blink detection threshold based on the blink feature parameters; S4, blink detection is performed using the updated blink detection threshold.
2. The dynamic threshold based blink detection method of claim 1, wherein, The step of updating the blink detection threshold based on the blink feature parameters includes: The blink detection threshold is updated based on at least one of the following blink feature parameters: The peak amplitude of the electromyographic signal; The peak-to-peak value of the electromyographic signal; The root mean square value of the electromyographic signal; The short-time energy of the electromyographic signal; The characteristic duration of the electromyographic signal.
3. The dynamic threshold based blink detection method of claim 1, wherein, The step of updating the blink detection threshold based on the blink feature parameters includes: Calculate the statistical characteristic parameters of each of the blinking characteristic parameters; the statistical characteristic parameters include at least one of the mean, median, variance, and root mean square value; Based on the statistical feature parameters, determine whether to trigger an update to the blink detection threshold; If it is determined that an update to the blink detection threshold is triggered, the blink detection threshold is updated based on the blink feature parameters.
4. The dynamic threshold based blink detection method of claim 3, wherein, The step of determining whether to trigger an update to the blink detection threshold based on the statistical feature parameters includes: The statistical feature parameters are determined to meet the preset threshold update conditions.
5. The dynamic threshold based blink detection method of claim 1, wherein, The step of updating the blink detection threshold based on the blink feature parameters includes: The updated blink detection threshold is obtained by weighting the blink feature parameters with the current blink detection threshold.
6. The blink detection method based on dynamic threshold according to claim 1, characterized in that, The method further includes: After each of the aforementioned time windows ends, the execution of S1-S3 is triggered.
7. The blink detection method based on dynamic threshold according to claim 1, characterized in that, The acquisition of the electromyographic signal set includes: Acquire electromyographic signals for a preset duration; Based on a preset time window, each electromyographic signal is extracted from the electromyographic signal of the preset duration. The preset time window is either a sliding time window or a fixed-duration window; the fixed-duration window has a fixed duration and two adjacent fixed-duration windows do not overlap in time; the sliding time window has a fixed duration and two adjacent sliding time windows partially overlap in time.
8. The blink detection method based on dynamic threshold according to claim 7, characterized in that, After acquiring the electromyographic signal of the preset duration, the method further includes: The preset duration electromyographic signal is preprocessed to extract the effective electromyographic signal from the preset duration electromyographic signal; The step of extracting each electromyographic signal from the preset duration electromyographic signal based on a preset time window includes: extracting each electromyographic signal from the preprocessed preset duration electromyographic signal based on the preset time window.
9. The blink detection method based on dynamic threshold according to claim 8, characterized in that, The preprocessing includes at least one of the following: The electromyographic signal of the preset duration is filtered based on the first preset frequency band; Remove power frequency interference of the second preset frequency from the electromyographic signal of the preset duration; The electromyographic signal of the preset duration is rectified; The preset duration electromyographic signal is smoothed. Baseline correction is performed on the preset duration electromyographic signal, where the baseline is the average or reference level of the electromyographic signal when the user does not have obvious active muscle contraction or blinking.
10. The blink detection method based on dynamic threshold according to claim 1, characterized in that, The step of performing blink detection using the updated blink detection threshold includes: Acquire the electromyographic signal to be tested; Extract the target blink feature parameters corresponding to the electromyographic signal to be tested; The blink duration is determined based on the target blink feature parameters and the updated blink detection threshold, wherein the blink duration is the duration during which the target blink feature parameters are greater than the updated blink detection threshold; If the duration of the blink is determined to be within a preset blink duration range, a blink is determined to have been detected.
11. The blink detection method based on dynamic threshold according to any one of claims 1-10, characterized in that, After performing blink detection using the updated blink detection threshold, the method further includes: Acquire the electromyographic signal to be tested; The corresponding blinking event is determined based on the target blinking feature parameters of the electromyographic signal to be measured. The type of blink event is determined based on the blink event and the time corresponding to the blink event.
12. The blink detection method based on dynamic threshold according to any one of claims 1-10, characterized in that, After performing blink detection on the electromyographic signal to be tested using the updated blink detection threshold, the method further includes: Acquire the electromyographic signal to be tested; Acquire the target blink feature parameters and / or temporal structure of the electromyographic signal to be tested; Based on the target blink feature parameters and / or time structure, it is determined whether the electromyographic signal to be tested includes an active blink event.
13. A blink detection device based on a dynamic threshold, characterized in that, include: The acquisition module is used to acquire an electromyographic signal set; the electromyographic signal set includes electromyographic signals of the user's eye region corresponding to the current time window and / or at least one time window before the current time window; The feature extraction module is used to extract blink feature parameters of each electromyographic signal in the electromyographic signal set; The update module is used to update the blink detection threshold based on the blink feature parameters; A detection module is used to detect blinks using the updated blink detection threshold.
14. A wearable device, characterized in that, include: Acquisition unit, memory, processor; The memory is connected to the acquisition unit, and the memory is also connected to the processor; The acquisition unit is used to acquire electromyographic signals and store them in the memory; The memory stores a program that, when executed by the processor, causes the processor to perform the blink detection method as described in any one of claims 1-12.
15. A readable storage medium, characterized in that, The readable storage medium stores a program that, when run on a processor, causes the processor to implement the blink detection method based on a dynamic threshold as described in any one of claims 1-12.