Eyelid state recognition method, device and equipment for brain wave scanning user and storage medium

Abstract

The invention relates to the technical field of brain wave scanning monitoring, in particular to a brain wave scanning user eyelid state recognition method and device, equipment and a storage medium. The method comprises the following steps: responding to brain wave scanning task triggering, and continuously obtaining a user face image; performing feature collection on the user face image to obtain an eyelid feature coordinate pair and a reference feature coordinate pair; determining a reference feature spacing value based on the reference feature coordinate pair, and determining an upper and lower eyelid spacing value of the current detection single eye based on the eyelid feature coordinate pair; combining the upper and lower eyelid spacing value with the reference feature spacing value to obtain a normalized spacing value corresponding to the current video frame; judging whether the normalized interval value is in a preset eye closing judgment interval in the continuous video frames or not, and if yes, determining that the current detected single eye is in an eyelid closing state; and when and only when both the left eye and the right eye of the user are in the eyelid closed state, judging that the user is in the eyelid closed state. The method can support self-service brain function detection in a non-professional environment.

Description

Technical Field

[0001] This application relates to the technical field of brainwave scanning monitoring, and in particular to a method, device, equipment, and storage medium for recognizing the eyelid state of a brainwave scanning user. Background Technology

[0002] Electroencephalography (EEG) is an important tool for recording and analyzing brain electrical activity. It captures weak electrical signals generated by neuronal activity by placing electrodes on the scalp and is widely used in neuroscience research, medical diagnosis, and psychological assessment. Its basic principle is based on the propagation characteristics of neuronal electrical activity. Through electrode acquisition, signal amplification, and filtering preprocessing, it generates an electrical activity map reflecting the functional state of the brain, providing fundamental data support for brain function analysis.

[0003] Currently, existing EEG technology requires specific tasks to induce characteristic electrical activity in the brain during the testing process. The alternation test between open and closed eye states is one of the core components. By comparing the EEG characteristics in different states, such as the enhanced alpha waves in the occipital lobe when the eyes are closed, clinical and research goals such as sleep quality assessment, peak frequency calculation, and emotional state detection can be achieved.

[0004] However, the device cannot confirm in real time whether the user keeps their eyes open or closed as required, which may lead to a discrepancy between the data collection and the actual behavior. This also makes it difficult to distinguish the effective data periods, and may ultimately lead to the data sufficiency judgment relying on human experience. This not only increases the workload of professionals, but also limits the autonomous application of EEG technology in non-professional scenarios such as families and psychological counseling institutions. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this application provides a method, device, equipment, and storage medium for recognizing the eyelid state of users during brainwave scanning. Through intelligent real-time monitoring, it actively senses and confirms the actual eyelid state of users, thereby providing reliable behavioral evidence for the accurate labeling, quality assessment, and subsequent analysis of EEG data, and effectively supporting self-service brain function testing in non-professional environments.

[0006] The technical solution adopted by this application to solve its technical problem is: In a first aspect, this application provides a method for recognizing the eyelid state of a user undergoing brainwave scanning, the method comprising: In response to the brainwave scanning task, the system continuously acquires the user's facial video stream and obtains the user's facial images corresponding to consecutive video frames. For each user's facial image, feature acquisition is performed to obtain the eyelid feature coordinate pair and the reference feature coordinate pair corresponding to the current video frame; The reference feature spacing value is determined based on the reference feature coordinate pair, and the upper and lower eyelid spacing value of the currently detected monocular eye is determined based on the eyelid feature coordinate pair; the currently detected monocular eye is the user's left eye or the user's right eye; The normalized distance value is obtained by combining the distance value between the upper and lower eyelids with the distance value of the reference feature. Determine whether the normalized spacing value is within the preset eye-closed determination range in the consecutive video frames; if so, determine that the currently detected single eye is in the eyelid closed state. A user is determined to be in a closed eyelid state if and only if both the user's left eye and the user's right eye are in the closed eyelid state.

[0007] Optionally, the step of performing feature acquisition on each of the user's facial images to obtain the eyelid feature coordinate pair and the reference feature coordinate pair corresponding to the current video frame includes: The user's facial image is input frame by frame into a preset feature extraction model. The feature extraction model is used to extract features from the current video frame to obtain the upper eyelid feature coordinates, the lower eyelid feature coordinates, and multiple reference feature coordinates. The upper eyelid feature coordinates and the lower eyelid feature coordinates of the same monocular eye are paired to form monocular feature coordinate pairs for the user's left eye and the user's right eye, respectively. The two pairs of monocular feature coordinate pairs are then integrated into the eyelid feature coordinate pairs. At least one pair of reference feature coordinates with a preset correspondence is determined from all the reference feature coordinates to form at least one pair of reference feature coordinates.

[0008] Optionally, the step of determining the reference feature spacing value based on the reference feature coordinate pair and determining the upper and lower eyelid spacing value of the currently detected monocular eye based on the eyelid feature coordinate pair includes: The coordinate spacing value is calculated based on the monocular feature coordinate pair corresponding to the currently detected monocular eye to obtain the upper and lower eyelid distance value; The coordinate spacing value is calculated based on all the reference feature coordinate pairs to obtain at least one reference feature spacing value.

[0009] Optionally, the step of normalizing the distance between the upper and lower eyelids and the reference feature distance value to obtain the normalized distance value corresponding to the current video frame includes: One reference feature spacing value is determined from all the aforementioned reference feature spacing values ​​as the calibration spacing value; Divide the distance between the upper and lower eyelids by the calibration distance value to obtain the normalized distance value corresponding to the current video frame.

[0010] Optionally, the step of determining whether the normalized spacing value is within a preset eye-closed determination interval in the consecutive video frames, and if so, determining that the currently detected monocular eye is in an eyelid-closed state, includes: Determine whether the normalized spacing value corresponding to the current video frame is within the closed-eye determination interval. If it is within the closed-eye determination interval, increment the closed-eye frame count of the current detected single eye; otherwise, set the closed-eye frame count to zero. In response to the count of closed-eye frames reaching a preset count threshold, the currently detected single eye is determined to be in an eyelid-closed state.

[0011] Optionally, after determining whether the normalized interval value is within a preset closed-eye determination interval in the consecutive video frames, the method further includes: If the normalized spacing value of the current video frame is not within the closed eye determination range, then the currently detected single eye is determined to be in an open eyelid state. If either the user's left eye or the user's right eye is in the open eyelid state, then the user is determined to be in the open eyelid state.

[0012] Optionally, after the step of determining that the user is in an eyelid-closed state if and only if both the user's left eye and the user's right eye are in the eyelid-closed state, the method further includes: In response to the user being in the closed eyelid state or the open eyelid state, update the corresponding eyelid state association parameters; Based on the updated eyelid state association parameters, a matching judgment is made with the current expected state of the brainwave scanning task. If the eyelid state association parameters do not match the current expected state, an abnormal state prompt message is output.

[0013] Secondly, this application provides an eyelid state recognition device for brainwave scanning users, comprising: The image acquisition module is used to continuously acquire the user's facial video stream in response to the triggering of the brainwave scanning task, and obtain the user's facial image corresponding to the continuous video frames. The feature acquisition module is used to acquire features from each of the user's facial images to obtain the eyelid feature coordinate pairs and reference feature coordinate pairs corresponding to the current video frame; The spacing calculation module is used to determine the reference feature spacing value based on the reference feature coordinate pair, and to determine the upper and lower eyelid spacing value of the currently detected monocular eye based on the eyelid feature coordinate pair; the currently detected monocular eye is the user's left eye or the user's right eye; The dimension processing module is used to combine the distance value between the upper and lower eyelids with the distance value of the reference feature to perform normalization processing, so as to obtain the normalized distance value corresponding to the current video frame; The monocular state determination module is used to determine whether the normalized spacing value is within the preset closed eye determination interval in the continuous video frames. If so, it is determined that the currently detected monocular eye is in the eyelid closed state. The user state determination module is used to determine that the user is in the eyelid closed state if and only if both the user's left eye and the user's right eye are in the eyelid closed state.

[0014] Thirdly, this application provides an electronic device, comprising: One or more processors; One or more memory units; And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the one or more processors, cause the electronic device to perform the methods described above.

[0015] Fourthly, this application provides a computer-readable storage medium storing a program or instructions that, when executed, implement the above-described method.

[0016] The working principle of this application is as follows: When the brainwave scanning task is triggered, the system continuously acquires the user's facial video stream and extracts continuous frame images. By acquiring features from each frame image, it simultaneously obtains eyelid feature coordinate pairs containing key points of the upper and lower eyelids, as well as reference feature coordinate pairs containing stable facial structural feature points. Based on these coordinates, the system calculates the reference feature distance value and the distance between the upper and lower eyelids of a single eye, and then normalizes them to convert them into a dimensionless proportional relationship, eliminating scale interference caused by changes in the distance between the user and the camera. Subsequently, by determining whether the normalized distance value is continuously within the preset closed-eye interval in the continuous video frames, it determines whether a single eye is in a stable closed state. Finally, the overall eyelid closure state of the user is determined only when both eyes meet the closure condition.

[0017] Based on the above working principle, the beneficial effects of this application are as follows: First, the user's state is dynamically monitored by continuously acquiring real-time video streams and processing features frame by frame. Then, normalization processing is used to eliminate interference from changes in head posture such as distance and rotation by introducing stable reference features. Combined with continuous frame interval judgment to filter interference factors such as instantaneous blinking, the accuracy of state recognition is ensured and invalid data due to user action deviation is avoided. Finally, the reliability of the results is further improved based on the binocular joint judgment mechanism, so that subsequent analysis can clearly distinguish the effective data period, thus forming an intelligent monitoring technology path for brain scanning.

[0018] Through the above technical approach, this method achieves automated and high-precision recognition of the user's eyelid state, which not only reduces the reliance on manual monitoring by professionals, but also ensures accurate synchronization between brainwave scan data and user behavior state, providing a reliable behavioral basis for subsequent brain function analysis and effectively improving the applicability and data reliability of brainwave detection technology in non-professional scenarios. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of the brainwave scanning user eyelid state recognition method provided in the embodiments of this application; Figure 2 This is a virtual structural diagram of the brainwave scanning user eyelid state recognition device provided in this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0021] The following will clearly and completely describe the concept, specific structure, and resulting technical effects of this application in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of this application. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the scope of protection of this application. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this application can be combined interactively without contradicting each other.

[0022] In the field of EEG technology, static brainwave scanning is an important diagnostic and assessment tool. This process typically requires the user to alternate between opening and closing their eyes, and the brain's functional state is assessed by comparing specific patterns of brainwaves (such as changes in alpha wave power and peak frequency) between the two states. The accuracy of this state-comparison analysis fundamentally depends on whether the user strictly follows the instructions to open or close their eyes within the specified time period.

[0023] Currently, this process primarily relies on EEG equipment operators issuing commands to users via voice or on-screen graphics. However, existing technologies have a significant monitoring blind spot: the system itself lacks the ability to perceive and verify the user's actual eyelid state in real time. It can only passively assume that the user complies with the commands, and cannot objectively record whether the user actually closed their eyelids during the "closed" period or kept their eyes open during the "open" period. This deficiency directly leads to two key problems: First, the collected EEG data may not match the preset behavioral state, which makes subsequent analysis based on state comparison lose its accuracy basis and may even lead to misjudgment.

[0024] Secondly, for specific user groups with easily distracted attention or low compliance with instructions (such as patients with ADHD or anxiety disorders), simple instruction prompts have limited effectiveness, and their frequent behavioral deviations cannot be detected and recorded by the system. This necessitates that the entire scanning process be supervised by professionals, severely restricting the popularization and application of EEG technology in non-professional settings such as homes, schools, and general community institutions.

[0025] Based on this, refer to Figure 1 , Figure 1 This is a flowchart illustrating the eyelid state recognition method for brainwave scanning users provided in this application embodiment. It shows several key steps involved in the intelligent real-time monitoring method provided to address the aforementioned technical deficiencies. Each key step is described in detail below: In step S1, in response to the triggering of the brainwave scanning task, the user's facial video stream is continuously acquired to obtain the user's facial images corresponding to consecutive video frames.

[0026] Among them, brainwave scanning refers to the process of recording and analyzing the electrical activity of the brain using EEG (electroencephalography) technology. It usually involves a series of operations that require the user to complete data collection in a specific state (such as with eyes open or closed).

[0027] Among them, facial video stream refers to a dynamic image sequence formed by continuously capturing a user's face through an image acquisition device (such as a camera), which can reflect the dynamic changes of the user's face in real time; continuous video frames are a series of static images obtained after the facial video stream is decomposed, with each frame corresponding to a specific point in time in the video stream; and user facial image is the static image of the user's face presented in a single video frame.

[0028] Specifically, when the brainwave scan task is initiated, the system immediately responds to the trigger signal, activating the image acquisition device to continuously capture the user's facial dynamics. During this process, the device continuously captures images of the user's face at regular time intervals, forming a continuous facial video stream. The system then processes the video stream, breaking it down into a series of orderly arranged facial images. These continuous images are arranged chronologically, completely recording the changes in the user's face throughout the entire scan task, providing a raw visual data foundation for subsequent analysis of the user's eye state (such as open or closed eyes).

[0029] More specifically, by continuously acquiring facial video streams and breaking them down into consecutive frame images, the system can ensure that it captures the user's facial state at every moment during the entire brainwave scan. This continuous data recording method provides a basis for subsequent accurate identification of whether the user complies with the open / close eye commands, enabling analysts to accurately know whether the data for each time period is valid, thereby improving the accuracy and reliability of EEG data analysis.

[0030] In step S2, feature acquisition is performed on each user's facial image to obtain the eyelid feature coordinate pair and the reference feature coordinate pair corresponding to the current video frame.

[0031] Among them, a video frame is a single frame in a video stream; an eyelid feature coordinate pair is a combination of pixel coordinates describing key locations on the eyelid edge, including the horizontal and vertical coordinates of feature points of the upper and lower eyelids (such as the midpoint of the upper eyelid and the midpoint of the lower eyelid) in the image; a reference feature coordinate pair is a combination of pixel coordinates describing key locations of other stable physiological structures on the face (such as the ears, the center of the eyebrows, the chin, etc.), used to assist in quantifying the state of the eyes.

[0032] Specifically, during the brainwave scan, the system continuously acquires the user's facial video stream and processes each frame of the user's facial image in real time. First, it detects and extracts features from the facial region in the image, outputting the pixel coordinates of key feature points on the eyelid edges (such as the midpoints of the upper and lower eyelids), forming eyelid feature coordinate pairs. Simultaneously, it identifies other stable reference features on the face (such as left and right ear features, the tip of the nose, and the chin), and outputs their pixel coordinates, forming reference feature coordinate pairs. These coordinate pairs, in pixels, directly reflect the spatial position of the feature points in the current video frame, providing fundamental data for subsequent quantification of eye opening and closing states.

[0033] More specifically, in this embodiment of the application, the feature acquisition step is performed through a preset feature extraction model. Based on this concept, the step of acquiring features from each user's facial image to obtain the eyelid feature coordinate pair and the reference feature coordinate pair corresponding to the current video frame includes: The user's facial image is input frame by frame into a preset feature extraction model. The feature extraction model is used to extract features from the current video frame to obtain the upper eyelid feature coordinates, lower eyelid feature coordinates, and multiple reference feature coordinates.

[0034] Among them, the feature extraction model is a pre-trained artificial intelligence model used to locate and output the pixel coordinates of key feature points from facial images. It is usually built on a deep learning architecture, such as CNN (Convolutional Neural Network), and has the ability to detect high-precision feature points in real time.

[0035] Among them, the upper eyelid feature coordinates are the pixel coordinates in the image of a specific position of the upper eyelid edge (such as the midpoint of the upper eyelid) output by the feature extraction model; the lower eyelid feature coordinates are the pixel coordinates in the image of a specific position of the lower eyelid edge (such as the midpoint of the lower eyelid) output by the feature extraction model; and the reference feature coordinates are the coordinates of stable facial structural feature points output by the feature extraction model for subsequent normalization processing, such as the pixel coordinates of the ear, nose tip, chin, etc., whose geometric relationship is relatively stable when the facial pose changes.

[0036] Specifically, the system inputs the user's facial images from the video stream frame by frame into a preset feature extraction model. The model performs pixel-level analysis on each frame of the image through a deep neural network, accurately locates and outputs the upper eyelid feature coordinates, lower eyelid feature coordinates (divided into left and right eyes), and multiple reference feature coordinates (such as ears, nose, etc.).

[0037] Subsequently, the upper eyelid feature coordinates and the lower eyelid feature coordinates of the same monocular eye are paired to form monocular feature coordinate pairs for the user's left eye and right eye, respectively. The two pairs of monocular feature coordinate pairs are then integrated into the eyelid feature coordinate pairs.

[0038] Among them, the monocular feature coordinate pair is a coordinate pair composed of the upper eyelid feature coordinate and the lower eyelid feature coordinate of the same eye, which is used to calculate the distance between the upper and lower eyelids of the eye; while the eyelid feature coordinate pair is a set composed of the user's left eye monocular feature coordinate pair and right eye monocular feature coordinate pair, which contains the key coordinate information of the eyelid edges of both eyes.

[0039] Specifically, for the left and right eyes, the upper eyelid feature coordinates and lower eyelid feature coordinates of the same monocular eye are paired to form a left eye monocular feature coordinate pair and a right eye monocular feature coordinate pair. These two pairs of coordinates are then integrated into a complete eyelid feature coordinate pair, thereby clearly distinguishing the eyelid structure of both eyes.

[0040] In addition, at least one pair of reference feature coordinates with a preset correspondence is determined from all the reference feature coordinates to form at least one pair of reference feature coordinates.

[0041] Among them, the reference feature coordinate pair is a pair of two coordinates selected from multiple reference feature coordinates that have a preset spatial correspondence, such as the left and right ear feature coordinate pair, or the eyebrow and chin feature coordinate pair, which are used to calculate the reference feature spacing value later.

[0042] Specifically, coordinates with a pre-defined spatial correspondence (such as left and right ear feature points) are selected from all reference feature coordinates to form at least one pair of reference feature coordinates, providing a stable scale benchmark for subsequent normalization processing. The entire process provides standardized raw data for eye state assessment through high-precision model localization and structured integration of coordinates.

[0043] In one specific embodiment, it is assumed that the system uses a CNN-based feature extraction model to process the user's facial video stream. When a frame of the user's facial image is input into the model, the model outputs the following coordinates: midpoint coordinates of the left upper eyelid (320, 210), midpoint coordinates of the left upper eyelid (320, 230), midpoint coordinates of the right upper eyelid (380, 212), midpoint coordinates of the right upper eyelid (380, 231); reference feature coordinates include the left ear feature point (280, 250) and the right ear feature point (420, 252). At this point, the system pairs (320, 210) and (320, 230) of the left eye as a single-eye feature coordinate pair for the left eye, and pairs (380, 212) and (380, 231) of the right eye as a single-eye feature coordinate pair for the right eye. The two are then integrated into an eyelid feature coordinate pair. Simultaneously, the system selects (280, 250) of the left ear and (420, 252) of the right ear from the reference feature coordinates to form a reference feature coordinate pair.

[0044] In step S3, a reference feature spacing value is determined based on the reference feature coordinate pair, and the upper and lower eyelid spacing value of the currently detected monocular is determined based on the eyelid feature coordinate pair.

[0045] The reference feature spacing value is the straight-line distance calculated based on the pixel coordinates of two feature points in the reference feature coordinate pair. It is used to reflect the spatial scale of the stable facial structure, such as the distance between the left and right ear feature points.

[0046] Specifically, after acquiring the eyelid feature coordinate pairs and reference feature coordinate pairs of the current video frame, the system performs geometric calculations on these two sets of coordinate data. The pixel distance between reference feature points is calculated using the reference feature coordinate pairs, yielding the reference feature spacing value. Simultaneously, the same distance calculation is performed on the upper and lower eyelid feature points in the eyelid feature coordinate pairs to obtain the upper and lower eyelid spacing value for the currently detected single eye. Both spacing values ​​are in pixels and quantify the scale of the stable facial structure and the opening / closing state of the eyes, respectively.

[0047] More specifically, embodiments of this application propose calculating the distance value by the difference in absolute values ​​between coordinate points. That is, the steps of determining the reference feature distance value based on the reference feature coordinate pair and determining the upper and lower eyelid distance value of the currently detected monocular eye based on the eyelid feature coordinate pair include: The coordinate spacing value is calculated based on the monocular feature coordinate pair corresponding to the currently detected monocular eye to obtain the upper and lower eyelid distance value; The coordinate spacing value is calculated based on all the reference feature coordinate pairs to obtain at least one reference feature spacing value.

[0048] The coordinate spacing calculation involves taking the absolute value of the difference between the pixel values ​​of corresponding dimensions in the coordinate pair to obtain the distance between the two points.

[0049] Specifically, when determining the spacing value, the system no longer relies on the coordinate spacing value in a single direction, but instead calculates the Euclidean distance between the two points in the coordinate pair. For the currently detected monocular, based on its monocular feature coordinate pair, i.e., the upper and lower eyelid feature coordinates, the straight-line distance between the two points is directly solved using the Euclidean distance formula to obtain the upper and lower eyelid spacing value. For the reference feature coordinate pair, the straight-line distance between the two points is also calculated using the Euclidean distance formula to obtain the reference feature spacing value. This calculation method does not distinguish between horizontal or vertical directions, directly reflecting the true physical distance between the two points in planar space, and can stably output accurate spacing values ​​regardless of changes in the user's facial posture, such as tilting or rotating.

[0050] Euclidean distance refers to the straight-line distance in a two-dimensional plane, calculated by taking the square root of the sum of the squared differences of the coordinates of two points. It eliminates direction dependence and is applicable to the calculation of the distance between two points under any orientation.

[0051] In one specific embodiment, assuming the currently detected monocular eye is the left eye, its monocular feature coordinate pair is the midpoint of the upper eyelid (320, 210) and the midpoint of the lower eyelid (325, 235). Due to the user's slight head tilt, the eyelid coordinates are offset in both the horizontal and vertical directions. Using Euclidean distance calculation, the distance between the upper and lower eyelids of the left eye is approximately 25.5 pixels. The reference feature coordinate pair is the left ear (280, 250) and the right ear (420, 245). Slight head rotation by the user causes a difference in the vertical direction of the ear coordinates, and the Euclidean distance is calculated to be 140.1 pixels. At this point, even with facial tilt, the Euclidean distance accurately captures the true spatial distance between the two points. In contrast, using traditional vertical or horizontal difference calculations would ignore the offsets in other directions, leading to inaccurate distance calculations. Through comprehensive calculation using Euclidean distance, the system maintains the stability of the normalized ratio even when the user is not stationary or parallel, ensuring that subsequent state judgments are not affected by posture.

[0052] In step S4, the distance between the upper and lower eyelids is normalized by combining the distance between the upper and lower eyelids with the distance between the reference features to obtain the normalized distance value corresponding to the current video frame.

[0053] Normalization is a process of proportionally calculating two physical quantities with the same dimensions (such as pixel distance), eliminating scale effects and converting them into dimensionless relative values. It is often used to eliminate external environmental interference to achieve stable measurement. Normalized distance value is a dimensionless value obtained through normalization that reflects the relative proportion between the distance between the upper and lower eyelids and the distance between the reference features. It is used to quantify the relative degree of eye opening and closing.

[0054] Specifically, after obtaining the distance between the upper and lower eyelids and the distance between the reference features in the current video frame, the system combines these two distance values ​​through preset calculations to convert the absolute pixel distance of the eye opening and closing into a proportional value relative to the scale of the stable facial structure. This eliminates scale interference caused by factors such as changes in the distance between the user and the camera and minor adjustments in head posture, making the eye state measurement in different video frames comparable.

[0055] More specifically, in this embodiment of the application, the step of normalizing the distance between the upper and lower eyelids and the reference feature distance value to obtain the normalized distance value corresponding to the current video frame includes: One reference feature spacing value is determined from all the aforementioned reference feature spacing values ​​as the calibration spacing value; Divide the distance between the upper and lower eyelids by the calibration distance value to obtain the normalized distance value corresponding to the current video frame.

[0056] The calibration spacing value is a reference spacing value selected from multiple reference feature spacing values ​​and used for normalization calculation. Typically, a reference feature spacing with high stability (i.e., small fluctuations when the posture changes) and large absolute distance is selected, such as the interauricular distance.

[0057] Specifically, the system selects one from all reference feature distance values ​​as the calibration distance value. The selection criteria prioritize stability and representativeness; for example, the Euclidean distance between the left and right ear feature coordinate pairs is selected as the calibration distance value. Subsequently, the distance between the upper and lower eyelids of the currently detected monocular is divided by the calibration distance value. The absolute pixel distance is converted into a relative proportion through division, resulting in the normalized distance value corresponding to the current video frame.

[0058] More specifically, preparing multiple reference feature spacing values ​​can significantly improve the system's adaptability to complex usage scenarios. When a user's facial posture changes, the stability of different reference features varies. For example, the distance between the ears is relatively stable when rotating horizontally, while the distance from the tip of the nose to the chin fluctuates less when tilted vertically. By providing multiple options, the system can dynamically select the most stable reference feature based on the real-time posture, avoiding normalization errors caused by distance distortion of a single reference under specific postures.

[0059] It is worth noting that the selection of calibration spacing values ​​can be based on user presets or dynamically selected by a preset analysis model based on real-time posture to find the most stable reference feature. In one specific embodiment, a classification model based on an additional preset decision tree / random forest is used to learn the mapping relationship between posture and stable reference features by training on historical data (such as the fluctuation amplitude of each reference feature under different facial postures, posture parameters, etc.). For example, by inputting the current facial posture parameters such as pitch angle and rotation angle, the model directly outputs the optimal reference feature type (such as the distance between the ears, the distance between the eyebrows and the chin).

[0060] In step S5, it is determined whether the normalized spacing value is within the preset eye-closed determination interval in the consecutive video frames. If so, it is determined that the currently detected single eye is in the eyelid closed state.

[0061] The eye-closing judgment interval is a normalized interval value range preset by the system, which includes the preset eye-closing threshold. and the threshold of the range of fluctuations when eyes are closed Its expression is ,in This is the normalized spacing value, used to define the threshold range of normalized spacing values ​​when the eye is in a closed state.

[0062] Among them, the eyelid closure state is the physiological state in which the upper and lower eyelids of the currently detected single eye are nearly or completely closed, and the corresponding normalized distance value falls within the preset eye closure judgment range.

[0063] Specifically, after obtaining the normalized spacing values ​​of the current and subsequent consecutive video frames, the system monitors these normalized spacing values ​​in real time. First, the system retrieves the preset eye-closed determination interval parameters, and then checks frame by frame whether the normalized spacing value is within this interval. If the normalized spacing values ​​of N consecutive frames (N is the minimum consecutive frame number threshold set by the system, such as 5 frames) all fall within the preset interval, then the currently detected single eye is determined to be in an eyelid-closed state; if the normalized spacing value of any frame exceeds the interval, it is not determined to be an eyelid-closed state, that is, it is determined to be an eyelid-open state.

[0064] More specifically, this application embodiment proposes to execute the above step S5 by setting an eye-closed frame counter. Based on this concept, the step of determining whether the normalized interval value is within a preset eye-closed determination interval in the continuous video frames, and if so, determining that the currently detected single eye is in an eyelid-closed state, includes: Determine whether the normalized spacing value corresponding to the current video frame is within the closed-eye determination interval. If it is within the closed-eye determination interval, increment the closed-eye frame count of the current detected single eye; otherwise, set the closed-eye frame count to zero. Among them, the closed-eye frame count is a counter used to accumulate the number of frames in a continuous video frame whose normalized spacing value is within the closed-eye determination interval, reflecting the duration of the eye state.

[0065] Furthermore, in response to the count of closed-eye frames reaching a preset counting threshold, it is determined that the currently detected monocular eye is in a closed-eye state.

[0066] The preset counting threshold is the minimum number of consecutive frames required by the system to determine the eyelid closure state, used to filter out transient interference.

[0067] Specifically, the system first calculates the normalized distance value of the currently detected monocular eye for each frame and determines whether it falls within a preset closed-eye determination interval. If it falls within the interval, it indicates that the eye may be in a closed eye state, and the closed-eye frame count is incremented by 1; if it does not fall within the interval, it is considered that the eye is not closed, and the closed-eye frame count is reset to 0. Subsequently, the system continuously monitors the closed-eye frame count. When the count reaches a preset counting threshold, it confirms that the currently detected monocular eye has been in a closed state for multiple consecutive frames, thus ultimately determining it to be in a closed eye state. This process, through the rigid constraint of "resetting the count upon entering a non-closed interval," ensures that only a continuously stable closed state can be identified, thus eliminating interference from non-closed states (such as a user secretly opening their eyes).

[0068] In one specific embodiment, assume the system presets the eye-closing judgment interval to 0.1 to 0.2, and the preset counting threshold to 5 frames (corresponding to approximately 0.17 seconds at 30fps). When the user enters the eye-closing phase as instructed, if the normalized spacing value of the first 3 frames is 0.15 (within the interval), the eye-closing frame count is accumulated to 3. In the 4th frame, the user attempts to secretly open their eyes, causing the eyelids to slightly open, resulting in a normalized spacing value of 0.25 (outside the interval), and the system immediately resets the count to 0. In the 5th frame, the user closes their eyes again, the spacing value returns to 0.15, and the count starts accumulating again from 1. Because the act of secretly opening the eyes causes the counting to be interrupted, even if the eyes are closed again subsequently, it is necessary to accumulate to 5 frames again before it can be determined as a closed state. In this process, the deviation in the spacing value caused by secretly opening the eyes directly interrupts the counting accumulation, ensuring that the system only responds to continuous, undisturbed closed states, effectively suppressing the possibility of users circumventing real state monitoring by secretly opening their eyes.

[0069] Conversely, after the step of determining whether the normalized interval value is within a preset closed-eye determination interval in the consecutive video frames, the method further includes: If the normalized spacing value of the current video frame is not within the closed eye determination range, then the currently detected single eye is determined to be in an open eyelid state. The open eyelid state refers to the system determining that the eye is in an open state when the normalized distance value of the currently detected monocular does not fall within the preset closed eye determination range, which is different from the closed eyelid state.

[0070] Furthermore, if either the user's left eye or the user's right eye is in the open eyelid state, then the user is determined to be in the open eyelid state.

[0071] The user's left eye and right eye are monitored independently, and their status judgments are independent of each other, which forms the basis for subsequent comprehensive judgment of the user's overall status.

[0072] Specifically, after determining whether the normalized distance value of the current detected monocular eye falls within the closed-eye determination range, the system further performs reverse deduction and global integration of the eye state. If the normalized distance value of a certain eye in the current video frame does not fall within the closed-eye determination range, the system immediately determines that the eye is in an open-eye state. Subsequently, the system logically integrates the states of the left and right eyes: as long as either the left or right eye is determined to be in an open-eye state, regardless of whether the other eye is closed, the system directly determines that the user is in an overall open-eye state.

[0073] In step S6, the user is determined to be in the eyelid closed state if and only if both the user's left eye and the user's right eye are in the eyelid closed state.

[0074] Specifically, after determining the eyelid closure status of the user's left and right eyes separately, the system performs a logical AND operation on the two results. In other words, the system only determines that the user is in an overall eyelid-closed state if both the left and right eye results are "eyelid-closed." If either eye is not in an eyelid-closed state (e.g., the left eye is closed while the right eye is open), or vice versa, the system does not determine that the user is in an eyelid-closed state, ensuring that the final determination is triggered only when both eyes simultaneously meet the closure condition.

[0075] More specifically, in the embodiments of this application, after the step of determining that the user is in an eyelid-closed state if and only if both the user's left eye and the user's right eye are in the eyelid-closed state, the method further includes: In response to the user being in the closed eyelid state or the open eyelid state, update the corresponding eyelid state association parameters; Based on the updated eyelid state association parameters, a matching judgment is made with the current expected state of the brainwave scanning task. If the eyelid state association parameters do not match the current expected state, an abnormal state prompt message is output.

[0076] Among them, eyelid state correlation parameters are key indicators used to quantify and record changes in a user's eyelid state over time, including but not limited to the duration of the current state, the timestamp of state transition, and the stability of the state per unit time.

[0077] Among them, the abnormal status prompt information is a real-time feedback signal issued by the system to the user or operator when the user's actual eyelid state does not match the current expected state. It may take the form of visual reminders (such as screen flashing), auditory alarms (such as beeping sounds) or text prompts, and is used to correct user behavior deviations.

[0078] Specifically, after determining that the user is in a closed eyelid state, the system will update the corresponding eyelid state-related parameters in real time according to the current actual state (closed or open eyelid). For example, if the system detects that the user has entered a closed eyelid state, it will accumulate the duration of closure and record the start time of the state; if the user switches to an open eyelid state, it will reset the closure duration and record the start time of the open state.

[0079] The system then matches the updated parameters with the expected state of the brainwave scan task, such as a stage requiring "closing eyes for 30 seconds". If the parameters show that the user's actual state does not match the expectation, such as expecting to close their eyes but actually keeping them open for 10 seconds, or the duration of closing their eyes does not meet the requirement, an abnormal state prompt message is triggered to reflect the deviation between the user's current behavior and the instruction.

[0080] Reference Figure 2 , Figure 2This is a virtual structural diagram of the eyelid state recognition device for brainwave scanning users provided in this application. A second aspect of this application provides an eyelid state recognition device for brainwave scanning users, comprising: The image acquisition module 100 is used to continuously acquire the user's facial video stream in response to the triggering of the brainwave scanning task, and obtain the user's facial image corresponding to the continuous video frames. The feature acquisition module 200 is used to acquire features from each of the user's facial images to obtain the eyelid feature coordinate pairs and reference feature coordinate pairs corresponding to the current video frame; The spacing calculation module 300 is used to determine the reference feature spacing value based on the reference feature coordinate pair, and to determine the upper and lower eyelid spacing value of the currently detected monocular eye based on the eyelid feature coordinate pair; the currently detected monocular eye is the user's left eye or the user's right eye; The dimension processing module 400 is used to combine the distance value between the upper and lower eyelids with the distance value of the reference feature to perform normalization processing to obtain the normalized distance value corresponding to the current video frame; The monocular state determination module 500 is used to determine whether the normalized spacing value is within the preset closed eye determination interval in the continuous video frames. If so, it is determined that the currently detected monocular eye is in the eyelid closed state. The user state determination module 600 is used to determine that the user is in the eyelid closed state if and only if both the user's left eye and the user's right eye are in the eyelid closed state.

[0081] The eyelid state recognition device for EEG scanning users described in this application embodiment can execute the eyelid state recognition method for EEG scanning users provided in the above embodiments. The eyelid state recognition device for EEG scanning users has the corresponding functional steps and beneficial effects of the eyelid state recognition method for EEG scanning users described in the above embodiments. For details, please refer to the embodiments of the eyelid state recognition method for EEG scanning users described above. This application embodiment will not be repeated here.

[0082] This application also provides an electronic device, please refer to... Figure 3 , Figure 3This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor and a memory, which can be connected via a bus or other means. The processor may be a Central Processing Unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the eyelid state recognition method for brainwave scanning users in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the eyelid state recognition method for brainwave scanning users in the above method embodiments.

[0083] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. One or more modules are stored in the memory and, when executed by the processor, perform the eyelid state recognition method for brainwave scanning users as described in the above method embodiments. Specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the above method embodiments, and will not be repeated here. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it may include the processes of the embodiments of the above methods. The storage medium may be a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above types of memory.

[0084] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0085] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0086] It should be noted that the above embodiments are illustrative of this application and not restrictive of this application, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims.

Claims

1. A method for recognizing the eyelid state of a user via brainwave scanning, characterized in that, The method includes: In response to the brainwave scanning task, the system continuously acquires the user's facial video stream and obtains the user's facial images corresponding to consecutive video frames. For each user's facial image, feature acquisition is performed to obtain the eyelid feature coordinate pair and the reference feature coordinate pair corresponding to the current video frame; The reference feature spacing value is determined based on the reference feature coordinate pair, and the upper and lower eyelid spacing value of the currently detected monocular eye is determined based on the eyelid feature coordinate pair; the currently detected monocular eye is the user's left eye or the user's right eye; The normalized distance value is obtained by combining the distance value between the upper and lower eyelids with the distance value of the reference feature. Determine whether the normalized spacing value is within the preset eye-closed determination range in the consecutive video frames; if so, determine that the currently detected single eye is in the eyelid closed state. A user is determined to be in a closed eyelid state if and only if both the user's left eye and the user's right eye are in the closed eyelid state.

2. The method for recognizing the eyelid state of a user via brainwave scanning according to claim 1, characterized in that, The step of performing feature acquisition on each user's facial image to obtain the eyelid feature coordinate pair and the reference feature coordinate pair corresponding to the current video frame includes: The user's facial image is input frame by frame into a preset feature extraction model. The feature extraction model is used to extract features from the current video frame to obtain the upper eyelid feature coordinates, the lower eyelid feature coordinates, and multiple reference feature coordinates. The upper eyelid feature coordinates and the lower eyelid feature coordinates of the same monocular eye are paired to form monocular feature coordinate pairs for the user's left eye and the user's right eye, respectively. The two pairs of monocular feature coordinate pairs are then integrated into the eyelid feature coordinate pairs. At least one pair of reference feature coordinates with a preset correspondence is determined from all the reference feature coordinates to form at least one pair of reference feature coordinates.

3. The method for recognizing the eyelid state of a user via brainwave scanning according to claim 2, characterized in that, The steps of determining the reference feature spacing value based on the reference feature coordinate pair and determining the upper and lower eyelid spacing value of the currently detected monocular eye based on the eyelid feature coordinate pair include: The coordinate spacing value is calculated based on the monocular feature coordinate pair corresponding to the currently detected monocular eye to obtain the upper and lower eyelid distance value; The coordinate spacing value is calculated based on all the reference feature coordinate pairs to obtain at least one reference feature spacing value.

4. The method for recognizing the eyelid state of a user via brainwave scanning according to claim 1, characterized in that, The step of normalizing the distance between the upper and lower eyelids and the reference feature distance value to obtain the normalized distance value corresponding to the current video frame includes: One reference feature spacing value is determined from all the aforementioned reference feature spacing values ​​as the calibration spacing value; Divide the distance between the upper and lower eyelids by the calibration distance value to obtain the normalized distance value corresponding to the current video frame.

5. The method for recognizing the eyelid state of a user via brainwave scanning according to claim 1, characterized in that, The step of determining whether the normalized spacing value is within a preset eye-closed determination interval in the consecutive video frames, and if so, determining that the currently detected monocular eye is in an eyelid-closed state, includes: Determine whether the normalized spacing value corresponding to the current video frame is within the closed-eye determination interval. If it is within the closed-eye determination interval, increment the closed-eye frame count of the current detected single eye; otherwise, set the closed-eye frame count to zero. In response to the count of closed-eye frames reaching a preset count threshold, the currently detected single eye is determined to be in an eyelid-closed state.

6. The method for recognizing the eyelid state of a user via brainwave scanning according to claim 5, characterized in that, After the step of determining whether the normalized spacing value is within the preset closed-eye determination interval in the consecutive video frames, the method further includes: If the normalized spacing value of the current video frame is not within the closed eye determination range, then the currently detected single eye is determined to be in an open eyelid state. If either the user's left eye or the user's right eye is in the open eyelid state, then the user is determined to be in the open eyelid state.

7. The method for recognizing the eyelid state of a user via brainwave scanning according to claim 6, characterized in that, After the step of determining that the user is in an eyelid-closed state if and only if both the user's left eye and the user's right eye are in the eyelid-closed state, the method further includes: In response to the user being in the closed eyelid state or the open eyelid state, update the corresponding eyelid state association parameters; Based on the updated eyelid state association parameters, a matching judgment is made with the current expected state of the brainwave scanning task. If the eyelid state association parameters do not match the current expected state, an abnormal state prompt message is output.

8. A device for recognizing the eyelid state of a user undergoing brainwave scanning, characterized in that, include: The image acquisition module is used to continuously acquire the user's facial video stream in response to the triggering of the brainwave scanning task, and obtain the user's facial image corresponding to the continuous video frames. The feature acquisition module is used to acquire features from each of the user's facial images to obtain the eyelid feature coordinate pairs and reference feature coordinate pairs corresponding to the current video frame; The spacing calculation module is used to determine the reference feature spacing value based on the reference feature coordinate pair, and to determine the upper and lower eyelid spacing value of the currently detected monocular eye based on the eyelid feature coordinate pair; the currently detected monocular eye is the user's left eye or the user's right eye; The dimension processing module is used to combine the distance value between the upper and lower eyelids with the distance value of the reference feature to perform normalization processing, so as to obtain the normalized distance value corresponding to the current video frame; The monocular state determination module is used to determine whether the normalized spacing value is within the preset closed eye determination interval in the continuous video frames. If so, it is determined that the currently detected monocular eye is in the eyelid closed state. The user state determination module is used to determine that the user is in the eyelid closed state if and only if both the user's left eye and the user's right eye are in the eyelid closed state.

9. An electronic device, characterized in that, include: One or more processors; One or more memory units; And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 7.

Images