A method and device for blink data statistics based on deep learning

Abstract

This paper discloses a method for counting blink data based on deep learning. First, multiple frames of human eye video stream images are acquired. Then, each frame of the human eye image is input into a trained eyelid detection network for detection, obtaining human eye image probability data and upper and lower eyelid center position data. Next, based on the human eye image probability data and upper and lower eyelid center position data, the upper and lower eyelid height values ​​are calculated to obtain an upper and lower eyelid height sequence. Finally, based on the upper and lower eyelid height sequence, the proportion of incomplete blinks during video capture is counted. This method can improve the efficiency and accuracy of eyelid position detection and can also count the proportion of incomplete blinks during video capture in real time based on the eyelid height values.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a blink data statistics method, apparatus, computing device and storage medium based on deep learning. Background Art

[0002] Blinking is a rapid eye-closing action, also known as the blink reflex. When blinking, the tear film can be evenly distributed on the surface of the eye to moisten the eyes. At the same time, people can also promote the discharge of lipids (commonly known as "oil") secreted by the meibomian glands through the action of blinking. As an important component of the tear film in the eyes, the role of lipids is to "lock in moisture and moisturize", protecting tears from evaporation, so as to improve the stability of the tear film and prevent dry eyes. If people do not blink completely, that is, each blink cannot completely cover the entire cornea, resulting in a decrease in the quality of blinking, the force of lipid discharge is not enough, which will also reduce the secretion of lipids. Due to the lack of lipid protection, the stability of the tear film is weakened, and the evaporation of tears is aggravated, which can easily lead to dry eyes. Therefore, observing and statistically analyzing the proportion of incomplete blinks is of great reference significance for the diagnosis of dry eyes.

[0003] The Chinese invention patent document with application number CN113616196A discloses a statistical method, device, medium and electronic device for blink data. In this method, a video stream of blinking is obtained, a seed point area extraction and analysis is performed on each frame of the image, and then the height of the image acquisition area is binarized to obtain the degree of incomplete blinking. However, the seed point selection of this method is relatively difficult and is easily affected by light and eyelashes, resulting in a reduced detection rate. By calculating the palpebral fissure height corresponding to the target area and determining the opening and closing state of the target area based on the palpebral fissure height, the blink data within a preset time period is statistically obtained based on the opening and closing state of the target area of ​​each frame of the video data. This method can reduce the difficulty of data statistics, but it can only count the opening and closing states, and cannot count the proportion of incomplete blinks during the video shooting of the human eye. In addition, the traditional image processing method is used to detect the position of the upper and lower eyelids, and the detection efficiency and accuracy are low, and the accuracy of subsequent statistical data cannot be guaranteed.

[0004] Therefore, it is necessary to provide a blink data statistics method that can improve the accuracy and efficiency of eyelid position detection, and count the proportion of incomplete blinks during human eye video shooting in real time, so as to improve the accuracy and efficiency of data statistics and solve the problems existing in the above-mentioned existing technologies. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a method, apparatus, computing device and storage medium for blink data statistics based on deep learning that overcome the above problems or at least partially solve the above problems.

[0006] According to one aspect of the present invention, a method for counting blink data based on deep learning is provided, in which, first, multiple frames of human eye video stream images are acquired; then, each frame of the human eye image is input into a trained eyelid detection network for detection to obtain human eye image probability data and upper and lower eyelid center position data; then, based on the human eye image probability data and the upper and lower eyelid center position data, the upper and lower eyelid height values ​​are calculated to obtain an upper and lower eyelid height sequence; finally, based on the upper and lower eyelid height sequence, the proportion of incomplete blinks during the human eye video shooting is counted.

[0007] This method detects human eye images based on a deep learning network. It can simultaneously obtain the probability that the image is a human eye image and the upper and lower eyelid position data, which can improve the accuracy and efficiency of eyelid position detection. In addition, based on the eyelid height value sequence, the proportion of incomplete blinks during human eye video shooting can be counted in real time, providing data support for incomplete blink diagnosis.

[0008] Optionally, in the method according to the present invention, a human eye video stream can be collected in real time by a camera; and then the human eye video stream is subjected to frame processing to obtain multiple frames of continuous human eye images.

[0009] Optionally, in the method according to the present invention, the acquired human eye image can be preprocessed to obtain an image of the target area between the upper eyelid and the lower eyelid, so that the target area image can be input into a trained eyelid detection network for detection. The preprocessing includes image rotation, cropping, grayscale transformation, binarization and normalization.

[0010] Optionally, in the method according to the present invention, the eyelid detection network includes a backbone network, a feature fusion module, a probability prediction module, and a position prediction module, wherein the backbone network includes an input layer and multiple alternating cascaded convolutional layers, normalization layers, and activation layers; the feature fusion module includes a cascaded first fully connected layer, a first normalization layer, and a first activation layer; the probability prediction module and the position prediction module are connected in parallel to the output end of the feature fusion module; the probability prediction module includes a second fully connected layer, a softmax layer, and a first output layer, and is suitable for predicting the probability that the input image is an image of a human eye. The position prediction module includes a third fully connected layer and a second output layer, and is suitable for predicting the center position coordinates of the upper eyelid and the lower eyelid.

[0011] Optionally, in the method according to the present invention, if the human eye probability value output by the eyelid detection network is greater than a predetermined threshold, the eyelid height value is determined based on the difference between the vertical coordinates of the upper eyelid center coordinate and the vertical coordinates of the lower eyelid center coordinate in the human eye image to obtain the upper and lower eyelid height sequence.

[0012] Optionally, in the method according to the present invention, the height values ​​in the upper and lower eyelid height sequence can be divided by the maximum height value in the height sequence to obtain a blink degree sequence; then, based on the blink degree sequence, the blink cycle is determined. When the minimum blink degree in a blink cycle is less than a first threshold, it is judged as a complete blink; when the minimum blink degree in a blink cycle is greater than the first threshold and less than a second threshold, it is judged as an incomplete blink; when the minimum blink degree in a blink cycle is greater than the second threshold, it is judged as a non-blink; finally, the number of incomplete blinks and the total number of blinks during the human eye video shooting are counted to obtain the incomplete blink ratio, wherein the human eye video shooting period includes multiple blink cycles, and the total number of blinks is equal to the sum of the number of complete blinks and the number of incomplete blinks.

[0013] Optionally, in the method according to the present invention, a process in which the blink degree changes from large to small and then to large again in the blink degree sequence is determined as a blink cycle.

[0014] According to another aspect of the present invention, a blink data statistics device based on deep learning is provided, which includes an acquisition module, a detection module, a calculation module and a statistics module, wherein the acquisition module can acquire multiple frames of human eye video stream images; the detection module can input each frame of human eye image acquired by the acquisition module into a trained eyelid detection network for detection, and obtain human eye image probability data and upper and lower eyelid center position data; the calculation module can calculate the upper and lower eyelid height values ​​based on the human eye image probability data and upper and lower eyelid center position data output by the detection module, and obtain the upper and lower eyelid height sequence; the statistics module can count the proportion of incomplete blinks during human eye video shooting based on the upper and lower eyelid height sequence obtained by the calculation module.

[0015] According to another aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be suitable for execution by the at least one processor, and the program instructions include instructions for executing the above method.

[0016] According to another aspect of the present invention, a readable storage medium storing program instructions is provided. When the program instructions are read and executed by a computing device, the computing device executes the above method.

[0017] According to the solution of the present invention, the probability of human eye images and the positions of upper and lower eyelids are detected through a deep learning neural network, which can improve the accuracy and efficiency of eyelid position detection, and can perform incomplete blink statistics based on the eyelid height value sequence, providing data support for incomplete blink diagnosis.

[0018] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0020] Figure 1 shows a structural diagram of a computing device 100 according to one embodiment of the present invention;

[0021] Figure 2 1 shows a flow chart of a blink data statistics method 200 based on deep learning according to an embodiment of the present invention;

[0022] Figure 3 FIG2 shows a schematic diagram of the structure of an eyelid detection network 300 according to an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of eyelid height calculation according to an embodiment of the present invention is shown;

[0024] Figure 5 A schematic structural diagram of a blink data statistics device 500 based on deep learning according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0026] Common blink detection methods generally use image processing technology to perform image enhancement and edge extraction to determine the position and height difference of the upper and lower eyelids. This method is easily affected by interference from other eye tissues, shooting angle, light source and other factors, resulting in low detection accuracy. Existing blink data statistics methods determine the opening and closing state based on the height of the palpebral fissure, which reduces the difficulty of blink data statistics. This method can only determine the opening and closing state of the eyes, and cannot count the proportion of incomplete blinks in real time. Therefore, this scheme uses a deep learning method to detect the eyelid position of the human eye video stream data in real time, and counts the proportion of incomplete blinks during the human eye video shooting in real time based on the eyelid position, which is of great reference significance for dry eye diagnosis.

[0027] Figure 1 FIG. 1 shows a block diagram of a computing device 100 according to an embodiment of the present invention. Figure 1 As shown, in a basic configuration 102, computing device 100 typically includes system memory 106 and one or more processors 104. A memory bus 108 may be used for communication between processor 104 and system memory 106.

[0028] Depending on the desired configuration, processor 104 may be any type of processor, including, but not limited to, a microprocessor (µP), a microcontroller (µC), a digital signal processor (DSP), or any combination thereof. Processor 104 may include one or more levels of cache, such as a level 1 cache 110 and a level 2 cache 112, a processor core 114, and registers 116. An example processor core 114 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 118 may be used with processor 104, or in some implementations, may be an internal component of processor 104.

[0029] Depending on the desired configuration, system memory 106 can be any type of memory, including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory), or any combination thereof. Physical memory in a computing device typically refers to volatile RAM. Data stored on disk must be loaded into physical memory before it can be read by processor 104. System memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some embodiments, applications 122 may be arranged so that one or more processors 104 execute instructions on the operating system using program data 124. Operating system 120, for example, may be Linux or Windows, and includes program instructions for handling basic system services and performing hardware-dependent tasks. Applications 122 include program instructions for implementing various user-desired functions. Applications 122 may include, but are not limited to, browsers, instant messaging software, software development tools (e.g., integrated development environments (IDEs) and compilers), etc. When applications 122 are installed in computing device 100, driver modules may be added to operating system 120.

[0030] When the computing device 100 is started, the processor 104 reads and executes the program instructions of the operating system 120 from the memory 106. Applications 122 run on top of the operating system 120, utilizing the interfaces provided by the operating system 120 and the underlying hardware to implement various user-desired functions. When a user launches an application 122, the application 122 is loaded into the memory 106, and the processor 104 reads and executes the program instructions of the application 122 from the memory 106.

[0031] Computing device 100 also includes a storage device 132 including removable storage 136 and non-removable storage 138 , both of which are connected to storage interface bus 134 .

[0032] Computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to basic configuration 102 via bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. These can be configured to facilitate communication with various external devices, such as a display or speakers, via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which can be configured to facilitate communication with external devices, such as input devices (e.g., a keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., a printer, scanner, etc.), via one or more I / O ports 158. Example communication devices 146 may include a network controller 160, which can be arranged to facilitate communication with one or more other computing devices 162 via a network communication link via one or more communication ports 164.

[0033] A network communication link can be an example of a communication medium. Communication media can generally be embodied as computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of its data sets or a change thereof can be performed in a manner that encodes information in the signal. As non-limiting examples, communication media can include wired media such as wired networks or dedicated networks, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR) or other wireless media. The term computer-readable media used herein can include both storage media and communication media. In the computing device 100 according to the present invention, the application 122 includes instructions for executing the deep learning-based blink data statistics method 200 of the present invention.

[0034] Figure 2 FIG. 2 shows a flow chart of a method 200 for counting blink data based on deep learning according to an embodiment of the present invention. Figure 2As shown, the method 200 starts at step S210, and a plurality of frames of human eye video stream images are obtained. In one embodiment of the present invention, a human eye video stream containing multiple blink cycles can be collected in real time by a camera, and the video stream contains multiple frames of images. The obtained video stream can be framed to obtain multiple frames of temporally continuous human eye images. In order to improve the detection efficiency and accuracy of the deep learning neural network for the input image, the obtained human eye image can be pre-processed by image rotation, cropping, grayscale processing, binarization processing, normalization, etc. to obtain an image of the target area between the upper eyelid and the lower eyelid, which is used as the input image of the subsequent eyelid detection network. The human eye image obtained in real time may contain the target area, or it may not contain the target area due to changes in the eye position during the image acquisition process, and further detection is required.

[0035] Then, step S220 is executed to input each frame of the human eye image into the trained eyelid detection network for detection to obtain human eye image probability data and upper and lower eyelid center position data.

[0036] In an embodiment of the present invention, the network structure of the eyelid detection network can be pre-constructed, and parameters such as weights can be randomly initialized. Then, the labeled human eye images are used as training images to input into the eyelid detection network for training. Based on the back propagation algorithm, the defined loss function is minimized, and the parameters of the network are continuously optimized to obtain a trained eyelid detection network.

[0037] Figure 3 FIG. 3 is a schematic diagram showing the structure of an eyelid detection network 300 according to an embodiment of the present invention. The eyelid detection network 300 includes a backbone network 310, a feature fusion module 320, a probability prediction module 330, and a position prediction module 340. Figure 3 As shown, the backbone network 310 may include an input layer and multiple alternating cascaded convolutional layers, batchnorm normalization layers, and Relu activation layers. The input layer preprocesses the input image, such as by normalizing the image, performing data augmentation, and performing random cropping. The convolutional layers extract features from the input image, such as edges, lines, and corners. The greater the number of convolutional layers, the more abstract the extracted image features become. The batchnorm (BN) normalization layer smooths the loss function space of the neural network, making gradient descent less likely to fall into local minima and reducing reliance on weight initialization. It also reduces the range of gradient values ​​during parameter updates, making gradient updates more predictable and less prone to gradient explosion and vanishing. This allows for random network initialization and the use of larger learning rates. The Relu activation layer strengthens the nonlinear relationships between the neural network layers. The number of cascaded convolutional, normalization, and activation layers in the backbone network can be appropriately increased to improve the accuracy of image feature extraction.

[0038] Feature fusion module 320 may include a cascaded first fully connected layer, a first normalization layer, and a first activation layer. Feature fusion aims to combine features extracted from the image into a single feature with greater discriminative power than the input features, an important means of improving network performance. The fully connected layer can combine features extracted by the backbone network into a single feature with greater discriminative power than the input features.

[0039] Probability prediction module 330 and position prediction module 340 are connected in parallel at the output of feature fusion module 320. Probability prediction module 330 includes a second fully connected layer, a softmax layer, and a first output layer, and is used to predict the probability that the input image is a human eye image. The softmax layer converts the output data of the fully connected layer into an output value after a probability expression, and the second output layer obtains the probability that the image is a human eye image. Position prediction module 340 includes a third fully connected layer and a second output layer, and is used to predict the center coordinates of the upper and lower eyelids.

[0040] Then, step S230 is executed to calculate the upper and lower eyelid height values ​​based on the human eye image probability data and the upper and lower eyelid center position data to obtain the upper and lower eyelid height sequence.

[0041] According to an embodiment of the present invention, if the eyelid detection network outputs a human eye probability value greater than a predetermined threshold, for example, the probability that an image is a human eye image is greater than 0.5, the eyelid height value is calculated based on the difference between the ordinates of the upper eyelid center and the lower eyelid center in the human eye image. If the eyelid detection network outputs a human eye probability value greater than a predetermined threshold, for example, the probability that an image is a human eye image is less than 0.5, the image is discarded from the eyelid height calculation.

[0042] Figure 4 FIG. 4 shows a schematic diagram of calculating the eyelid height value according to an embodiment of the present invention. Figure 4 As shown, the eyelid height value H is equal to the difference between the ordinate y0 of the upper eyelid center coordinate and the ordinate y1 of the lower eyelid center coordinate. The eyelid height value is calculated for each frame of filtered human eye image, and finally the eyelid height value sequence corresponding to the human eye image is obtained.

[0043] Finally, step S240 is executed to count the proportion of incomplete blinks during the video capture of the human eye based on the upper and lower eyelid height sequence.

[0044] According to an embodiment of the present invention, first, the height values ​​in the upper and lower eyelid height sequence are divided by the maximum height value in the height sequence to obtain a blink degree sequence; then, based on the blink degree sequence, the blink cycle is determined, that is, the process of the blink degree changing from large to small and then to large again can be determined as a blink cycle. When the minimum blink degree in a blink cycle is less than a first threshold, it is judged as a complete blink; when the minimum blink degree in a blink cycle is greater than the first threshold and less than the second threshold, it is judged as an incomplete blink; when the minimum blink degree in a blink cycle is greater than the second threshold, it is judged as a non-blink. For example, when the minimum blink degree in a blink cycle is less than 25%, it is judged as a complete blink; when the minimum blink degree is between 25% and 70%, it is judged as an incomplete blink; when the minimum blink degree is greater than 70%, it is judged as a non-blink. Finally, the number of incomplete blinks and the total number of blinks during the human eye video shooting are counted to obtain the incomplete blink ratio, where the human eye video shooting period includes multiple blink cycles, and the total number of blinks is equal to the sum of the number of complete blinks and the number of incomplete blinks.

[0045] Figure 5 FIG. 5 shows a schematic diagram of a structure of a blink data statistics device 500 based on deep learning according to an embodiment of the present invention. Figure 5 As shown, the device 500 includes an acquisition module 510, a detection module 520, a calculation module 530, and a statistics module 540. The acquisition module 510 can acquire multiple frames of human eye video stream images. The detection module 520 can input each frame of human eye image acquired by the acquisition module 510 into a trained eyelid detection network for detection to obtain human eye image probability data and upper and lower eyelid center position data. The calculation module 530 can calculate the upper and lower eyelid height values ​​based on the human eye image probability data and upper and lower eyelid center position data output by the detection module 520 to obtain an upper and lower eyelid height sequence. The statistics module 540 can count the proportion of incomplete blinks during the human eye video shooting based on the upper and lower eyelid height sequence obtained by the calculation module 530.

[0046] Through the above scheme, the probability of human eye images and the positions of upper and lower eyelids are detected by deep learning neural networks, which can improve the accuracy and efficiency of eyelid position detection, and can perform incomplete blink statistics based on the eyelid height value sequence, providing data support for incomplete blink diagnosis.

[0047] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0048] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims that follow the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0049] Those skilled in the art will appreciate that the modules, units, or components of the devices in the examples disclosed herein may be arranged in the device described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or further divided into multiple submodules.

[0050] Those skilled in the art will appreciate that the modules in the devices of the embodiments can be adaptively modified and installed in one or more devices different from the embodiments. The modules, units, or components in the embodiments can be combined into a single module, unit, or component, and furthermore, they can be divided into multiple sub-modules, sub-units, or sub-components. All features disclosed in this specification (including the accompanying claims, abstract, and drawings), and all processes or units of any method or device disclosed therein, can be combined in any combination, unless at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0051] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0052] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions described. Thus, a processor having the necessary instructions for implementing the method or method element forms a device for implementing the method or method element. Furthermore, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0053] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved and are not intended to imply that the objects so described must have a given order in time, space, ranking, or in any other manner.

[0054] Although the present invention has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art, having benefit of the foregoing description, that other embodiments are contemplated within the scope of the invention thus described. Furthermore, it should be noted that the language used in this specification has been selected primarily for readability and didactic purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention. Consequently, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative and not restrictive of the scope of the invention, which is defined by the appended claims.

Claims

1. A method for analyzing blink data statistics based on deep learning, suitable for execution in a computing device, characterized in that: The method comprises: Obtain multiple frames of human eye video stream images; Preprocessing the acquired human eye image to obtain an image of the target area between the upper eyelid and the lower eyelid, so that the target area image can be input into a trained eyelid detection network for detection, wherein the preprocessing includes image rotation, cropping, grayscale conversion, binarization, and normalization; Input each frame of the target area image into the trained eyelid detection network for detection to obtain human eye image probability data and upper and lower eyelid center position data; If the eye probability value output by the eyelid detection network is greater than a predetermined threshold, the eyelid height value is determined according to the difference between the ordinate of the upper eyelid center coordinate and the ordinate of the lower eyelid center coordinate in the human eye image, thereby obtaining an upper and lower eyelid height sequence; Divide the height values ​​in the upper and lower eyelid height sequence by the maximum height value in the height sequence to obtain the blink degree sequence; The process in which the blink degree changes from large to small and then to large again in the blink degree sequence is determined as a blink cycle; When the minimum blink degree in a blink cycle is less than the first threshold, it is judged as a complete blink; when the minimum blink degree in a blink cycle is greater than the first threshold and less than the second threshold, it is judged as an incomplete blink; when the minimum blink degree in a blink cycle is greater than the second threshold, it is judged as a non-blink; The number of incomplete blinks and the total number of blinks during the video capture of the human eye are counted to obtain an incomplete blink ratio, wherein the video capture of the human eye includes multiple blink cycles, and the total number of blinks is equal to the sum of the number of complete blinks and the number of incomplete blinks.

2. The method according to claim 1, characterized in that The steps of acquiring multiple frames of human eye video stream images in real time include: Collect human eye video stream in real time through the camera; The human eye video stream is subjected to frame processing to obtain multiple frames of continuous human eye images.

3. The method according to claim 1, characterized in that The eyelid detection network includes a backbone network, a feature fusion module, a probability prediction module and a position prediction module. The backbone network includes an input layer and multiple alternating cascaded convolutional layers, normalization layers, and activation layers. The feature fusion module includes a cascaded first fully connected layer, a first normalization layer, and a first activation layer. The probability prediction module and the position prediction module are connected in parallel to the output end of the feature fusion module. The probability prediction module includes a second fully connected layer, a softmax layer, and a first output layer, which is suitable for predicting the probability that the input image is a human eye image. The position prediction module includes a third fully connected layer and a second output layer, which is suitable for predicting the center position coordinates of the upper eyelid and the lower eyelid.

4. A blink data statistics device based on deep learning, characterized in that: The device comprises: An acquisition module is adapted to acquire multiple frames of human eye video stream images; preprocess the acquired human eye images to obtain an image of a target area between the upper eyelid and the lower eyelid, so as to input the target area image into a trained eyelid detection network for detection, wherein the preprocessing includes image rotation, cropping, grayscale conversion, binarization, and normalization; a detection module adapted to input each frame of the target area image acquired by the acquisition module into a trained eyelid detection network for detection, thereby obtaining human eye image probability data and upper and lower eyelid center position data; a calculation module, adapted to determine an eyelid height value according to a difference between a ordinate of a center coordinate of an upper eyelid and a ordinate of a center coordinate of a lower eyelid in the eye image if the eye probability value output by the eyelid detection network is greater than a predetermined threshold, thereby obtaining an upper and lower eyelid height sequence; The statistical module is suitable for dividing the height values ​​in the upper and lower eyelid height sequence by the maximum height value in the height sequence to obtain a blink degree sequence; determining the process in which the blink degree in the blink degree sequence changes from large to small and then to large as a blink cycle; when the minimum blink degree in a blink cycle is less than a first threshold, it is judged as a complete blink; when the minimum blink degree in a blink cycle is greater than the first threshold and less than a second threshold, it is judged as an incomplete blink; when the minimum blink degree in a blink cycle is greater than the second threshold, it is judged as a non-blink; counting the number of incomplete blinks and the total number of blinks during the human eye video shooting to obtain the incomplete blink ratio, the human eye video shooting period includes multiple blink cycles, and the total number of blinks is equal to the sum of the number of complete blinks and the number of incomplete blinks.

5. A computing device comprising: at least one processor; and A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1 to 3.

6. A readable storage medium storing program instructions, wherein when the program instructions are read and executed by a computing device, the computing device is caused to execute the method according to any one of claims 1 to 3.

Images