Eye opening and closing state detection method and device, electronic equipment and medium

Abstract

The application provides an eye open-close state detection method and device, electronic equipment and medium. The method comprises the following steps: acquiring a to-be-detected eye image, determining eye aspect ratio information of the to-be-detected eye image, and extracting HOG features of the to-be-detected eye image; inputting the eye aspect ratio information and the HOG features into a target support vector machine model for classification, determining an open-close eye state category corresponding to the to-be-detected eye image, and the target support vector machine model is a support vector machine model optimized by a particle swarm algorithm. The eye open-close state detection method and device, electronic equipment and medium provided by the application improve the detection efficiency and accuracy of the eye open-close state.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and medium for detecting the open and closed state of the eyes. Background Technology

[0002] Railway dispatching is a command position in railway operations, and its decision-making directly affects railway operational safety. Fatigue detection technology for railway dispatchers is one of the key technologies for preventing railway traffic accidents. This involves collecting video image data of dispatchers at work and using computer vision-based detection methods to determine their fatigue state, with eye condition detection being a crucial step.

[0003] In existing technologies, image binarization or Local Binary Pattern (LBP) feature extraction methods are commonly used to determine whether eyes are closed. Image binarization determines eye state by the difference between black pixels in two consecutive frames. However, this method has drawbacks: cropping significantly affects the distance between the eye and the camera, and if the person being tested has their eyes continuously closed, the difference in black pixels cannot be reflected. LBP feature extraction suffers from poor robustness under complex lighting conditions. Both of these problems lead to low eye state recognition rates. Summary of the Invention

[0004] This invention provides a method, device, electronic device, and medium for detecting eye opening and closing states, in order to solve the problem of low human eye recognition rate in the prior art.

[0005] This invention provides a method for detecting eye opening and closing status, comprising:

[0006] Acquire the image of the eye to be detected

[0007] Determine the aspect ratio information of the eye in the image to be detected, and extract the HOG features of the eye image to be detected;

[0008] The eye aspect ratio information and the HOG features are input into the target support vector machine model for classification to determine the open / closed eye state category of the eye image to be detected. The target support vector machine model is a support vector machine model optimized by particle swarm optimization algorithm.

[0009] In some embodiments, the target support vector machine model is trained in the following manner:

[0010] Obtain eye image samples;

[0011] Based on the aspect ratio information of the eyes in the eye image samples, the HOG features of the eye image samples, and the open / closed state category of the eye image samples, a feature vector dataset corresponding to the eye image samples is constructed.

[0012] The target support vector machine model is trained based on the feature vector dataset.

[0013] In some embodiments, training the target support vector machine model based on the feature vector dataset includes:

[0014] The feature vector dataset is processed using the particle swarm optimization algorithm to obtain the optimal parameters;

[0015] The target support vector machine model is trained based on the optimal parameters and the feature vector dataset.

[0016] In some embodiments, acquiring eye image samples includes:

[0017] Obtain a dataset of facial landmark images;

[0018] The facial landmark image dataset is input into a deep learning network to detect eye landmarks, thereby obtaining an eye image dataset.

[0019] The eye image dataset is converted to grayscale to obtain the eye image samples.

[0020] In some embodiments, inputting the facial landmark image dataset into a deep learning network for eye landmark detection to obtain an eye image dataset includes:

[0021] The facial key point image dataset is input into a deep learning network for feature extraction to determine the nose tip location point and the position of the face frame line;

[0022] The eye image dataset is selected based on the location of the nose tip of the face and the position of the line of the face frame.

[0023] In some embodiments, the backbone feature extraction network of the deep learning network is a lightweight convolutional network feature extractor.

[0024] The present invention also provides an eye opening and closing state detection device, comprising:

[0025] The first acquisition module is used to acquire the image of the eye to be detected.

[0026] The first processing module is used to determine the aspect ratio information of the eyes in the eye image to be detected and to extract the HOG features of the eye image to be detected.

[0027] The first determining module is used to input the eye aspect ratio information and the HOG features into the target support vector machine model for classification, and determine the open / closed eye state category corresponding to the eye image to be detected.

[0028] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the eye opening and closing state detection method as described above.

[0029] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the eye opening and closing state detection method as described above.

[0030] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the eye opening and closing state detection method as described above.

[0031] The present invention provides a method, apparatus, electronic device, and medium for detecting eye opening and closing states. By inputting the aspect ratio information and HOG features of the eye image to be detected into a target support vector machine model for classification, it can quickly realize the detection of eye opening and closing states. It effectively improves the ability to recognize eye opening and closing states in images under conditions such as facial expressions and uneven lighting, and enhances the robustness of the target support vector machine model. Through particle swarm optimization algorithm, the classification ability of the target support vector machine model is improved, ensuring the effectiveness of the target support vector machine model. At the same time, the support vector machine algorithm used has low computing power, ensuring that the target support vector machine model can perform eye opening and closing state detection in small industrial control computers or embedded systems with low computing power. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 This is one of the flowcharts of the eye opening and closing state detection method provided by the present invention;

[0034] Figure 2 This is a schematic diagram of key points of the eye in the eye opening and closing state detection method provided by the present invention;

[0035] Figure 3 This is the second flowchart of the eye opening and closing state detection method provided by the present invention;

[0036] Figure 4 This is a schematic diagram of the eye opening and closing state detection device provided by the present invention;

[0037] Figure 5This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0039] The following description, in conjunction with the accompanying drawings, describes the eye opening and closing state detection method, apparatus, electronic device, and medium provided by the present invention.

[0040] Figure 1 This is one of the flowcharts illustrating the eye opening / closing state detection method provided by the present invention. (Refer to...) Figure 1 The eye opening and closing state detection method provided by the present invention includes steps 110, 120 and 130.

[0041] Step 110: Obtain the image of the eye to be detected.

[0042] Step 120: Determine the aspect ratio information of the eyes in the image to be detected, and extract the HOG features of the image to be detected;

[0043] Step 130: Input the eye aspect ratio information and HOG features into the target support vector machine model for classification to determine the open / closed eye state category of the eye image to be detected. The target support vector machine model is a support vector machine model optimized by particle swarm optimization algorithm.

[0044] It should be noted that the execution subject of the eye opening / closing state detection method provided by the present invention can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. The present invention does not impose specific limitations.

[0045] The technical solution of the present invention will be described in detail below using the computer execution of the eye opening and closing state detection method provided by the present invention as an example.

[0046] In practice, the eye image refers to the image of the eye region within the facial area, such as the left eye image or the right eye image. Eye opening / closing state detection refers to detecting the opening and closing status of the eyes. Eye opening / closing state detection can be used in fields such as fatigue monitoring, liveness detection, and facial expression recognition.

[0047] The eye image to be detected in this embodiment is an image for which eye opening and closing status detection is required. The eye image to be detected can be extracted from real-time acquired face images, including but not limited to obtaining it after face recognition from personnel requiring eye opening and closing status detection, such as train dispatchers or drivers; alternatively, images can be directly acquired from the eye area of ​​a specified person. Then, a deep learning network can be used to perform key point detection on the acquired face or eye images, thereby filtering out eye images containing key eye points. The deep learning network includes, but is not limited to, the Retina-Face network model.

[0048] After acquiring the image of the eye to be detected, the aspect ratio of the eye in the image is calculated based on multiple key points of the eye contained in the image, thus obtaining the eye aspect ratio information. The eye aspect ratio (EAR) is used to indicate the ratio between the height and width of the eye.

[0049] like Figure 2 As shown, the image of the eye to be detected contains six key eye points: P1, P2, P3, P4, P5, and P6. The aspect ratio of the eye can be calculated using the following formula:

[0050]

[0051] After acquiring the image of the eye to be detected, a Histogram of Oriented Gradients (HOG) can be extracted from the image. HOG features are feature descriptors used for object detection in computer vision and image processing. HOG features are constructed by calculating and statistically analyzing the gradient orientation histograms of local image regions. The specific method for obtaining HOG features will be described in detail in subsequent embodiments.

[0052] In step 130, the aspect ratio information of the eye image to be detected and the HOG features of the eye image to be detected are input into the trained target support vector machine model for classification. This allows the determination of the open / closed eye state category of the eye image to be detected, thus realizing the detection of the open / closed eye state. The open / closed eye state category is the classification result of the target support vector machine model, which can be "open eye state" or "closed eye state", or "open eye image" or "closed eye image", etc.

[0053] Among them, the target support vector machine model is a support vector machine model optimized by the particle swarm algorithm (PSO-SVM).

[0054] It should be noted that the basic idea of ​​the Support Vector Machine (SVM) model is to find a hyperplane to separate two classes of samples. When they are linearly separable, the optimal hyperplane requires that the samples are correctly separated and the classification margin is maximized. When they are linearly inseparable, a multidimensional hyperplane is sought to separate the samples.

[0055] Particle Swarm Optimization-Support (PSO) is a novel intelligent biomimetic optimization algorithm based on the simulation of bird predation. This algorithm treats each individual in the swarm as a massless, volumeless point in a multi-dimensional space, flying at a certain speed. During the iterative process, it dynamically adjusts its flight speed based on its own and its companions' flight experience to correct its direction and velocity. PSO searches for the optimal solution to the problem by adjusting each particle to a better region using a fitness function.

[0056] The algorithm flow for optimizing the Support Vector Machine (SVM) model using the Particle Swarm Optimization (PSO) algorithm is as follows: First, a dataset is selected as the input data for the PSO algorithm. Then, the particles and velocities are initialized, and the particle fitness is calculated. Next, based on the fitness values, the individual and swarm extreme values ​​are updated. Then, particle fitness is calculated again, and particle positions and velocities are updated. Under optimal conditions, the optimal SVM parameters are obtained. The SVM model is trained based on the optimal SVM parameters and the training dataset to obtain the trained target SVM model, which can then be used for classification. The specific training process of the target SVM model will be described in subsequent embodiments.

[0057] In some embodiments, the face image to be detected can be input into a deep learning network for eye keypoint detection to filter out the eye images to be detected. Then, the aspect ratio information of the eyes in the eye images to be detected is calculated, and the HOG features of the eye images to be detected are extracted. The aspect ratio information and HOG features are input into a target support vector machine model for classification, and the classification results are fed back to the data software interface, which can realize the detection of the open and closed state of the eyes.

[0058] The eye opening / closing state detection method provided by this invention inputs the aspect ratio information and HOG features of the eye image to be detected into a target support vector machine model for classification. This method can quickly detect the eye opening / closing state, effectively improving the eye opening / closing recognition ability of eye images under conditions such as facial expressions and uneven lighting. It also enhances the robustness of the target support vector machine model. Through particle swarm optimization algorithm, the classification ability of the target support vector machine model is improved, ensuring the effectiveness of the target support vector machine model. At the same time, the support vector machine algorithm used has low computational power consumption, ensuring that the target support vector machine model can perform eye opening / closing state detection on small industrial control computers or embedded systems with low computing power.

[0059] In some embodiments, the target support vector machine model is trained as follows:

[0060] Obtain eye image samples;

[0061] Based on the aspect ratio information of the eyes in the eye image samples, the HOG features of the eye image samples, and the open / closed state categories of the eye image samples, a feature vector dataset corresponding to the eye image samples is constructed.

[0062] The target support vector machine model is trained based on the feature vector dataset.

[0063] In practice, the target support vector machine model can be trained based on eye image samples. These eye image samples can include several eye images, which can be images of the eyes open or closed. For example... Figure 2 As shown, multiple key points can be marked in an eye image. This embodiment of the invention does not impose a specific limit on the number of key points. These key points can be used to calculate the aspect ratio information of each eye image in the eye image sample.

[0064] In some embodiments, eye image samples are obtained in the following manner:

[0065] Obtain a dataset of facial landmark images;

[0066] The facial landmark image dataset is input into a deep learning network to detect eye landmarks, resulting in an eye image dataset.

[0067] The eye image dataset is converted to grayscale to obtain eye image samples.

[0068] In practice, facial landmark datasets can be created by combining real-world work scenarios of train dispatchers or drivers. For example, videos of dispatchers or drivers at work can be acquired, and image frames containing facial information can be extracted from the videos. A preset number of video images can be collected, or the dispatchers or drivers in their work scenarios can be directly filmed to obtain a preset number of images. The preset number can be set according to actual needs; for example, 1000 images can be collected as the initial dataset. This embodiment of the invention does not impose a specific limit on the preset number. Then, facial landmark annotations are performed on the initial dataset to complete the creation of the facial landmark dataset.

[0069] After creating the facial landmark dataset, it is fed into a deep learning network for eye landmark detection, thereby selecting the eye image dataset. Then, the eye image dataset is converted to grayscale to reduce the influence of differences in lighting and eyeglass reflections, ultimately yielding eye image samples.

[0070] In some embodiments, the backbone feature extraction network of a deep learning network is a lightweight convolutional network feature extractor.

[0071] In practice, a lightweight convolutional neural network feature extractor can be used as the backbone feature extraction network of the deep learning network. For example, it can include the MobileNet V3 network. Other networks can also be selected as the backbone feature extraction network, including but not limited to the VGG (Visual Geometry Group) network, depending on the actual needs. This embodiment of the invention does not impose specific limitations on this.

[0072] In some embodiments, a facial landmark image dataset is input into a deep learning network for eye landmark detection to obtain an eye image dataset, including:

[0073] The facial landmark image dataset is input into a deep learning network for feature extraction to determine the location of the nose tip and the position of the facial bounding box.

[0074] Based on the location of the nose tip and the position of the face frame, the eye image dataset was selected.

[0075] In practice, a lightweight convolutional network feature extractor is used as the backbone feature extraction network of the deep learning network to extract features from the facial key point image dataset. This can determine the location of the nose tip and the position of the face frame. Then, the image of the eye region can be cropped to obtain the eye image. The eye image dataset can then be selected, which is the benchmark eye image dataset.

[0076] The eye opening / closing state detection method provided by this invention improves sample acquisition efficiency by using a deep learning network to obtain eye image samples.

[0077] After acquiring eye image samples, the aspect ratio of each eye image can be calculated and the HOG features of each eye image can be extracted based on the key points of the eye in the eye image samples, thereby determining the aspect ratio information of all eye image samples.

[0078] In some embodiments, the HOG features of the eye image are obtained in the following manner:

[0079] Calculate the gradient of each pixel in any eye image; divide the eye image into square blocks, and then divide each block into multiple smaller squares (cells), for example: 6*6 pixels / cell; calculate the gradient distribution histogram in each cell to obtain the descriptor of each cell; calculate the gradient direction distribution of each pixel and project it onto the histogram weighted by gradient magnitude; group several cells into a block, for example: 3*3 cells / block; concatenate the descriptors of each cell to obtain the HOG feature descriptor of the block; concatenate the descriptors of each block in the eye image to obtain the feature description of the eye image, which is the HOG feature of the eye image; when extracting the HOG feature, use the gradient operators [-1, 0, 1] and [1, 0, -1] respectively. T The gradient operator performs a convolution operation on the original eye image to obtain the gradient components in the horizontal and vertical directions, and then calculates the gradient magnitude and direction of the pixel, as shown in equations (1) and (2).

[0080] G x (x,y)=H(x+1,y)-H(x-1,y) (1)

[0081] G y (x, y)=H(x, y+1)-H(x, y-1) (2)

[0082] Among them, G x (x, y), G yH(x, y) and H(x, y) represent the horizontal gradient of pixel (x, y) in the eye image, the vertical gradient of pixel (x, y) in the eye image, and the pixel value of the eye image, respectively. The gradient magnitude of pixel (x, y) is:

[0083]

[0084] The gradient direction of pixel (x, y) is:

[0085]

[0086] Therefore, the HOG features of each eye image in the eye image sample can be extracted.

[0087] In this embodiment of the invention, a local classifier can be trained using a support vector machine (SVM) based on the HOG features of each eye image in the eye image samples.

[0088] All trained local classifiers were tested using eye image samples. Different eye images corresponded to different local classifiers. HOG features were extracted from the eye image samples for each image, and these HOG features were input into the corresponding local classifier. Based on the classification results output by each local classifier, the local classifier with a classification accuracy greater than a preset classification probability was selected as the preferred local classifier, and the corresponding eye image was recorded.

[0089] The classification results output by all preferred local classifiers are combined to form a feature vector {(x i y i )}, where y i y represents the open / closed eye state category of the i-th eye image. i =1 indicates an image sample with eyes open, y i =-1 represents a closed-eye image sample, x i =[HOG i EAR i ] T x i This represents the HOG features and eye aspect ratio (EAR) for the i-th eye image.

[0090] In practice, the feature vector {(x} of each eye image can be constructed based on the aspect ratio information of the eye image sample, the HOG features of the eye image sample, and the corresponding open / closed eye state category of the eye image sample. i y i This allows us to construct a feature vector dataset corresponding to eye image samples. Then, we can use the feature vector {(x)} to... i y iThe dataset is randomly divided into training and test sets according to a set ratio. The training and test sets are used to train the target support vector machine model.

[0091] In some embodiments, training a target support vector machine model based on a feature vector dataset includes:

[0092] Based on the particle swarm optimization algorithm, the feature vector dataset is processed to obtain the optimal parameters;

[0093] The target support vector machine model is trained based on the optimal parameters and feature vector dataset.

[0094] In actual implementation, the Particle Swarm Optimization (PSO) algorithm is initialized by setting parameters and substituting the training set from the feature vector dataset into PSO.

[0095] Randomly initialize a population of n particles. Assume the population is X = (X1, X2, ..., Xn) in a D-dimensional search space. n If a particle travels at a certain speed in space, then the position and state attribute of particle i is set as vector X. i =(x i1 x i2 x i3 , ..., x iD ) T The position X of each particle is calculated based on the objective function. i The corresponding fitness value. The velocity vector of the i-th particle is V. i =(v i1 v i2 , ..., x iD The optimal position vector of an individual is P. i =(P i1 P i2 , ..., P iD ) T And the population extrema, with the population extrema location vector being P. g =(P g1 P g2 , ..., P gD ) T Each particle updates its velocity and position using the following formula:

[0096]

[0097] Where i = 1, 2, ..., N; N is the total number of particles in this group; Vi is the velocity of the particle; P iD P represents the particle's current optimal position. gDω represents the current optimal position of all particles; k represents the inertia weight; c1 and c2 are learning factors used to adjust the maximum learning step size; r1 and r2 are random numbers distributed between (0, 1) to improve randomness.

[0098] After PSO calculation, the optimal parameters are obtained and substituted into the target support vector machine model. The support vector machine model is trained using the training set data from the feature vector dataset for eye-open / closed state detection, and the model's performance is tested using the test set data from the feature vector dataset. Finally, the optimized target support vector machine model is obtained.

[0099] The eye opening and closing state detection method provided by this invention extracts eye aspect ratio information and HOG features to detect eye state, taking into account the special application environment of railway and urban rail train dispatchers. It also applies an optimized target support vector machine model to improve the accuracy of eye opening and closing state detection and enhance the robustness of the target support vector machine model.

[0100] Figure 3 This is the second schematic flowchart of the eye opening and closing state detection method provided by the present invention. (Refer to...) Figure 3 The eye opening / closing state detection method provided by the present invention includes the following steps:

[0101] S1. Combine real-world work scenario data of railway and urban rail dispatchers, collect corresponding videos of staff, and create a dataset of facial images of railway and urban rail dispatchers.

[0102] S2. Use a deep learning network to detect key points of the human eye. Compare the positioning point of the nose tip of the face with the position of the frame line of the face to select the reference eye, that is, to extract the image of the eye to be detected.

[0103] S3. Calculate the aspect ratio information of the eye in the image to be detected;

[0104] S4. Extract the HOG features of the eye image to be detected;

[0105] S5. Establish a support vector machine model optimized by the particle swarm optimization algorithm;

[0106] S6. The trained particle swarm optimization model (PSO-SVM) is used for eye-opening and closing state detection to obtain the detection results.

[0107] In summary, the embodiments of the present invention have the following beneficial effects:

[0108] Using an optimized support vector machine model, detection accuracy can be guaranteed under the premise of low computing power and real-time requirements.

[0109] The eye opening and closing state detection device provided by the present invention will be described below. The eye opening and closing state detection device described below can be referred to in correspondence with the eye opening and closing state detection method described above.

[0110] Figure 4 This is a schematic diagram of the eye opening / closing state detection device provided by the present invention. (Refer to...) Figure 4 The eye opening and closing state detection device provided by the present invention includes: a first acquisition module 410, a first processing module 420 and a first determination module 430.

[0111] The first acquisition module 410 is used to acquire the eye image to be detected.

[0112] The first processing module 420 is used to determine the aspect ratio information of the eyes in the eye image to be detected and to extract the HOG features of the eye image to be detected.

[0113] The first determining module 430 is used to input the eye aspect ratio information and the HOG features into the target support vector machine model for classification, and determine the open / closed eye state category corresponding to the eye image to be detected.

[0114] The eye opening / closing state detection device provided by this invention quickly detects open / closed eye states by inputting the aspect ratio information and HOG features of the eye image to be detected into a target support vector machine model for classification. This effectively improves the eye opening / closing recognition capability of eye images under conditions such as facial expressions and uneven lighting, enhances the robustness of the target support vector machine model, and improves the classification capability of the target support vector machine model through particle swarm optimization algorithm, ensuring the effectiveness of the target support vector machine model. At the same time, the support vector machine algorithm used has low computing power, ensuring that the target support vector machine model can perform eye opening / closing state detection on small industrial control computers or embedded systems with low computing power.

[0115] In some embodiments, the target support vector machine model is trained in the following manner:

[0116] Obtain eye image samples;

[0117] Based on the aspect ratio information of the eyes in the eye image samples, the HOG features of the eye image samples, and the open / closed state category of the eye image samples, a feature vector dataset corresponding to the eye image samples is constructed.

[0118] The target support vector machine model is trained based on the feature vector dataset.

[0119] In some embodiments, training the target support vector machine model based on the feature vector dataset includes:

[0120] The feature vector dataset is processed using the particle swarm optimization algorithm to obtain the optimal parameters;

[0121] The target support vector machine model is trained based on the optimal parameters and the feature vector dataset.

[0122] In some embodiments, acquiring eye image samples includes:

[0123] Obtain a dataset of facial landmark images;

[0124] The facial landmark image dataset is input into a deep learning network to detect eye landmarks, thereby obtaining an eye image dataset.

[0125] The eye image dataset is converted to grayscale to obtain the eye image samples.

[0126] In some embodiments, inputting the facial landmark image dataset into a deep learning network for eye landmark detection to obtain an eye image dataset includes:

[0127] The facial key point image dataset is input into a deep learning network for feature extraction to determine the nose tip location point and the position of the face frame line;

[0128] The eye image dataset is selected based on the location of the nose tip of the face and the position of the line of the face frame.

[0129] In some embodiments, the backbone feature extraction network of the deep learning network is a lightweight convolutional network feature extractor.

[0130] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an eye opening / closing state detection method, which includes:

[0131] Acquire the image of the eye to be detected

[0132] Determine the aspect ratio information of the eye in the image to be detected, and extract the HOG features of the eye image to be detected;

[0133] The eye aspect ratio information and the HOG features are input into the target support vector machine model for classification to determine the open / closed eye state category of the eye image to be detected. The target support vector machine model is a support vector machine model optimized by particle swarm optimization algorithm.

[0134] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the eye opening and closing state detection method provided by the above methods, the method comprising:

[0136] Acquire the image of the eye to be detected

[0137] Determine the aspect ratio information of the eye in the image to be detected, and extract the HOG features of the eye image to be detected;

[0138] The eye aspect ratio information and the HOG features are input into the target support vector machine model for classification to determine the open / closed eye state category of the eye image to be detected. The target support vector machine model is a support vector machine model optimized by particle swarm optimization algorithm.

[0139] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the eye opening / closing state detection method provided by the above methods, the method comprising:

[0140] Acquire the image of the eye to be detected

[0141] Determine the aspect ratio information of the eye in the image to be detected, and extract the HOG features of the eye image to be detected;

[0142] The eye aspect ratio information and the HOG features are input into the target support vector machine model for classification to determine the open / closed eye state category of the eye image to be detected. The target support vector machine model is a support vector machine model optimized by particle swarm optimization algorithm.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting eye opening and closing status, characterized in that, include: Acquire the image of the eye to be detected Determine the aspect ratio information of the eye in the image to be detected, and extract the HOG features of the eye image to be detected; The eye aspect ratio information and the HOG features are input into a target support vector machine model for classification to determine the open / closed eye state category of the eye image to be detected. The target support vector machine model is a support vector machine model optimized by particle swarm optimization. The target support vector machine model is trained in the following manner: Obtain eye image samples; based on the eye aspect ratio information of the eye image samples, the HOG features of the eye image samples, and the open / closed eye state category corresponding to the eye image samples, construct the feature vector dataset corresponding to the eye image samples; Based on the feature vector dataset, train the target support vector machine model; Training the target support vector machine model based on the feature vector dataset includes: The feature vector dataset is processed using the particle swarm optimization algorithm to obtain optimal parameters; the target support vector machine model is then trained based on the optimal parameters and the feature vector dataset. The extraction of HOG features from the eye image to be detected includes: Calculate the gradient of each pixel in the eye image to be detected; divide the eye image to be detected into square intervals, and then divide each square interval into multiple smaller squares; calculate the gradient distribution histogram in each smaller square to obtain the descriptor of each smaller square; calculate the gradient direction distribution of each pixel and project it onto the histogram according to the gradient magnitude; combine multiple smaller squares into a square interval; concatenate the descriptors of each smaller square to obtain the HOG feature descriptor of the square interval; concatenate the descriptors of each square interval in the eye image to obtain the HOG feature of the eye image to be detected. Specifically, when extracting HOG features, gradient operators of [-1, 0, 1] and [1, 0, -1] are used respectively. T The gradient operator performs a convolution operation on the original eye image to obtain gradient components in the horizontal and vertical directions; the eye image to be detected is divided into 6... 6-pixel squares, 3 Three small squares form a square interval.

2. The method for detecting eye opening and closing state according to claim 1, characterized in that, The acquisition of eye image samples includes: Obtain a dataset of facial landmark images; The facial landmark image dataset is input into a deep learning network to detect eye landmarks, thereby obtaining an eye image dataset. The eye image dataset is converted to grayscale to obtain the eye image samples.

3. The method for detecting eye opening and closing state according to claim 2, characterized in that, The step of inputting the facial landmark image dataset into a deep learning network to perform eye landmark detection, resulting in an eye image dataset, includes: The facial key point image dataset is input into a deep learning network for feature extraction to determine the nose tip location point and the position of the face frame line; The eye image dataset is selected based on the location of the nose tip of the face and the position of the line of the face frame.

4. The method for detecting eye opening and closing state according to claim 2 or 3, characterized in that, The backbone feature extraction network of the deep learning network is a lightweight convolutional network feature extractor.

5. A device for detecting eye opening and closing status, characterized in that, include: The first acquisition module is used to acquire the image of the eye to be detected. The first processing module is used to determine the aspect ratio information of the eyes in the eye image to be detected and to extract the HOG features of the eye image to be detected. The first determining module is used to input the eye aspect ratio information and the HOG features into a target support vector machine model for classification, thereby determining the open / closed eye state category corresponding to the eye image to be detected; the target support vector machine model is trained in the following manner: Obtain eye image samples; based on the eye aspect ratio information of the eye image samples, the HOG features of the eye image samples, and the open / closed eye state category corresponding to the eye image samples, construct the feature vector dataset corresponding to the eye image samples; Based on the feature vector dataset, train the target support vector machine model; Training the target support vector machine model based on the feature vector dataset includes: The feature vector dataset is processed using the particle swarm optimization algorithm to obtain optimal parameters; the target support vector machine model is then trained based on the optimal parameters and the feature vector dataset. The first processing module is specifically used for: Calculate the gradient of each pixel in the eye image to be detected; divide the eye image to be detected into square intervals, and then divide each square interval into multiple smaller squares; calculate the gradient distribution histogram in each smaller square to obtain the descriptor of each smaller square; calculate the gradient direction distribution of each pixel and project it onto the histogram according to the gradient magnitude; combine multiple smaller squares into a square interval; concatenate the descriptors of each smaller square to obtain the HOG feature descriptor of the square interval; concatenate the descriptors of each square interval in the eye image to obtain the HOG feature of the eye image to be detected. Specifically, when extracting HOG features, gradient operators of [-1, 0, 1] and [1, 0, -1] are used respectively. T The gradient operator performs a convolution operation on the original eye image to obtain gradient components in the horizontal and vertical directions; the eye image to be detected is divided into 6... 6-pixel squares, 3 Three small squares form a square interval.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the eye opening and closing state detection method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the eye opening and closing state detection method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the eye opening and closing state detection method as described in any one of claims 1 to 4.

Images