AD scale eye closing identification method based on hourglass network structure
Through the deep learning method of hourglass network structure, combined with multi-scale directional attention model and local feature extraction technology, the eye key points of elderly patients are accurately positioned, and the traditional methods of low accuracy and poor robustness of eye-closing detection in middle-aged and elderly people are solved, and efficient eye-closing status recognition is achieved, supporting early screening and cognitive function evaluation of Alzheimer's disease.
Patent Information
- Application Number
- CN202510223813.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-27
- Publication Date
- 2025-07-04
AI Technical Summary
Traditional eye-closing motion detection methods are low in accuracy and poor in the elderly population. Especially when faced with problems such as light changes, facial posture and eyelid sagging, the accuracy is insufficient, making it difficult to meet the needs of early screening and cognitive function assessment of Alzheimer's disease.
Deep learning method based on hourglass network structure is adopted, combined with multi-scale directional attention model and local feature extraction technology, by accurately positioning the eye key points, calculating the opening and closing degree of the eyes, and identifying the eye-closing state of elderly patients.
It significantly reduces the false detection rate and missed detection rate, improves the accuracy of eye-closing detection in the elderly, and provides reliable and low-cost technical support for early screening and cognitive function assessment of Alzheimer's disease.
Smart Images
Figure CN120260105A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of closed-eye action detection, and particularly relates to a method for recognizing closed eyes in the AD scale based on the hourglass network structure. Background Art
[0002] With the acceleration of the global aging process, the proportion of the elderly population has increased significantly. Alzheimer's disease (AD), as the most common cognitive impairment disease among the elderly, has imposed a serious burden on the health, family and society of patients. Early screening and intervention are the keys to slowing down the progression of Alzheimer's disease, and neuropsychological scales are important tools currently used for cognitive function assessment. However, in the actual application of these scales, traditional assessment methods often rely on the subjective judgment of doctors, such as observing whether the patient can complete the closed-eye action according to the instructions. This assessment based on subjective observation has problems of inconsistent standards and insufficient accuracy, and is prone to a decline in diagnostic efficiency in long-term clinical assessments.
[0003] The closed-eye action detection technology has been widely applied in fields such as driver monitoring systems and biometric recognition. The current technologies can be divided into two categories: those based on physiological characteristics and those based on facial characteristics. The methods based on physiological characteristics rely on invasive sensors to monitor electrooculogram or electroencephalogram signals. Although they have high precision, the equipment is expensive and the operation is complex. The methods based on facial characteristics rely on computer vision and deep learning technologies, which have advantages such as low cost and non-contact, but they are insufficiently adaptable to problems such as changes in illumination, facial postures, and the specific eyelid relaxation of the elderly in complex scenes. In addition, most of the existing deep learning methods are designed for the general population and do not consider the particularities of the elderly in appearance characteristics and action patterns, such as low blink frequency, small action amplitude, and eyelid ptosis, resulting in poor accuracy and robustness in the closed-eye detection of the elderly. Summary of the Invention
[0004] The purpose of the present invention is to provide a method for recognizing closed eyes in the AD scale based on the hourglass network structure, including the following steps:
[0005] 1) Obtain the face image of the elderly patient to be recognized.
[0006] 2) Locate the key points at the eyes of the elderly patient on the face image of the elderly patient to be recognized.
[0007] 3) Calculate the degree of opening and closing of the eyes of the elderly patient based on the located key points.
[0008] 4) Analyze the eye state of the elderly patient through the degree of opening and closing of the eyes of the elderly patient to recognize the closed-eye state of the elderly patient.
[0009] Further, the face image of the elderly patient to be recognized is obtained by performing face detection on the images in the video through the target detection model Pyramid Box-Lite.
[0010] The target detection model Pyramid Box-Lite includes a scale-fair backbone layer, a low feature pyramid layer, a context-sensitive prediction layer, and a PyramidBox loss layer.
[0011] The scale-fair backbone layer is used to extract multi-scale features of the input image and output a multi-scale feature map.
[0012] The low feature pyramid layer adopts a feature fusion mechanism for the multi-scale feature map to generate an enhanced low-level feature map.
[0013] The context-sensitive prediction layer extracts context features from the enhanced low-level feature map and generates a context-sensitive feature map.
[0014] The PyramidBox loss layer predicts face bounding boxes, face key points, and context information for the context-sensitive feature map.
[0015] Further, the key points at the eyes of the elderly patient include key points located at the inner canthus, outer canthus, upper eyelid, and lower eyelid.
[0016] Further, the number of key points located at the upper eyelid and the lower eyelid is the same.
[0017] Further, in step 2), the steps of locating the key points at the eyes of the elderly patient are as follows:
[0018] Input the face image of the elderly patient to be recognized into a convolutional neural network model based on the hourglass network to locate the key points at the eyes of the elderly patient.
[0019] Further, the construction steps of the convolutional neural network model based on the hourglass network are as follows:
[0020] s1 Divide the historical face images of the elderly patients into several tiles.
[0021] s2 Generate a spatial attention matrix according to the spatial distance between each tile and the key points.
[0022] s3 Use a 3*3 convolutional kernel to extract the eye contours in each tile to generate an orientation attention matrix.
[0023] s4 Combine the spatial attention matrix and the orientation attention matrix to obtain a self-attention matrix.
[0024] s5 Based on the self-attention matrix, construct a convolutional neural network model based on the hourglass network.
[0025] The loss function of the convolutional neural network model based on the hourglass network is as follows:
[0026]
[0027] In the formula, L is the total loss function. n is the hourglass module index, and N is the total number of hourglass modules. P n,ij is the predicted value output by the nth layer hourglass module. T n,ij is the target value corresponding to the nth layer. i and j are the height and width indices respectively, H and W are the height and width of each output, and λ n is the weighting coefficient.
[0028] The parameter update formula of the convolutional neural network model based on the hourglass network is as follows:
[0029]
[0030] In the formula, t represents time, and θ t+1 , θ t represent the parameters at times t + 1 and t respectively. α is the learning rate.
[0031] Furthermore, the eye states of the elderly patients include the open-eye state, the closed-eye state, and the blinking state.
[0032] Furthermore, the degree of eye opening and closing of the elderly patients is calculated through the EAR value.
[0033] Among them, the EAR value of one eye is as follows:
[0034]
[0035] In the formula, EAR is the eye aspect ratio, and both p2 and p3 are the positions of the key points of the upper eyelid. p1 and p4 are the positions of the key points of the inner and outer eye corners respectively. p5 and p6 are the positions of the key points of the lower eyelid.
[0036] Furthermore, when the EAR value is less than the preset threshold, the elderly patient is in the closed-eye state.
[0037] The technical effect of the present invention is beyond doubt. The present invention proposes a method for identifying closed eyes of the elderly based on the hourglass network structure. This method optimizes the positioning and recognition accuracy of the eye features of the elderly by combining deep learning, multi-scale directional attention model, and local feature extraction technology. By accurately positioning the key points of the eyes and introducing the shape factor, it effectively solves the problem of insufficient detection accuracy caused by appearance changes and action characteristic differences in traditional methods, providing reliable technical support for the early screening and intervention of cognitive impairment.
[0038] The present invention aims to solve the problems of low accuracy and poor robustness of traditional blink detection methods in the elderly population, and proposes a closed-eye detection method based on the hourglass network structure. By combining deep learning, attention mechanism and adaptive ambiguity reduction technology, the detection system is optimized from multiple aspects such as architecture design, feature extraction and detection process to meet the clinical needs of cognitive function assessment of the elderly.
[0039] By dividing the facial image into multiple small blocks and dynamically adjusting the attention degree according to the distance between each block and the key points of the eyes, precise focusing on the eye region is achieved. This adaptive adjustment based on the attention mechanism in the present invention enables the algorithm to highlight the key regions around the eyes in the case of complex backgrounds and irregular edges, especially suitable for the blurred or changing edge information in the facial images of the elderly. Secondly, in order to better capture the eye contour and optimize edge extraction, the present invention designs a multi-angle convolutional kernel edge detection technology. This technology can effectively cope with factors such as large facial expression changes and skin relaxation in the elderly, ensuring the accurate extraction of the eye contour. At the same time, by combining spatial attention and direction attention information, a comprehensive scoring matrix is generated, enabling the algorithm to not only effectively perceive the details around the eyes but also accurately capture the contour information of the eyes.
[0040] The present invention optimizes the adaptability to problems such as illumination changes, facial pose differences and eyelid ptosis by accurately locating the key points of the eyes, significantly reducing the false detection rate and missed detection rate. This method can efficiently identify the unique motion characteristics of the elderly, providing reliable and low-cost technical support for the early screening and cognitive function assessment of Alzheimer's disease, and having important clinical application value and social significance. Brief Description of the Drawings
[0041] Figure 1 is the overall flowchart of the algorithm of the present invention;
[0042] Figure 2 is a flowchart of generating the direction attention matrix of the present invention;
[0043] Figure 3 is the hourglass network convolution model diagram of the present invention;
[0044] Figure 4 is the schematic diagram of the Pyramid Box-Lite structure of the object detection model;
[0045] Figure 5 is the schematic diagram of the structure of the convolutional neural network model based on the hourglass network;
[0046] Figure 6 is the schematic diagram of the key point positions of the EAR value. Detailed Implementation Manner
[0047] The present invention will be further described below in conjunction with embodiments, but it should not be understood that the above-mentioned subject matter scope of the present invention is limited to the following embodiments. Without departing from the above technical idea of the present invention, various substitutions and changes made according to ordinary technical knowledge and customary means in the art should be included within the protection scope of the present invention.
[0048] Embodiment 1:
[0049] See Figures 1 to 6 , a method for identifying closed eyes in an AD scale based on a sandglass network structure, comprising the following steps:
[0050] 1) Obtain a face image of an elderly patient to be identified.
[0051] 2) Locate the key points at the eyes of the elderly patient on the face image of the elderly patient to be identified.
[0052] 3) Calculate the degree of eye opening and closing of the elderly patient based on the located key points.
[0053] 4) Analyze the eye state of the elderly patient through the degree of eye opening and closing of the elderly patient, and identify the closed-eye state of the elderly patient.
[0054] Embodiment 2:
[0055] A method for identifying closed eyes in an AD scale based on a sandglass network structure, the main technical content is as in Embodiment 1. Further, the face image of the elderly patient to be identified is obtained by performing face detection on the images in the video through the target detection model Pyramid Box-Lite.
[0056] The target detection model Pyramid Box-Lite includes a scale-fair backbone layer, a low feature pyramid layer, a context-sensitive prediction layer, and a PyramidBox loss layer.
[0057] The scale-fair backbone layer is used to extract multi-scale features of the input image and output a multi-scale feature map.
[0058] The low feature pyramid layer adopts a feature fusion mechanism for the multi-scale feature map to generate an enhanced low-level feature map.
[0059] The context-sensitive prediction layer extracts context features from the enhanced low-level feature map and generates a context-sensitive feature map.
[0060] The PyramidBox loss layer predicts face boxes, face key points, and context information for the context-sensitive feature map.
[0061] Embodiment 3:
[0062] A method for recognizing closed eyes in the AD scale based on the hourglass network structure. The main technical content can be found in any one of Embodiments 1 to 2. Further, the key points at the eyes of the elderly patients include the key points located at the inner canthus, outer canthus, upper eyelid, and lower eyelid.
[0063] Embodiment 4:
[0064] A method for recognizing closed eyes in the AD scale based on the hourglass network structure. The main technical content can be found in any one of Embodiments 1 to 3. Further, the number of key points located at the upper eyelid and the lower eyelid is the same.
[0065] Embodiment 5:
[0066] A method for recognizing closed eyes in the AD scale based on the hourglass network structure. The main technical content can be found in any one of Embodiments 1 to 4. Further, in step 2), the steps of locating the key points at the eyes of the elderly patients are as follows:
[0067] Input the face image of the elderly patient to be recognized into the convolutional neural network model based on the hourglass network to locate the key points at the eyes of the elderly patient.
[0068] Embodiment 6:
[0069] A method for recognizing closed eyes in the AD scale based on the hourglass network structure. The main technical content can be found in any one of Embodiments 1 to 5. Further, the steps of constructing the convolutional neural network model based on the hourglass network are as follows:
[0070] s1 Divide the historical face images of the elderly patients into several patches.
[0071] s2 Generate a spatial attention matrix according to the spatial distance between each patch and the key points.
[0072] s3 Use a 3*3 convolutional kernel to extract the eye contour in each patch to generate a direction attention matrix.
[0073] s4 Combine the spatial attention matrix and the direction attention matrix to obtain a self-attention matrix.
[0074] s5 Based on the self-attention matrix, construct a convolutional neural network model based on the hourglass network.
[0075] The loss function of the convolutional neural network model based on the hourglass network is as follows:
[0076]
[0077] In the formula, L is the total loss function. n is the hourglass module index, and N is the total number of hourglass modules. P n,ij is the predicted value output by the nth layer hourglass module. T n,ijis the target value corresponding to the nth layer. i and j are the height and width indices respectively, H and W are the height and width of each output, and λ n is the weighting coefficient.
[0078] The parameter update formula of the convolutional neural network model based on the hourglass network is as follows:
[0079]
[0080] In the formula, t represents time, and θ t+1 , θ t represent the parameters at times t + 1 and t respectively. α is the learning rate.
[0081] Example 7:
[0082] A method for identifying closed eyes in the AD scale based on the hourglass network structure. The main technical content can be found in any one of Examples 1 to 6. Further, the eye states of the elderly patients include open eye state, closed eye state, and blinking state.
[0083] Example 8:
[0084] A method for identifying closed eyes in the AD scale based on the hourglass network structure. The main technical content can be found in any one of Examples 1 to 7. Further, the degree of eye opening and closing of the elderly patients is calculated through the EAR value.
[0085] Among them, the EAR value of one eye is as follows:
[0086]
[0087] In the formula, EAR is the eye aspect ratio, and p2 and p3 are the positions of the key points on the upper eyelid. p1 and p4 are the positions of the key points at the inner and outer corners of the eye respectively. p5 and p6 are the positions of the key points on the lower eyelid.
[0088] Example 9:
[0089] A method for identifying closed eyes in the AD scale based on the hourglass network structure. The main technical content can be found in any one of Examples 1 to 8. Further, when the EAR value is less than the preset threshold, the elderly patient is in the closed eye state.
[0090] Example 10:
[0091] See Figures 1 to 6 , a method for identifying closed eyes in the AD scale based on the hourglass network structure, includes the following steps:
[0092] 1) Obtain the face image of the elderly patient to be identified.
[0093] 2) Locate 12 key points at the eyes of the elderly patient on the face image of the elderly patient to be identified.
[0094] 3) Calculate the opening degree of the elderly patient's eyes based on the positioning key points.
[0095] 4) Analyze the eye state of the elderly patient through the opening degree of the elderly patient's eyes, and identify the closed-eye state of the elderly patient.
[0096] Example 11:
[0097] A method for identifying closed eyes of the AD scale based on the hourglass network structure. The main technical content is shown in Example 10. Further, the face image of the elderly patient to be identified is obtained by performing face detection on the images in the video through the target detection model Pyramid Box-Lite of Baidu.
[0098] The target detection model Pyramid Box-Lite includes a scale-fair backbone layer, a low feature pyramid layer, a context-sensitive prediction layer, and a PyramidBox loss layer.
[0099] The scale-fair backbone layer is used to extract multi-scale features of the input image and output a multi-scale feature map.
[0100] The low feature pyramid layer adopts a feature fusion mechanism for the multi-scale feature map to generate an enhanced low-level feature map.
[0101] The context-sensitive prediction layer extracts context features from the enhanced low-level feature map and generates a context-sensitive feature map.
[0102] The PyramidBox loss layer predicts the face box, face key points, and context information for the context-sensitive feature map.
[0103] Example 12:
[0104] A method for identifying closed eyes of the AD scale based on the hourglass network structure. The main technical content is shown in any one of Examples 10 to 11. Further, the key points at the eyes of the elderly patient include the key points located at the inner canthus, outer canthus, upper eyelid, and lower eyelid.
[0105] During the annotation process of a large amount of face image data, professional annotators will perform key point annotation on the upper and lower eyelids of the eyes in each image according to the above anatomical and visual feature principles. Taking the eyeball as the central reference, the position where the outermost side of the upper eyelid meets the orbital margin is set as a key point, usually the 37th point. Then, along the arc of the upper eyelid, the 38th point is selected at a position approximately in the middle of the upper eyelid close to the eyeball. This point is generally in the area where the natural curvature of the upper eyelid is most obvious and can better reflect the curvature and position of the upper eyelid.
[0106] Key points for the right eye: Points from the 36th to the 41st are used to determine the right eye, specifically:
[0107] Corner points of the eye: The 36th and 39th points are the outer and inner corner points of the right eye respectively.
[0108] Upper eyelid points: The 37th and 38th points are located in the outer and middle regions of the upper eyelid of the right eye.
[0109] Lower eyelid points: The 40th and 41st points are located in the outer and middle regions of the lower eyelid of the right eye.
[0110] Key points for the left eye: Points from the 42nd to the 47th are used to determine the left eye, specifically:
[0111] Corner points of the eye: The 42nd and 45th points are the outer and inner corner points of the left eye respectively.
[0112] Upper eyelid points: The 43rd and 44th points are located in the outer and middle regions of the upper eyelid of the left eye.
[0113] Lower eyelid points: The 46th and 47th points are located in the outer and middle regions of the lower eyelid of the left eye.
[0114] Embodiment 13:
[0115] An AD scale closed-eye recognition method based on the hourglass network structure, the main technical content is shown in any one of Embodiments 10 to 12. Further, the number of key points located on the upper eyelid and the lower eyelid is the same.
[0116] Embodiment 14:
[0117] An AD scale closed-eye recognition method based on the hourglass network structure, the main technical content is shown in any one of Embodiments 10 to 13. Further, in step 2), the steps of locating the key points at the eyes of the elderly patient are:
[0118] Input the face image of the elderly patient to be recognized into the convolutional neural network model based on the hourglass network to locate the key points at the eyes of the elderly patient.
[0119] After inputting the whole image, the image block can be located through the program.
[0120] Embodiment 15:
[0121] An AD scale closed-eye recognition method based on the hourglass network structure, the main technical content is shown in any one of Embodiments 10 to 14. Further, the construction steps of the convolutional neural network model based on the hourglass network are as follows:
[0122] s1 Divide the historical face images of elderly patients into several image blocks.
[0123] s2 Generate a spatial attention matrix based on the spatial distance between each patch and the key points. By calculating the distance from each region in the image to the key points of the eyes, the attention degree of different regions can be dynamically adjusted, so as to capture the detailed information around the eyes more accurately.
[0124] s3 Use a 3*3 convolutional kernel to extract the eye contour in each patch, generating an orientation attention matrix, which is used to guide the network to focus more on capturing the eye edges and avoid ignoring those key and irregular eye regions during the processing.
[0125] s4 Combine the spatial attention matrix and the orientation attention matrix to obtain a self-attention matrix.
[0126] s5 Based on the self-attention matrix, construct a convolutional neural network model based on the hourglass network. At each stage of the network, combine the aforementioned spatial attention and orientation attention mechanisms to enhance the sensitivity of the model to eye features and achieve the accurate localization of key points of the eyes of the elderly.
[0127] Input of the convolutional neural network model based on the hourglass network: Assume the input image is I ∈ R H×W×C , where H, W, and C represent the height, width, and number of channels of the image respectively. Usually, preprocessing operations such as normalizing the input image are performed to convert it into a format acceptable to the model.
[0128] Output: The output of the model is usually a heatmap M ∈ R H′×W′×K , where H and W are the height and width of the output heatmap, and C is the number of key point categories to be predicted. Each channel corresponds to a heatmap of a key point, and each pixel value in the heatmap represents the probability that the position belongs to the corresponding key point.
[0129] The loss function of the convolutional neural network model based on the hourglass network is as follows:
[0130]
[0131] In the formula, L is the total loss function. n is the hourglass module index, and N is the total number of hourglass modules. P n,ij is the predicted value output by the nth layer hourglass module. T n,ij is the target value corresponding to the nth layer. i and j are the height and width indices respectively, H and W are the height and width of each output, and λ n is the weighting coefficient, which is used to adjust the contribution of losses at different levels to the total loss.
[0132] The parameters of the model are updated using Stochastic Gradient Descent (SGD) and its variant algorithms. The parameter update formula for the convolutional neural network model based on the hourglass network is as follows:
[0133]
[0134] In the formula, t represents time, and θ t+1 and θ t represent the parameters at times t + 1 and t respectively. α is the learning rate.
[0135] Example 16:
[0136] An AD scale closed-eye recognition method based on the hourglass network structure. The main technical content can be found in any one of Examples 10 to 15. Further, the eye states of the elderly patients include the open-eye state, the closed-eye state, and the blinking state.
[0137] Example 17:
[0138] An AD scale closed-eye recognition method based on the hourglass network structure. The main technical content can be found in any one of Examples 10 to 16. Further, the degree of eye opening and closing of the elderly patients is calculated through the EAR value.
[0139] Among them, the EAR value of one eye is as follows:
[0140]
[0141] In the formula, EAR is the eye aspect ratio, and p2 and p3 are the positions of the key points on the upper eyelid. p1 and p4 are the positions of the key points at the inner and outer corners of the eye respectively. p5 and p6 are the positions of the key points on the lower eyelid.
[0142] Example 18:
[0143] An AD scale closed-eye recognition method based on the hourglass network structure. The main technical content can be found in any one of Examples 10 to 17. Further, when the EAR value is less than the preset threshold, the elderly patient is in the closed-eye state.
[0144] The preset threshold generally takes values from 0.1 to 0.15.
[0145] Example 19:
[0146] A system applying the method described in any one of Examples 10 - 18 includes an image acquisition module, a positioning module, an opening and closing degree calculation module, and an eye state recognition module.
[0147] The image acquisition module is used to acquire the face image of the elderly patient to be recognized.
[0148] The positioning module is used to locate the key points at the eyes of the elderly patient on the face image of the elderly patient to be recognized.
[0149] The opening degree calculation module calculates the opening degree of the eyes of the elderly patient based on the located key points.
[0150] The eye state recognition module analyzes the eye state of the elderly patient through the opening degree of the eyes of the elderly patient and recognizes the closed-eye state of the elderly patient.
[0151] Embodiment 20:
[0152] See Figures 1 to 6 , a method for recognizing closed eyes of the AD scale based on the hourglass network structure, comprising the following steps:
[0153] S1. Extract the face in the image frame based on the video consultation data of the elderly patient;
[0154] S2. Accurately locate 12 key points of the patient's eyes based on the face image of the elderly patient, including the key points at the corners of the eyes and the upper and lower eyelids;
[0155] S3. Propose an adaptive closed-eye recognition method based on the eye opening degree.
[0156] Among them, S1 includes the following steps:
[0157] S101. Use Baidu's object detection model Pyramid Box-Lite to perform face detection on the images in the video and extract the patient's face from them.
[0158] S2 includes the following steps:
[0159] S201. As Figure 2 shown, divide the entire facial image into several small block images, and generate a spatial attention matrix according to the spatial distance between each small block and the key points of the eyes. By calculating the distances from each region in the image to the key points of the eyes, the attention degree of different regions can be dynamically adjusted, so as to more accurately capture the detailed information around the eyes.
[0160] S202. Use a 3*3 convolutional kernel to extract the eye contour, and at the same time generate a direction attention matrix, which is used to guide the network to focus more on capturing the eye edges and avoid ignoring those key and irregular eye regions during the processing.
[0161] S203. Combine the generated spatial attention matrix and the direction attention matrix and integrate them into the self-attention mechanism.
[0162] S204. Construct a convolutional neural network model based on the hourglass network. At each stage of the network, combine the aforementioned spatial attention and orientation attention mechanisms to enhance the sensitivity of the model to eye features and achieve accurate localization of key points of the eyes of the elderly.
[0163] S3 includes the following steps:
[0164] S301. Calculate the eye opening degree using 12 key points of the eyes.
[0165] S302. Apply a sliding window algorithm to the video sequence to analyze the EAR values in consecutive frames. Through a preset threshold, classify the eye state of each frame image as open, closed, or blinking state.
Claims
1. An AD scale closed-eye recognition method based on the hourglass network structure, characterized in that, It includes the following steps: 1) Obtain the face image of the elderly patient to be recognized. 2) Locate the key points at the eyes of the elderly patient on the face image of the elderly patient to be recognized; 3) Calculate the opening degree of the eyes of the elderly patient based on the located key points. 4) Analyze the eye state of the elderly patient through the opening degree of the eyes of the elderly patient, and recognize the closed-eye state of the elderly patient.
2. The method for recognizing closed eyes of the AD scale based on the hourglass network structure according to claim 1, characterized in that, The face image of the elderly patient to be recognized is obtained by performing face detection on the images in the video through the object detection model Pyramid Box-Lite; The object detection model Pyramid Box-Lite includes a scale-fair backbone layer, a low feature pyramid layer, a context-sensitive prediction layer, and a PyramidBox loss layer; The scale-fair backbone layer is used to extract multi-scale features of the input image and output a multi-scale feature map; The low feature pyramid layer adopts a feature fusion mechanism for the multi-scale feature map to generate an enhanced low-level feature map; The context-sensitive prediction layer extracts context features from the enhanced low-level feature map and generates a context-sensitive feature map; The PyramidBox loss layer predicts face boxes, face key points, and context information for the context-sensitive feature map.
3. The method for identifying closed eyes of the AD scale based on the hourglass network structure according to claim 1, characterized in that The key points at the eyes of the elderly patient include the key points located at the inner canthus, outer canthus, upper eyelid, and lower eyelid.
4. The AD scale closed-eye recognition method based on the hourglass network structure according to claim 3, characterized in that The number of key points located on the upper eyelid and the lower eyelid is the same.
5. The method for identifying closed eyes of the AD scale based on the hourglass network structure according to claim 1, characterized in that, In step 2), the steps for locating the key points at the eyes of the elderly patient are as follows: Input the face image of the elderly patient to be recognized into a convolutional neural network model based on the hourglass network to locate the key points at the eyes of the elderly patient.
6. The AD scale closed-eye recognition method based on the hourglass network structure according to claim 5, characterized in that, The construction steps of the convolutional neural network model based on the hourglass network are as follows: s1 Divide the historical face images of the elderly patients into several patches; s2 Generate a spatial attention matrix according to the spatial distance between each patch and the key points; s3 Use a 3*3 convolution kernel to extract the eye contour in each patch to generate a direction attention matrix; s4 Combine the spatial attention matrix and the direction attention matrix to obtain a self-attention matrix; s5 Based on the self-attention matrix, construct a convolutional neural network model based on the hourglass network; The loss function of the convolutional neural network model based on the hourglass network is as follows: where \(L\) is the total loss function; \(n\) is the hourglass module index, \(N\) is the total number of hourglass modules; \(P\) n,ij is the predicted value output by the \(n\)-th layer hourglass module; \(T\) n,ij is the target value corresponding to the \(n\)-th layer; \(i\) and \(j\) are the height and width indices respectively, \(H\) and \(W\) are the height and width of each output, and \(\lambda\) n is the weighting coefficient; The parameter update formula of the convolutional neural network model based on the hourglass network is as follows: where t represents time, and θ t+1 , θ t represent the parameters at times t + 1 and t respectively; α is the learning rate.
7. A method for identifying closed eyes of the AD scale based on the hourglass network structure according to claim 1, characterized in that The eye states of the elderly patient include an open-eye state, a closed-eye state, and a blinking state.
8. A method for identifying closed eyes in an AD scale based on an hourglass network structure according to claim 1, characterized in that, The opening degree of the eyes of the elderly patient is calculated through the EAR value; Among them, the EAR value of one eye is as follows: In the formula, EAR is the eye aspect ratio, p2 and p3 are the positions of the key points on the upper eyelid; p1 and p4 are the positions of the key points at the inner and outer canthi respectively; p5 and p6 are the positions of the key points on the lower eyelid.
9. The method for identifying closed eyes of the AD scale based on the hourglass network structure according to claim 8, characterized in that, When the EAR value is less than the preset threshold, the elderly patient is in a closed-eye state.