A Patient Attention Determination Method Based on Information Entropy in a Meta-Clinic Scenario

By employing the information entropy method in a meta-clinic setting, combined with RetinaFace and residual network models, the entropy values ​​of patients' head posture and eye movement are calculated. This solves the problem of accuracy in judging patients' attention levels and improves the efficiency and quality of mental and psychological state assessment.

CN117218154BActive Publication Date: 2026-03-06HUNAN TRASEN SCI & TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311067313.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-08-23
Publication Date
2026-03-06
Estimated Expiration
2043-08-23

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately determine a patient's level of focus in a meta-clinic setting, impacting the accuracy and efficiency of mental and psychological state assessments.

Method used

Using an information entropy-based approach, this study employs face detection, head pose estimation, and eye tracking. By utilizing RetinaFace network and residual network models, the Euler angles of the patient's head pose and the entropy values ​​of the eye movement state are calculated to quantify the patient's level of focus.

Benefits of technology

It improves the accuracy and real-time nature of patient focus assessment, reduces patient consultation costs, and enhances the closeness of doctor-patient interaction and the quality of medical care.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117218154B_ABST
    Figure CN117218154B_ABST
Patent Text Reader

Abstract

This invention provides a method for judging patient attention based on information entropy in a meta-clinic setting, comprising the following steps: detecting a face using a face detection module, outputting the face position and binocular coordinates; extracting binocular images using the coordinates, predicting pupil positions using a trained residual network model, calculating the vertical direction of eye gaze based on the position coordinates, and determining the difference between the vertical direction and the central axis as the eye movement state value; calculating the Euler angles of the patient's head posture, and calculating the standard deviation and rate of change of the Euler angle components; substituting the eye movement state value, the standard deviation of the head posture, and the rate of change into the information entropy formula to calculate the entropy value, inverting the entropy value to obtain the attention level; comparing the attention level with a set threshold to determine the patient's attention level and providing corresponding prompts or suggestions. This invention can objectively assess the patient's mental and psychological state and attention level, thereby providing doctors with more effective diagnostic and treatment references. It is applicable to patients receiving remote medical services in a meta-clinic setting.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of remote intelligent diagnostic technology, specifically to a method for judging patient attention based on information entropy in a meta-clinic setting. Background Technology

[0002] In modern medicine, mental and psychological disorders have become a major social concern. With an aging population and a faster pace of life, various stressors and emotional problems have become daily challenges. Mental and psychological status assessment is a crucial issue in the medical field, used to aid in the diagnosis of mental illnesses such as depression and anxiety. Attention assessment is an important reference for evaluating a patient's mental and psychological condition. Therefore, research into techniques for assessing patient attention can help improve the quality of medical care, promote emotional communication between doctors and patients, and assist psychiatrists in better diagnosing and treating patients.

[0003] The meta-clinic is a new type of medical scenario that utilizes high-tech equipment and information technology to generate a virtual doctor-patient interactive treatment environment. It concentrates medical resources in a single room, providing a one-stop medical service. A meta-clinic typically includes multiple functional areas such as a doctor's workstation, patient bed, diagnostic equipment, and information technology equipment. Doctors can view patient medical records, monitor physiological signals, and perform diagnoses and treatments at their workstations; patients can receive treatment and have their physiological signals monitored at their bedsides; diagnostic equipment includes various medical devices such as electrocardiographs, blood pressure monitors, and thermometers; and information technology equipment includes video surveillance equipment, telemedicine devices, and intelligent diagnostic devices, enabling remote medical care and information sharing between doctors and patients. The meta-clinic can also provide more personalized medical services, fostering closer interaction between patients and doctors. The design concept of the meta-clinic aims to improve medical efficiency, reduce medical costs, enhance the medical experience, and also provides new ideas for the development of medical technology. Therefore, in the context of a meta-clinic, patient focus discrimination method based on information entropy can improve the accuracy and efficiency of mental health assessment, providing better medical services for patients.

[0004] In the context of a meta-clinic, during doctor-patient interactions, patient focus can be assessed based on head posture and eye movements. This helps doctors respond to appropriate stimuli, ultimately aiding in the diagnosis of mental illnesses such as depression and anxiety, and providing doctors with more accurate assessment data. Patient focus refers to a patient's reaction to a specific event during interaction with a psychiatrist, manifested as facial and head posture and eye movements. The specific event refers to a medical scale question or an action guided by the doctor during the doctor-patient interaction. Since a patient's facial and head posture, as well as eye movements, are dynamic over a period of time, the concept of information entropy can be introduced to describe these changes.

[0005] Existing research utilizes various techniques, such as calculating attention based on head posture and duration of eye closure. However, these methods only consider a portion of attention information. For example, head posture-based attention calculations only consider the head's posture in the current image, failing to connect it to previous states, making it difficult to reflect the patient's true attention level over a given period. Similarly, methods based on duration of eye closure struggle to determine the patient's gaze position when the head is stationary and eyes are open. Therefore, further research into patient attention assessment techniques during doctor-patient interactions, aiming to improve accuracy and real-time performance, will contribute to improving healthcare quality, enhancing doctor-patient relationships, and promoting the development of the healthcare system. Summary of the Invention

[0006] To address the above issues, this invention provides a patient attention discrimination method based on information entropy in a meta-clinic setting, used for assessing the mental and psychological state of special populations. This method solves the problem that doctors often struggle to simultaneously assess whether patients are focused during consultations; it also addresses how to quantify patient attention and how to characterize patient attention through head posture estimation and eye movement.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for determining patient attention based on information entropy in a meta-clinic setting includes the following steps:

[0009] 1) Detect faces in the video using the face detection module, and output the position of the face and the coordinates of the facial key points (both eyes);

[0010] 2) Capture images of both eyes using coordinates, predict the position of the pupils using a trained residual network model, calculate the vertical direction of eye gaze based on the position coordinates, and the difference in distance from the central axis is the eye movement state value.

[0011] 3) Calculate the Euler angles of the patient's head posture, and calculate the standard deviation and rate of change of each Euler angle component;

[0012] 4) Substitute the eye movement state value, the standard deviation of head posture, and the rate of change into the information entropy formula to calculate the entropy value. Invert the entropy value to obtain the final focus level.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0014] 1. Unlike the traditional doctor-patient consultation model, in the meta-consultation room setting, patients with mental health issues do not need to travel long distances to hospitals to queue for treatment. They can seek medical treatment at a regular computer with a camera or in a specially set up local treatment venue, which can save patients the cost of medical treatment.

[0015] 2. Based on face detection, head pose estimation, and eye tracking, this proposal quantifies a patient's level of focus by describing changes in the patient's head pose and eyes.

[0016] 3. Information entropy is introduced to describe changes in the patient's head posture and eye movements, and the patient's level of focus is calculated using the entropy formula.

[0017] As a further improvement to the above scheme, the face detection module uses the RetinaFace network model architecture. This network model adopts a pyramid structure, generates multiple feature layers through convolution and upsampling operations, and then performs classification, regression and prediction of facial key points on each feature layer.

[0018] The technical objectives of the aforementioned improvements are: to accurately detect faces in videos and output their location and key points; and to use the RetinaFace network model architecture for face detection, which offers advantages such as high accuracy, high efficiency, and multi-tasking capabilities. This invention utilizes a pyramid structure to generate feature layers of different scales, and then performs classification, regression, and facial key point prediction on each feature layer. The classification task outputs the classification category of the face pixels; the regression task outputs the location of the face in the image; and the facial key point task outputs several numerical values ​​representing the coordinates of features such as the eyes, nose tip, and corners of the mouth.

[0019] As a further improvement to the above scheme, the RetinaFace network model architecture uses ResNet-50 as the base network. This base network includes five convolutional blocks, each convolutional block includes several residual units, and each residual unit includes two convolutional layers and one skip connection layer. The skip connection layer adds the input feature map to the output feature map. The face detection module includes a linear rectified function, which is used to perform pixel correction processing on several image frames in the first image set and replace all negative values ​​in the feature map of the first image set with 0, that is, replace all negative values ​​in several image frames in the first image set with 0. The face detection module also includes the following steps:

[0020] A 1×1 convolution operation with a stride of 1 is applied to the feature layer Conv5_x to form the feature layer M5;

[0021] Upsampling is performed on feature layer M5, and it is fused with the feature layer Conv4_x after a 1×1 convolution operation with stride 1. The fused feature layer is then upsampled once to form feature layer M4.

[0022] After performing an upsampling operation on feature layer M4, it is fused with the feature layer Conv3_x after performing a 1×1 convolution operation with a stride of 1. The fused feature layer is then upsampled again to form feature layer M3.

[0023] After performing an upsampling operation on feature layer M3, it is fused with the feature layer Conv2_x after performing a 1x1 convolution operation with a stride of 1. The fused feature layer is then upsampled again to form feature layer M2.

[0024] Then, 3x3 convolution operations with a stride of 2 are applied to the feature layers M2, M3, M4, M5, and Conv5_x respectively to form effective feature layers P2, P3, P4, P5, and P6;

[0025] Finally, classification, regression, and prediction of facial landmarks are performed on the effective feature layers P2, P3, P4, P5, and P6 respectively.

[0026] The technical objectives of the above improvements are: to improve the performance and accuracy of the face detection model; to use ResNet-50 as the base network and employ a pyramid structure to generate feature layers at multiple scales, thereby improving the performance and accuracy of the face detection model; to use a linear rectified function to perform pixel correction processing on image frames, thereby eliminating the influence of negative values ​​on the face detection results; to use ResNet-50 as the base network to extract features from images in the video, then use the pyramid structure to generate multiple feature layers, and finally perform classification, regression, and facial landmark prediction on each feature layer. ResNet-50 includes five convolutional blocks, each containing several residual units. Each residual unit includes two convolutional layers and one skip connection layer. The skip connection layer adds the input feature map to the output feature map, thereby avoiding gradient vanishing and overfitting problems. The pyramid structure generates feature layers of different scales through convolution and upsampling operations, thus adapting to faces of different sizes. The linear rectified function replaces all negative values ​​in the image frame with 0, thereby eliminating the influence of negative values ​​on the face detection results.

[0027] As a further improvement to the above scheme, the residual network model is a deep convolutional neural network. The network includes multiple residual units, each of which includes two convolutional layers and a skip connection layer. The skip connection layer adds the input feature map to the output feature map.

[0028] The above-mentioned improved technologies aim to: accurately detect the position of the pupils in binocular images; use a residual network model for pupil detection, which has the advantages of deep learning, residual learning, and skip connections; use a deep convolutional neural network to extract features from binocular images, and then use residual units for feature enhancement. The residual units add the input feature map and the output feature map through skip connection layers, thereby avoiding gradient vanishing and overfitting problems.

[0029] As a further improvement to the above scheme, the Euler angle components include roll angle, pitch angle and yaw angle; wherein the roll angle represents the angle through which the head symmetry plane rotates about the head central axis; the pitch angle represents the angle between the head central axis and the horizontal plane; and the yaw angle represents the angle between the projection of the head central axis on the horizontal plane and the earth axis.

[0030] The technical objective of the above improvements is to describe changes in head posture mathematically, using Euler angle components. This method is simple and easy to understand, and can effectively represent the rotational motion of the head. Using the coordinate system of the rigid head body, three Euler angle components are calculated, representing the roll angle, pitch angle, and yaw angle of the head, respectively. The roll angle represents the angle through which the head's plane of symmetry rotates about the head's central axis; the pitch angle represents the angle between the head's central axis and the horizontal plane; and the yaw angle represents the angle between the projection of the head's central axis onto the horizontal plane and the earth's axis.

[0031] As a further improvement to the above scheme, the standard deviation represents the dispersion of the data; the rate of change represents the average value of the data change between adjacent collection points.

[0032] The technical objectives of the above improvements are: to mathematically measure the degree and frequency of head posture changes; to use standard deviation and rate of change to measure the degree and frequency of head posture changes, which is simple, effective, and can reflect data fluctuations; and to calculate the standard deviation and rate of change of each Euler angle component for each Euler angle component collected over a period of time, using Euler angle data collected over that period. The standard deviation represents the dispersion of the data; a larger standard deviation indicates a greater change in head posture. The rate of change represents the average change in data between adjacent collection points; a larger rate of change indicates a faster change in head posture.

[0033] As a further improvement to the above scheme, the information entropy formula is:

[0034] attention=abs(-c1∑(p*log(p))-c2m)

[0035] Where c1 and c2 are adjustment coefficients; p represents the probability value of the degree of change in head posture and the frequency distribution; m represents the eye movement state value; and abs() is the absolute value function.

[0036] The technical objectives of the above improvements are: to achieve mathematical assessment of patient focus; to use the information entropy formula to assess patient focus, which comprehensively considers both head posture and eye movement status, thus more accurately reflecting the patient's attention level; and to utilize the physical concept of information entropy to describe the uncertainty of the data, where a lower entropy value indicates high concentration and vice versa. In the information entropy formula, c1 and c2 are adjustment coefficients used to adjust the range and weight of focus scores according to the actual situation; p represents the probability value of the degree and frequency distribution of head posture changes, which can be determined based on the data distribution; m represents the eye movement status value, which can be calculated based on the difference between the vertical direction of eye gaze and the central axis; and abs() is the absolute value function used to convert the entropy value into a positive number.

[0037] As a further improvement to the above scheme, the eye movement state value m is calculated according to the following formula:

[0038]

[0039] Where d1 and d2 represent the vertical distances from the central axis of the left and right eyeballs, respectively; l represents the horizontal distance between the two eyes.

[0040] The technical objectives of the above improvements are: to mathematically calculate the distance difference between the vertical direction of eye gaze and the central axis; to use the above formula to calculate the eye movement state value m, which can simply and effectively reflect the degree of eye deflection, thus affecting the calculation of focus; to use a face detection module to obtain the coordinates of both eyes, and then use a residual network model to predict the position of the pupils, calculate d1, d2, and l based on the position coordinates, and then substitute them into the above formula to calculate m; when m is larger, it indicates that the distance difference between the vertical direction of eye gaze and the central axis is larger, and vice versa.

[0041] As a further improvement to the above scheme, the range of the focus value is [0, +∞); when the focus is higher, the entropy value is lower and its absolute value is higher; when the focus is lower, the entropy value is higher and its absolute value is lower.

[0042] The technical objectives of the above improvements are: to represent a patient's level of focus numerically; to use this numerical value to represent the patient's level of focus, which can intuitively reflect the patient's degree of concentration, thus facilitating diagnosis and treatment by doctors; and to calculate the entropy value using the information entropy formula, then invert the entropy value to obtain the final focus level. Since information entropy is a non-negative number, the range of focus level is [0, +∞). Higher focus levels result in lower entropy values ​​and higher absolute values; conversely, lower focus levels result in higher entropy values ​​and lower absolute values.

[0043] As a further improvement to the above scheme, the attention threshold is determined according to different meta-clinic scenarios and patient types; when the attention is below the threshold, it indicates that the patient is not concentrating and needs to adjust their state; when the attention is above the threshold, it is considered that the patient is concentrating and can be treated normally.

[0044] The technical objectives of the above improvements are: to establish reasonable attention thresholds based on different meta-clinic scenarios and patient types; to use attention thresholds to set these thresholds, a method that can be flexibly adjusted according to different meta-clinic scenarios and patient types, thereby improving the rationality and effectiveness of attention assessment; and to compare the attention threshold with the patient's attention level to determine the patient's attention level. When attention is below the threshold, it indicates that the patient is not concentrating and needs to adjust their state; when attention is above the threshold, it is considered that the patient is concentrating and can proceed with treatment normally. The attention threshold is determined based on different meta-clinic scenarios and patient types, taking into account personal characteristics such as the patient's age, gender, education level, and psychological state.

[0045] As a further improvement to the above solution, the meta-clinic scenario refers to a virtual clinic environment that enables remote medical services through Internet technology; the patient type refers to the patient's personal characteristics such as age, gender, education level, and psychological state.

[0046] The technical objective of the aforementioned improvements is to utilize internet technology to create a virtual consultation environment for remote medical services, referred to as a meta-consultation scenario. Within this scenario, doctors and patients can communicate via video, audio, and other means. For patients with mental health issues, they are categorized into different patient types based on their age, gender, education level, psychological state, and other individual characteristics. Different patient types may have different levels of focus and different diagnostic criteria. Attached Figure Description

[0047] Figure 1 This forms the overall framework of the solution.

[0048] Figure 2 This is the RetinaFace face detection network architecture.

[0049] Figure 3 This is a schematic diagram of Euler angles for head posture.

[0050] Figure 4 This is a schematic diagram for calculating eye movement state values. Detailed Implementation

[0051] To enable those skilled in the art to better understand the technical solution, the technical solution is described in detail below with reference to the embodiments. The description in this section is only exemplary and explanatory, and should not be used to limit the scope of protection of this application.

[0052] like Figure 1-4 As shown, a method for judging patient attention based on information entropy in a meta-clinic scenario is presented, and the technical solution is as follows:

[0053] Equipment Requirements: The Yuan Clinic client requires basic equipment such as terminals, cameras, and audio devices that can connect to the cloud server.

[0054] The overall framework of the solution includes a face detection module, a head pose estimation module, an eye tracking module, and a focus discrimination module.

[0055] Face detection: This proposal adopts a RetinaFace network model architecture for face detection; wherein, the RetinaFace network model architecture adopts a pyramid structure; the face detection model includes a linear rectified function, which is used to perform pixel correction processing on a number of image frames in the first image set and replace all negative values ​​in the feature map of the first image set with 0, that is, replace all negative values ​​in a number of image frames in the first image set with 0;

[0056] Feature maps Conv1_x, Conv2_x, Conv3_x, Conv4_x, and Conv5_x are formed through bottom-up convolution operations;

[0057] A 1×1 convolution operation with a stride of 1 is applied to the feature layer Conv5_x to form the feature layer M5;

[0058] Upsampling is performed on feature layer M5, and it is fused with the feature layer Conv4_x after a 1×1 convolution operation with stride 1. The fused feature layer is then upsampled once to form feature layer M4.

[0059] After performing an upsampling operation on feature layer M4, it is fused with the feature layer Conv3_x after performing a 1×1 convolution operation with a stride of 1. The fused feature layer is then upsampled again to form feature layer M3.

[0060] After performing an upsampling operation on feature layer M3, it is fused with the feature layer Conv2_x after performing a 1x1 convolution operation with a stride of 1. The fused feature layer is then upsampled again to form feature layer M2.

[0061] Then, 3x3 convolution operations with stride 2 are applied to the M2, M3, M4, M5, and Conv5_x feature layers respectively to form effective feature layers P2, P3, P4, P5, and P6. Finally, classification, regression, and prediction of facial key points are performed on the effective feature layers P2, P3, P4, P5, and P6 respectively.

[0062] The classification task outputs the classification category (cls) of the face pixels; the regression task outputs the position (box) of the face in the image, including the coordinates of four vertices; and the PFLD facial landmark recognition algorithm is used to predict the facial landmarks and output several values. In this invention, the PFLD facial landmark recognition algorithm is used to predict the facial landmarks (landmarks) and output ten values, which are the coordinates of the eyes, the tip of the nose, and the corners of the mouth, respectively. This invention does not impose specific limitations and can be set as needed to form a second image set.

[0063] Head pose estimation (Eulerian angles): First, establish a coordinate system for the rigid body of the human head in the image, and calculate the three Euler angle components (roll angle, pitch angle, and yaw angle) about the rigid body of the head, such as... Figure 3 As shown, the roll angle represents the angle through which the head's symmetry plane rotates around the head's central axis, with right roll being positive; the pitch angle represents the angle between the head's central axis and the horizontal plane, with head tilt being positive; and the yaw angle represents the angle between the projection of the head's central axis onto the horizontal plane and the earth's axis, with right yaw being positive.

[0064] Eye tracking: such as Figure 4As shown, the face detection module can obtain the coordinates of the patient's facial key points (both eyes). Using the coordinates, separate images of the two eyes (left and right eyes) are extracted from the original image. Then, the trained residual network model is used to detect the two eyeballs and obtain their positions. The distances of the two eyeballs from the center point of the eye can be calculated. The distances of the two eyeballs from the center point are divided by the total length of the eye, normalized, and then added together and divided by 2 to obtain the final eye movement state value.

[0065] The eye movement state value m is calculated using the following formula:

[0066]

[0067] Where d1 and d2 represent the vertical distances from the central axis of the left and right eyeballs, respectively; l represents the horizontal distance between the two eyes.

[0068] Attention level assessment: In the meta-clinic setting, this proposal uses information entropy to assess the patient's level of concentration. A relatively low entropy value indicates that the patient is highly focused, while a high entropy value indicates that the patient is inattentive.

[0069] Specifically, this proposal assigns entropy probability values ​​to three behaviors: the degree and frequency of changes in the patient's head posture, and the eye movement state.

[0070] The standard deviation of head posture data is calculated as follows: First, collect Euler angle data of head posture over a period of time, for example, once per second. Second, for each Euler angle component, calculate the standard deviation of that component over the acquisition time. The standard deviation represents the dispersion of the data; a larger standard deviation indicates greater variation in head posture. Finally, the average standard deviation of the three Euler angle components can be taken as an indicator of the degree of variation in head posture.

[0071] The method for calculating the average rate of change of head posture is as follows: First, collect Euler angle data of head posture over a period of time. Second, for each Euler angle component, calculate the difference between adjacent acquisition points to obtain the change in each component. Finally, average the changes in each component to obtain the average rate of change of head posture, which serves as an indicator of the frequency of change.

[0072] Eye movement state calculation method: First, detect the eyeball in the image and obtain the coordinates of the eyeball. Second, calculate the vertical direction of the eyeball's gaze based on the coordinates. The difference between the vertical direction of the eyeball's gaze and the central axis is the eye movement state value.

[0073] After collecting data on the rate and frequency of head posture changes and eye movements, this proposal employs a maximum-minimum normalization method to normalize these data, ensuring the values ​​are between 0 and 1. Finally, the patient's level of attention is calculated using the following formula:

[0074] attention=abs(-c1∑(p*log(p))-c2m)

[0075] Here, parameters c1 and c2 are adjustment coefficients used to adjust the range and weight of the attention score according to the actual situation. p represents the probability value of the degree and frequency of changes in the patient's head posture. m represents the patient's eye movement state value. abs() is the absolute value function. Therefore, the higher the level of attention, the lower the entropy value and the higher its absolute value.

[0076] The specific implementation method is as follows:

[0077] The first step is for the patient to click on the AI-assisted diagnosis section in the client and click "agree to turn on the camera." The cloud server then starts the AI-assisted diagnosis algorithm, which includes a method for judging the patient's focus.

[0078] The second step is for the face detection module to detect faces in the video and output the position of the face and the coordinates of the key points of the face (both eyes).

[0079] The third step involves capturing images of both eyes using coordinates, predicting the position of the pupils using a trained residual network, calculating the vertical direction of eye gaze based on the position coordinates, and determining the difference in distance from the central axis as the eye movement state value.

[0080] The fourth step is to calculate the Euler angles of the patient's head posture, and then calculate the standard deviation and rate of change of each Euler angle component.

[0081] The fifth step involves substituting the eye movement state value, the standard deviation of head posture, and the rate of change into the information entropy formula to calculate the entropy value. Inverting the entropy value yields the final focus level.

[0082] Suppose a patient engages in a video conversation for a period of time in a virtual consultation room setting; their head posture and eye movements are as follows: Figure 2 As shown. The probability values ​​for the degree and frequency of changes in head posture are p1 = 0.2, p2 = 0.3, and p3 = 0.5, corresponding to low, medium, and high levels, respectively; the eye movement state value is m = 0.4, indicating a small difference between the vertical direction of eye gaze and the central axis; taking the accommodation coefficients c1 = c2 = 1, the patient's focus level is:

[0083] attention=abs(-1×(0.2×log0.2+0.3×log0.3+0.5×log0.5)-1×0.4)

[0084] attention≈1.37

[0085] Assuming the attention threshold is 0.8, this patient's attention level is higher than the threshold, indicating that the patient is focused and can be treated normally.

[0086] Example 2: Suppose another patient engages in video communication for a period of time in a virtual consultation room setting, and their head posture and eye movement are as follows: Figure 3 As shown. The probability values ​​for the degree and frequency of head posture changes are p1 = 0.5, p2 = 0.3, and p3 = 0.2, corresponding to low, medium, and high levels, respectively; the eye movement state value is m = 0.8, indicating a large difference between the vertical direction of eye gaze and the central axis; taking the accommodation coefficients c1 = c2 = 1, the patient's focus level is:

[0087] attention=abs(-1×(0.5×log0.5+0.3×log0.3+0.2×log0.2)-1×0.8)

[0088] attention≈0.49

[0089] Assuming the attention threshold is 0.8, this patient's attention level is below the threshold, indicating that the patient has poor concentration and needs to adjust their state.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Specific examples have been used in this document to illustrate the principles and implementation methods of the technical solutions of this application. The above examples are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are merely preferred embodiments of this application. It should be pointed out that, due to the limitations of written expression and the objective existence of infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes, or combinations, or the direct application of the patent concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. A method for determining patient attention degree based on information entropy in a meta-clinic scene, characterized in that, The method comprises the following steps: 1) detecting a face in a video by a face detection module, outputting a position of the face and coordinates of key points of the face, including eyes; 2) taking an eye picture by the coordinates, predicting a position of an eye pupil by a trained residual network model, calculating a vertical direction of a gaze of the eye ball according to the position coordinates, and taking a difference between the vertical direction and a central axis as an eye movement state value; 3) calculating Euler angles of a head posture of the patient, calculating standard deviations and change rates of each Euler angle component; The Euler angle components include a roll angle, a pitch angle and a yaw angle; the roll angle represents an angle through which a head symmetry plane rotates around a head central axis; the pitch angle represents an angle between the head central axis and a horizontal plane; and the yaw angle represents an angle between a projection of the head central axis on a horizontal plane and a ground axis; The standard deviation represents a discrete degree of data; and the change rate represents an average value of a data change amount between adjacent collection points; 4) substituting the eye movement state value, the standard deviations and the change rates of the head posture into an information entropy formula, calculating an entropy value, taking a negative of the entropy value, and obtaining a final concentration; the information entropy formula is: attention=abs(-c1∑(p*log(p))-c2m) Wherein, c1 and c2 are adjustment coefficients; p represents a probability value of a change degree and a frequency distribution of the head posture; m represents the eye movement state value; and abs() is an absolute value function.

2. The method of claim 1, wherein, The face detection module uses a RetinaFace network model architecture, the network model adopts a pyramid structure, generates a plurality of feature layers through convolution operation and up-sampling operation, and then performs classification, regression and prediction of face key points on each feature layer.

3. The method of claim 2, wherein, The RetinaFace network model architecture uses ResNet-50 as a basic network, the basic network includes five convolution blocks, each convolution block includes a plurality of residual units, each residual unit includes two convolution layers and a skip connection layer, the skip connection layer adds the input feature map and the output feature map; the face detection module includes a linear rectifier function, the linear rectifier function is used for pixel correction processing on a plurality of image frames in a first image set, and all negative values in a feature map in the first image set are replaced with 0, that is, all negative values in the plurality of image frames in the first image set are replaced with 0; the face detection module comprises the following steps: 1) performing 1×1 convolution operation with a step length of 1 on the feature layer Conv5_x to form a feature layer M5; 2) performing up-sampling operation on the feature layer M5, fusing the feature layer after performing 1×1 convolution operation with a step length of 1 on the feature layer Conv4_x, and forming a feature layer M4 after performing up-sampling operation on the fused feature layer; 3) performing up-sampling operation on the feature layer M4, fusing the feature layer after performing 1×1 convolution operation with a step length of 1 on the feature layer Conv3_x, and forming a feature layer M3 after performing up-sampling operation on the fused feature layer again; 4) the feature layer M3 is subjected to one upsampling operation, and then fused with the feature layer obtained by using a 1*1 convolution operation with a step of 1 on the feature layer Conv2_x, and then subjected to one upsampling operation to form a feature layer M2; 5) the feature layers M2, M3, M4, M5 and Conv5_x are subjected to 3*3 convolution operations with a step of 2 to form effective feature layers P2, P3, P4, P5 and P6; 6) finally, classification, regression and face key point prediction are performed on the effective feature layers P2, P3, P4, P5 and P6, respectively.

4. The method of claim 1, wherein, The residual network model is a deep convolutional neural network, which comprises a plurality of residual units, each of which comprises two convolution layers and a skip connection layer, and the skip connection layer adds the input feature map and the output feature map.

5. The method of claim 1, wherein, The eye movement state value m is calculated according to the following formula: wherein d1 and d2 respectively represent the distance between the vertical direction of the eyeball gaze of the left eye and the right eye and the central axis; and l represents the horizontal distance between the two eyes.

6. The method of claim 1, wherein, The value range of the concentration degree is [0, +∞); when the concentration degree is higher, the entropy value is lower, and the absolute value is higher; when the concentration degree is lower, the entropy value is higher, and the absolute value is lower.

Citation Information

Patent Citations

  • Personnel work image attention detection method and system based on deep learning

    CN110705500A

  • Fatigue driving detection method based on deep learning and entropy combination

    CN114220144A