Children pain intelligent evaluation system based on AI facial expression analysis

Abstract

The invention relates to the field of medical auxiliary diagnosis, in particular to an intelligent child pain assessment system based on AI facial expression analysis, a facial detection module obtains a child facial image, and performs three-stage cascade detection by using a multi-task cascade convolutional neural network model to obtain facial key point pixel coordinate information; the key point positioning module carries out noise filtering processing on the coordinate information to obtain more accurate face key point coordinate information, the information is transmitted to the key point thermodynamic diagram representation module, and the key point thermodynamic diagram representation module generates a face key point thermodynamic diagram representing the degree of pains of children in combination with the face key point coordinate information and a thermodynamic diagram mechanism. The child pain intelligent evaluation module evaluates the degree of pain of the child based on the thermodynamic diagrams and obtains an evaluation result, and finally, the pain evaluation result visualization module visualizes the evaluation result, and the accuracy of pain characterization is improved through high-precision face detection and key point thermodynamic diagram technology, so that the accuracy of pain characterization is improved. And an effective method is provided for evaluating the pains of children.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical auxiliary diagnosis, in particular to a child pain intelligent evaluation system based on AI facial expression analysis, which is especially suitable for objective evaluation of the degree of pain of children patients who cannot accurately express the feeling of pain. BACKGROUND

[0002] Pain, as a subjective feeling, has important reference value in the process of clinical diagnosis and treatment. For adult patients, the degree, location and nature of pain can usually be accurately described through language. However, for young children patients, especially infants and special children who cannot accurately express, the pain evaluation has always been a difficulty in the medical field.

[0003] At present, the evaluation of children's pain in clinical practice mainly depends on the following methods: one is to observe the behavior of children, such as crying, restlessness, etc.; two is to evaluate through scale, such as facial expression pain scale (FLACC), Wong-Baker facial expression pain scale, etc.; three is to evaluate through physiological indicators, such as changes in heart rate, blood pressure, respiration, etc. These methods have certain limitations, such as strong subjectivity, low reliability of evaluation results, poor real-time performance, etc.

[0004] With the development of artificial intelligence technology, especially the progress of computer vision and deep learning technology, it is possible to use AI technology to analyze children's facial expressions to evaluate the degree of pain. However, the existing AI-based pain evaluation system has the following problems: one is that the facial expression capture is not accurate, resulting in inaccurate evaluation results; two is that the evaluation model lacks specific analysis of children's unique facial expression features; three is that it cannot effectively handle the interference brought by individual differences; four is the lack of intuitive evaluation result display.

[0005] Therefore, there is an urgent need for an intelligent system that can objectively, accurately and real-time evaluate the degree of children's pain to assist medical staff in reasonable pain management and treatment. SUMMARY

[0006] The purpose of the present application is to solve the problems existing in the prior art, and to provide a child pain intelligent evaluation system based on AI facial expression analysis. The system realizes high-precision face detection through multi-task cascading convolutional neural network, uses key point heat map technology to represent the degree of pain, and realizes objective evaluation through double model comparison evaluation mechanism, finally displays the evaluation results in a visual way, providing objective, accurate and real-time evaluation basis for children's pain for medical staff.

[0007] The present application provides a child pain intelligent evaluation system based on AI facial expression analysis, which comprises:

[0008] The face detection module is used to:

[0009] obtaining a child face image;

[0010] inputting the child face image into a multi-task cascaded convolutional neural network model for three-level cascaded detection to obtain child face key point pixel coordinate information;

[0011] A key point positioning module in communication connection with the face detection module, configured to:

[0012] receive the child face key point pixel coordinate information sent by the face detection module;

[0013] perform noise filtering processing on the child face key point pixel coordinate information to obtain face key point coordinate information;

[0014] A key point heat map representation module in communication connection with the key point positioning module, configured to:

[0015] receive the face key point coordinate information sent by the key point positioning module;

[0016] generate a face key point heat map representing the degree of child pain based on the combination of the face key point coordinate information and the heat map mechanism;

[0017] A child pain intelligent assessment module in communication connection with the key point heat map representation module, configured to:

[0018] receive the face key point heat map sent by the key point heat map representation module;

[0019] assess the pain of the child based on the face key point heat map to obtain an assessment result;

[0020] A pain assessment result visualization module in communication connection with the child pain intelligent assessment module, configured to:

[0021] receive the assessment result sent by the child pain intelligent assessment module;

[0022] visualize the assessment result.

[0023] Preferably, the child pain intelligent assessment module comprises:

[0024] A face key point heat map positioning sub-module configured to use a face key point heat map to position a detected face region according to the face key point heat map sent by the key point heat map representation module to obtain a face key point heat map;

[0025] A key point heat map score extraction sub-module in communication connection with the face key point heat map positioning sub-module, configured to obtain all key point heat map scores according to the face key point heat map sent by the face key point heat map representation module.

[0026] a child pain model training submodule, in communication connection with the key point heat map score extraction submodule, configured to train a child pain model according to all key point heat map scores output by the key point positioning module;

[0027] a child pain assessment submodule, in communication connection with the key point heat map score extraction submodule and the child pain model training submodule, configured to input results output by the key point heat map score extraction submodule into a child pain model for classification to obtain an assessment result.

[0028] Preferably, the facial key point heat map is represented as:

[0029] ,

[0030] wherein, is a heat map matrix of the i-th key point in the facial region, represents normalizing to a pixel value between (0, 1); represents the Euclidean distance between the point and the point . represents the i-th region represented in the heat map.

[0031] Preferably, obtaining all key point heat map scores is represented as:

[0032] ,

[0033] wherein, is a key point heat map score matrix.

[0034] Preferably, the child pain assessment model is constructed using a Dlib library; the child pain assessment model includes a pain assessment model and a no-pain assessment model; the pain assessment model is used to assess whether a child is in pain; the no-pain assessment model is used to assess whether a child is not in pain; the pain assessment model and the no-pain assessment model are obtained according to the child pain model training submodule using a preset number of training set data, the preset number being 50-100; the training set data includes child facial key point heat map scores.

[0035] Preferably, the key point heat map score extraction submodule outputs results including a child pain score matrix and a child no-pain score matrix; the child pain score matrix and the child no-pain score matrix are subjected to difference processing in a heat map subtraction manner to obtain a child pain assessment score, represented as:

[0036] ​

[0037] is a child pain assessment score, is a child pain score matrix, is a child pain-free score matrix.

[0038] As preferred, the evaluation result of the child facial pain assessment is expressed as:

[0039]

[0040] is a child pain assessment result, is a preset pain score threshold value; when , it indicates that the child is in pain, and the evaluation result output by the child pain intelligent assessment module is in pain; when , it indicates that the child is not in pain, and the evaluation result output by the child pain intelligent assessment module is not in pain.

[0041] As preferred, the face detection module comprises a face detection sub-module and an eye detection sub-module; the face detection module comprises a face detection model, an eye detection model, and an eye detection model; the face detection model is used for face detection on the child face image obtained by the face detection module, and outputs a face image; the eye detection model is used for eye detection on the child face image obtained by the face detection module, and outputs an eye image; the eye detection model is used for eye detection on the child face image obtained by the face detection module, and outputs an eye image; the face detection model adopts an MTCNN model, and the MTCNN model comprises a first-level detector, a second-level detector, and a third-level detector; the first-level detector is used for first-level face detection on the child face image obtained by the face detection module; the second-level detector is used for second-level face detection on the child face image obtained by the face detection module; the third-level detector is used for third-level face detection on the child face image obtained by the face detection module; the MTCNN model converts the intermediate result obtained by the first-level face detection on the child face image into the input image size of the second-level detector and the third-level detector through a down-sampling mechanism.

[0042] Preferably, in the MTCNN model, the trained eye detection model is used as a child face key point data processing model to obtain child face key point pixel coordinate information, expressed as:

[0043]

[0044] ​​​​​The weights are constants, typically ranging from 0.5 to 1.5. The two-dimensional pixel coordinates of the eye center output by the eye detection model; The output of the eye detection model is a two-dimensional pixel coordinate matrix of three key points, where each column represents the inner corner of the left eye. The outer corner of the left eye and the outer corner of the right eye The coordinates; This is the standardized coordinate matrix of key eye points obtained after transformation, where each column represents the standardized inner canthus of the left eye. The outer corner of the left eye and the outer corner of the right eye The coordinates.

[0045] In actual calculations, this formula represents the relative positional relationship between the coordinates of the eye's center and the coordinates of each corner of the eye. When calculating each key point separately, the formula expands to:

[0046] ,

[0047] This mapping method uses the stability of the eye region as a benchmark reference system and calculates more accurate facial key point locations through relative positional relationships, effectively solving the challenge of key point localization caused by facial expression changes in children under pain conditions.

[0048] Preferably, the key point localization module includes: a heatmap filtering submodule and a key point coordinate extraction submodule; the heatmap filtering submodule is used to perform mean filtering on the key point heatmap response values ​​output by the face detection module, using the following formula:

[0049] ,

[0050] In the formula, For heat map at location The response value at that location, The x-coordinate of the pixel in the heatmap. The vertical coordinate of the pixel in the heatmap. The filtered heatmap response value. This represents the number of pixels contained in the filter window. For the point The filtering window area centered on the center is typically... A square area of ​​size The recommended value is 3 or 5.

[0051] The key point coordinate extraction submodule is used to perform local maximum detection on the filtered heatmap to obtain stable key point coordinates of the child's face, represented as follows:

[0052] ,

[0053] In the formula, The coordinates of the detected key points, For The local neighborhood centered on the center is usually taken for or The window size; this formula represents the point A point whose heatmap response value is higher than that of all other points in its neighborhood is identified as a key point. To further improve the stability of key point localization, the system also performs weighted smoothing on the key point coordinates of consecutive frames along the time axis:

[0054] ,

[0055] In the formula, Indicates time The smoothed keypoint coordinates, For a moment The coordinates of the key points were directly detected. For a moment The smoothed keypoint coordinates, The smoothing coefficient ranges from (0,1], typically between 0.3 and 0.7. A larger value is used for high confidence levels, and a smaller value for low confidence levels. This two-stage processing method (first performing spatial domain filtering on the heatmap, then coordinate smoothing in the time domain) avoids the meaningless translation problems that may result from directly applying mean filtering to coordinate values. It also fully utilizes the spatial distribution information and temporal continuity of the heatmap, significantly improving the stability and robustness of key point localization. It is particularly suitable for handling facial tremors and expression changes that may occur in children under pain conditions.

[0056] The beneficial effects of this invention include:

[0057] 1. Through a three-level cascaded detection mechanism of multi-task cascaded convolutional neural network (MTCNN), high-precision detection of children's faces was achieved. Even under different lighting, angles and expression conditions, the facial regions and key points can be accurately located, providing a reliable foundation for subsequent analysis.

[0058] 2. Using key point heatmap technology to represent changes in children's facial expressions, discrete key point information is transformed into a continuous heat map distribution, which captures facial expression changes more comprehensively and improves the accuracy of pain representation.

[0059] 3. The system innovatively designs a "pain evaluation model" and a "non-pain evaluation model" double model comparison mechanism, judges the degree of pain through the difference between the output results of the two models, effectively eliminates the interference caused by individual differences, and improves the accuracy of evaluation.

[0060] 4. The system displays the evaluation results in a visual manner, including a pain degree heat map and a pain / no pain judgment result, which facilitates medical staff to intuitively understand the evaluation results and improves the efficiency of clinical decision-making.

[0061] 5. The system adopts a modular design, and the interfaces of each functional module are clear, which facilitates the expansion or optimization of functions according to clinical needs and adapts to the needs of different medical environments. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is the overall architecture diagram of the child pain intelligent evaluation system based on AI facial expression analysis of the present application;

[0063] Figure 2 is a structural diagram of the face detection module of the present application;

[0064] Figure 3 is a workflow diagram of the MTCNN model three-level detector of the present application;

[0065] Figure 4 is a structural diagram of the key point positioning module of the present application;

[0066] Figure 5 is a working principle diagram of the key point heat map representation module of the present application;

[0067] Figure 6 is a structural diagram of the child pain intelligent evaluation module of the present application;

[0068] Figure 7 is a working principle diagram of the double model comparison evaluation mechanism of the present application;

[0069] Figure 8 is a display effect diagram of the pain evaluation result visualization module of the present application. DETAILED DESCRIPTION

[0070] Please refer to the accompanying Figures 1-8 , the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0071] As Figure 1 shown, the child pain intelligent evaluation system based on AI facial expression analysis provided by the present application comprises a face detection module 1, a key point positioning module 2, a key point heat map representation module 3, a child pain intelligent evaluation module 4 and a pain evaluation result visualization module 5. Each module is connected to each other through a data communication interface, forming a complete pain evaluation process.

[0072] The face detection module 1 is configured to acquire a child face image and input the acquired child face image into a multi-task cascaded convolutional neural network (MTCNN) model for three-level cascaded detection to obtain child face key point pixel coordinate information. The key point positioning module 2 is in communication connection with the face detection module 1 and is configured to receive the child face key point pixel coordinate information sent by the face detection module 1, perform noise filtering processing on the information, and obtain more accurate face key point coordinate information. The key point heat map representation module 3 is in communication connection with the key point positioning module 2 and is configured to receive the face key point coordinate information sent by the key point positioning module 2, combine the coordinate information with a heat map mechanism, and generate a face key point heat map representing the degree of pain of the child. The child pain intelligent assessment module 4 is in communication connection with the key point heat map representation module 3 and is configured to receive the face key point heat map sent by the key point heat map representation module 3, assess the pain of the child based on the heat map, and obtain an assessment result. The pain assessment result visualization module 5 is in communication connection with the child pain intelligent assessment module 4 and is configured to receive the assessment result sent by the child pain intelligent assessment module 4 and display the assessment result in a visualized manner, so as to facilitate medical staff to understand and use.

[0073] Preferably, as shown in Figure 2 The face detection module 1 includes a face detection sub-module 11 and an eye detection sub-module 12. The face detection sub-module 11 includes a face detection model 111, an eye detection model 112, and an eye detection model 113. The face detection model 111 is configured to perform face detection on the child face image acquired by the face detection module 1 and output a face image; the eye detection model 112 is configured to perform eye detection on the child face image acquired by the face detection module 1 and output an eye image; and the eye detection model 113 is configured to perform eye detection on the child face image acquired by the face detection module 1 and output an eye image.

[0074] In an embodiment of the present application, the face detection model adopts an MTCNN model, as shown in Figure 3 The model includes a first-level detector 114, a second-level detector 115, and a third-level detector 116. The first-level detector 114 is configured to perform first-level face detection on the child face image acquired by the face detection module 1; the second-level detector 115 is configured to perform second-level face detection on the child face image acquired by the face detection module 1; and the third-level detector 116 is configured to perform third-level face detection on the child face image acquired by the face detection module 1. The MTCNN model converts the intermediate result of the first-level face detection on the child face image into an input image size of the second-level detector and the third-level detector through a down-sampling mechanism.

[0075] Specifically, the three-stage detection process of the MTCNN model is as follows: first, the first-stage detector uses a full convolutional network to perform a rapid scan on the input image to generate candidate windows; second, the second-stage detector performs fine detection on these candidate windows to filter out most non-face regions; and finally, the third-stage detector further accurately detects the face region and outputs the facial key point positions. This cascaded structure significantly improves the accuracy and efficiency of face detection. In actual applications, the scanning step of the first-stage detector is usually set to 2 pixels, and the scanning steps of the second-stage and third-stage detectors are set to 4 pixels and 8 pixels, respectively, to improve the processing speed while ensuring detection accuracy.

[0076] In addition, in the MTCNN model, the trained eye detection model is used as a child face key point data processing model to obtain child face key point pixel coordinate information, represented as:

[0077]

[0078] wherein, is a constant weight, usually ranging from 0.5 to 1.5; is the two-dimensional pixel coordinates of the eye center output by the eye detection model; is a two-dimensional pixel coordinate matrix of the three key points output by the eye detection model, where each column represents the coordinates of the left inner corner of the eye , the left outer corner of the eye and the right outer corner of the eye ; is a standardized eye key point coordinate matrix obtained after transformation, where each column represents the coordinates of the standardized left inner corner of the eye , the left outer corner of the eye and the right outer corner of the eye .

[0079] In actual calculation, this formula represents the relative position relationship mapping between the eye center coordinates and the corner coordinates. When each key point is calculated separately, the formula expands to:

[0080] ,

[0081] This mapping method uses the stability of the eye region as a reference system to calculate more accurate facial key point positions through relative position relationships, effectively solving the key point positioning challenge caused by facial expression changes under the child's pain state.

[0082] Preferably, the key point positioning module comprises a heat map filtering submodule 21 and a key point coordinate extraction submodule 22; the heat map filtering submodule 21 is used to perform mean filtering on the key point heat map response value output by the face detection module, and the formula is:

[0083] ,

[0084] In the formula, is the response value of the heat map at position , is the horizontal coordinate of the heat map pixel point, is the vertical coordinate of the heat map pixel point, is the filtered heat map response value, is the number of pixel points contained in the filtering window; is the filtering window region centered at point , which is usually a square region of size, The recommended value is 3 or 5.

[0085] The key point coordinate extraction submodule 22 is used for local maximum value detection on the filtered heat map to obtain stable child face key point coordinates, denoted as:

[0086] ,

[0087] In the formula, is the detected key point coordinate, is the local neighborhood centered at , which is usually is the window size of or ; the formula indicates that the heat map response value of point is higher than the response values of all other points in its neighborhood, so it is identified as the key point position. To further improve the stability of key point positioning, the system also performs weighted smoothing processing on the key point coordinates of consecutive frames on the time axis:

[0088] ,

[0089] In the formula, denotes the smoothed key point coordinate at time , is the directly detected key point coordinate at time , is the smoothed key point coordinate at time , The smoothing coefficient is in the range of (0, 1], and is usually between 0.3 and 0.7. A larger value is taken when the confidence is high, and a smaller value is taken when the confidence is low. This two-stage processing method (first spatial domain filtering of the heat map, and then coordinate smoothing in the time domain) avoids the problem of meaningless translation that may be caused by directly performing mean filtering on the coordinate values, while fully utilizing the spatial distribution information and time continuity of the heat map, greatly improving the stability and robustness of key point positioning, and is particularly suitable for processing facial jitter and expression changes that may occur in children in a painful state.

[0090] The principle of mean filtering is to replace the current pixel value with the average value of all pixels in the neighborhood of the current pixel. This can effectively remove high-frequency noise in the image, making the position of the facial key point more stable. In practical applications, noise filtering is crucial for improving the stability of key point positioning, especially for children patients, as there may be factors such as facial jitter and changes in lighting, which may cause noise interference in the original key point position.

[0091] Through noise filtering and key point coordinate extraction, the accuracy and stability of facial key point positioning can be significantly improved. This is crucial for subsequent pain assessment, as small deviations in the position of facial key points can lead to significant changes in the results of pain assessment.

[0092] Preferably, the key point heat map representation module 3 receives the facial key point coordinate information sent by the key point positioning module 2, and generates a facial key point heat map representing the degree of pain of the child based on these coordinate information. The facial key point heat map is represented as:

[0093] ,

[0094] wherein, is the heat map matrix of the i-th key point in the facial region, representing the heat influence value of the i-th key point on the coordinate point . represents the Euclidean distance between the point and the key point , and the calculation formula is , wherein is the coordinate of the i-th key point; is a parameter that controls the diffusion range of the heat map, and is usually between 1 and 5. In this embodiment, the preferred value is 2. A smaller value will make the heat map more concentrated, and a larger value will make the heat map more dispersed; represents the natural exponential function.

[0095] ​The core idea of heat map technology is to convert discrete key points into continuous heat distribution, capturing facial expression changes more comprehensively. In actual implementation, a two-dimensional Gaussian distribution centered on each key point is generated, with the response decreasing with distance. Then, the Gaussian distributions of all key points are superimposed to form the final heat map. This method can effectively represent subtle changes in facial expressions, especially the features commonly seen in pain expressions such as forehead wrinkles and tightly closed eye corners.

[0096] Preferably, as shown in Figure 6 , the child pain intelligent assessment module 4 includes a facial key point heat map positioning sub-module 41, a key point heat map score extraction sub-module 42, a child pain model training sub-module 43, and a child pain assessment sub-module 44.

[0097] The facial key point heat map positioning sub-module 41 is used to position the detected facial region using the facial key point heat map according to the facial key point heat map sent by the key point heat map representation module 3, to obtain the facial key point heat map. The key point heat map score extraction sub-module 42 is in communication connection with the facial key point heat map positioning sub-module 41, and is used to obtain all key point heat map scores according to the facial key point heat map sent by the key point heat map representation module 3. All key point heat map scores are represented as:

[0098]

[0099] wherein, is the heat map score of the i-th key point, representing the contribution of the key point to the pain expression; represents the maximum value operation; represents the coordinate point on the key point heat map; represents the response region of the i-th key point, defined as a rectangular region centered on the key point, usually taking a 5x5 or 7x7 region around the key point; is the heat value of the i-th key point at the coordinate . The process of key point heat map score extraction is to extract the maximum response value of each key region from the heat map as the pain feature score of the region. This method can accurately capture the key information in facial expression changes and provide reliable feature input for subsequent pain assessment.

[0100]

[0101] ​​​​The pediatric pain model training submodule 43 is communicatively connected to the keypoint heatmap score extraction submodule 42, and is used to train the pediatric pain model based on all keypoint heatmap scores output by the keypoint localization module 2. The pediatric pain assessment submodule 44 is communicatively connected to both the keypoint heatmap score extraction submodule 42 and the pediatric pain model training submodule 43, and is used to input the results output by the keypoint heatmap score extraction submodule 42 into the pediatric pain model for classification to obtain the assessment results.

[0102] In one embodiment of the present invention, such as Figure 7 As shown, the pediatric pain assessment model is built using the Dlib library. This assessment model includes a pain assessment model and a no-pain assessment model. The pain assessment model is used to assess whether a child is in pain; the no-pain assessment model is used to assess whether a child is not in pain. Both models are obtained from the pediatric pain model training submodule 43 using a preset number of training set data, which is 50-100 sets. The training set data includes facial key point heatmap scores for children, and each set of training data contains a child's facial image and its labeled pain state (pain or no pain).

[0103] The advantage of using a dual-model structure is that it can assess pain from both positive and negative perspectives, improving the accuracy of the judgment. In practical applications, the training dataset typically contains 50-100 sets of labeled facial expression samples from children, with pain samples and non-pain samples each accounting for about half. This amount of data ensures the effectiveness of model training without leading to overfitting. For children of specific age groups, the composition of the training samples can be appropriately adjusted. For example, for infants and toddlers, the proportion of crying samples can be increased; for preschool children, the proportion of facial features such as frowning and tightly closed eyes can be increased.

[0104] The key point heatmap score extraction submodule 42 outputs a child pain score matrix and a child no-pain score matrix. These two matrices are subtracted using heatmap subtraction to obtain the child's pain assessment score, represented as:

[0105] ,

[0106] Among them, Score is the child's pain assessment score, which represents the probability of the child's current pain state, and the value range is usually between [-1, 1]. The pain score matrix for children is the score output by the pain assessment model, representing the degree to which pain features are detected. The pain-free score matrix for children is the score output by the pain-free assessment model, representing the degree to which pain-free features are detected.

[0107] The core idea of this difference calculation method is to eliminate the interference of individual differences by comparing the output results of the two opposite direction evaluation models, and to improve the accuracy of evaluation. In practice, when the evaluation score is positive and the absolute value is large, it indicates that the child is more likely to be in pain; when the evaluation score is negative or close to zero, it indicates that the child is more likely to be in a pain-free state.

[0108] The evaluation result of the child's facial pain assessment is represented as:

[0109]

[0110] is the final binary classification result, which can be "pain" or "no pain"; is the preset pain score threshold, usually taking a value between 0.3-0.7, and the preferred value in this embodiment is 0.5, which is the best judgment threshold determined according to clinical verification; is the child pain assessment score calculated above. When , it indicates that the child is in pain, and the evaluation result output by the child pain intelligent assessment module 4 is "pain"; when , it indicates that the child is not in pain, and the evaluation result output by the child pain intelligent assessment module 4 is "no pain".

[0111] The setting of the threshold has an important impact on the accuracy of the evaluation result. If the threshold is set too low, it may result in too many false positive results; if the threshold is set too high, it may result in too many false negative results. In actual application, the threshold can be adjusted according to the characteristics of children of different ages and clinical needs, for example, for young children, the threshold can be appropriately lowered to 0.3-0.4 to improve the pain detection rate; for older children, the threshold can be appropriately increased to 0.6-0.7 to reduce misjudgment.

[0112] Preferably, as shown in Figure 8 , the pain assessment result visualization module 5 is used to receive the evaluation result sent by the child pain intelligent assessment module 4 and display the evaluation result in a visualized manner. The visualized display methods include but are not limited to: facial key point connection diagram, heat map display, pain / no pain text identification, pain degree level display, etc. This intuitive visualized display method facilitates medical staff to quickly understand the evaluation result and improves the efficiency of clinical decision-making.

[0113] ​​In practical applications, the system can select different visualization modes according to different application scenarios. For example, in an emergency environment, the pain / no pain judgment result and the pain level can be displayed first, so that medical staff can make quick decisions; in long-term pain management, a pain trend chart can be added to track the changes in pain; in a pediatric ward, a more friendly interface design can be used to reduce the fear of children.

[0114] The workflow of the present application is as follows: first, the face detection module 1 obtains the child's face image and performs three-level cascade detection through the MTCNN model to obtain the face key point pixel coordinate information; then, the key point positioning module 2 performs noise filtering processing on the coordinate information to obtain more accurate face key point coordinate information; next, the key point heat map representation module 3 generates a face key point heat map representing the child's pain level based on the coordinate information; then, the child pain intelligent assessment module 4 assesses the child's pain based on the face key point heat map through a double-model comparison mechanism to obtain an assessment result; finally, the pain assessment result visualization module 5 displays the assessment result in a visualized manner to assist medical staff in pain management and treatment decision-making.

[0115] In the embodiment, the system can be deployed in pediatric wards, emergency departments, operating rooms and other places where child pain assessment is needed in hospitals. The hardware devices required for system operation include but are not limited to high-definition cameras, computing servers, display terminals, etc. The system software can be designed modularly to facilitate customization and optimization according to the needs of different medical environments.

[0116] Through the child pain intelligent assessment system based on AI facial expression analysis provided by the present application, medical staff can obtain objective, accurate and real-time child pain assessment results, effectively solving the problem of strong subjectivity and low reliability of traditional assessment methods, and providing a powerful tool for child pain management.

[0117] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A child pain intelligent assessment system based on AI facial expression analysis, characterized in that, include: The face detection module is used for: Obtain images of children's faces; The child's facial image is input into a multi-task cascaded convolutional neural network model for three-level cascaded detection to obtain the pixel coordinate information of key points on the child's face. The key point localization module, which is communicatively connected to the face detection module, is used for: Receive the pixel coordinate information of key points on the child's face sent by the face detection module; The pixel coordinate information of the child's facial key points is subjected to noise filtering to obtain the facial key point coordinate information. The key point heatmap representation module, which is communicatively connected to the key point localization module, is used for: Receive facial key point coordinate information sent by the key point positioning module; Based on the combination of the facial key point coordinate information and the heat map mechanism, a facial key point heat map representing the degree of pain in children is generated. The intelligent pain assessment module for children is communicatively connected to the key point heatmap representation module and is used for: Receive the facial key point heatmap sent by the key point heatmap representation module; Based on the facial key point heat map, the child's pain was assessed, and the assessment results were obtained. The pain assessment result visualization module, which is communicatively connected to the intelligent pain assessment module for children, is used for: Receive the assessment results sent by the child pain intelligent assessment module; The evaluation results are visualized.

2. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 1, characterized in that, The intelligent pain assessment module for children includes: The facial key point heatmap localization submodule is used to locate the detected facial region based on the facial key point heatmap sent by the key point heatmap representation module, and obtain the facial key point heatmap. The key point heatmap score extraction submodule is communicatively connected to the facial key point heatmap positioning submodule and is used to obtain the scores of all key points heatmaps based on the facial key point heatmap sent by the facial key point heatmap representation module. The child pain model training submodule is communicatively connected to the key point heatmap score extraction submodule and is used to train the child pain model based on all key point heatmap scores output by the key point localization module. The child pain assessment submodule is communicatively connected to the key point heatmap score extraction submodule and the child pain model training submodule. It is used to input the output of the key point heatmap score extraction submodule into the child pain model for classification to obtain the assessment result.

3. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 2, characterized in that, The facial key point heatmap is represented as follows: ， in, Let be the heatmap matrix of the i-th key point in the facial region; Indicates will Pixel values ​​normalized to (0,1); Point and points The Euclidean distance between them; The heatmap represents the first... Each region.

4. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 2, characterized in that, The heatmap scores for all key points are represented as follows: ， in, The heatmap fraction matrix for key points. The first one represented in the heat map The scores for each region.

5. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 2, characterized in that, The child pain assessment model is constructed using the Dlib library; the child pain assessment model includes: a pain assessment model and a no-pain assessment model; the pain assessment model is used to assess whether a child is in pain; the no-pain assessment model is used to assess whether a child is not in pain; the pain assessment model and the no-pain assessment model are obtained from the child pain model training submodule using a preset number of training set data, the preset number being 50-100; the training set data includes the child's facial key point heatmap scores.

6. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 5, characterized in that, The output of the key point heatmap score extraction submodule includes: a child pain score matrix and a child no-pain score matrix; the child pain score matrix and the child no-pain score matrix are subtracted using a heatmap subtraction method to obtain the child pain assessment score, represented as: ， in, For children's pain assessment scores, For children's pain score matrix, A pain-free score matrix for children.

7. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 5, characterized in that, The assessment results of facial pain assessment in children are expressed as follows: ， in, For the results of pain assessment in children, The preset pain score threshold; when When the child is in pain, the intelligent pain assessment module outputs an assessment result indicating that the child is in pain; when When the time is specified, it indicates that the child is not in pain, and the assessment result output by the child pain intelligent assessment module is "no pain".

8. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 1, characterized in that, The face detection module includes a face detection submodule and an eye detection submodule. The face detection module includes a face detection model, an eye detection model, and an eye detection model. The face detection model performs face detection on the child's face image acquired by the face detection module and outputs a face image. The eye detection model performs eye detection on the child's face image acquired by the face detection module and outputs an eye image. The eye detection model performs eye detection on the child's face image acquired by the face detection module and outputs an eye image. The face detection model uses the MTCNN model, which includes a first-level detector, a second-level detector, and a third-level detector. The first-level detector performs first-level face detection on the child's face image acquired by the face detection module. The second-level detector performs second-level face detection on the child's face image acquired by the face detection module. The third-level detector performs third-level face detection on the child's face image acquired by the face detection module. The MTCNN model uses a downsampling mechanism to convert the intermediate results obtained from the first-level face detection of the child's face image into the input image size for the second-level and third-level detectors.

9. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 7, characterized in that, In the MTCNN model, the trained eye detection model is used as the child's facial key point data processing model to obtain the pixel coordinate information of the child's facial key points, which is represented as: in, The weights are constants, typically ranging from 0.5 to 1.

5. The two-dimensional pixel coordinates of the eye center output by the eye detection model; The output of the eye detection model is a two-dimensional pixel coordinate matrix of three key points, where each column represents the inner corner of the left eye. The outer corner of the left eye and the outer corner of the right eye The coordinates; This is the standardized coordinate matrix of key eye points obtained after transformation, where each column represents the standardized inner canthus of the left eye. The outer corner of the left eye and the outer corner of the right eye The coordinates.

10. The intelligent pain assessment system for children based on AI facial expression analysis according to claim 7, characterized in that, The key point localization module includes a heatmap filtering submodule and a key point coordinate extraction submodule; the heatmap filtering submodule is used to perform mean filtering on the key point heatmap response values ​​output by the face detection module, using the following formula: ， In the formula, For heat map at location The response value at that location, The x-coordinate of the pixel in the heatmap. The vertical coordinate of the pixel in the heatmap. The filtered heatmap response value. This represents the number of pixels contained in the filter window. For the point The filtering window area centered on the center is typically... A square area of ​​size The recommended value is 3 or 5.

Images