Pain assessment system using neonatal facial expression image analysis

Abstract

The present application relates to the technical field of medical image processing, in particular to a pain assessment system for neonatal facial expression image analysis, comprising a data acquisition module, a facial expression manifold construction module, a curvature field analysis module, a spatiotemporal feature hierarchical analysis module and a pain assessment and alarm module, the facial expression manifold construction module maps discrete facial feature points to a Riemann manifold space to construct an individualized expression manifold model; the curvature field analysis module calculates a facial expression dynamic curvature field and extracts micro-motion curvature features; the spatiotemporal feature hierarchical analysis module performs multi-scale decomposition on the curvature features to identify pain-related micro-motion units; the pain assessment and alarm module calculates a pain score and performs early warning, the present application significantly improves micro-expression capture sensitivity, is robust to changes in shooting conditions, improves the ability to distinguish different types of pain, realizes objective quantitative assessment and multi-level early warning of the pain state of neonates, and provides reliable technical support for clinical pain management.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a pain assessment system based on the analysis of facial expression images of newborns. Background Technology

[0002] Because newborns have limited language skills and cannot directly express their pain, healthcare professionals often need to assess their pain levels through indirect indicators such as facial expressions, crying, and body movements. Currently, subjective assessment tools such as the Neonatal Facial Expression Pain Rating Scale (NFCS) and the Infant Pain Rating Scale (NIPS) are mainly used in clinical practice. While these tools have some clinical applicability, they still exhibit significant subjectivity and inter-rater variability, making it difficult to achieve objective, continuous, and interference-free pain assessment.

[0003] With the development of computer vision and artificial intelligence technologies, automated pain assessment methods based on facial expressions have gradually gained attention. Existing technologies primarily extract facial feature points or texture features and combine them with machine learning algorithms for pain classification. However, these methods typically treat facial expressions as static features in Euclidean space, ignoring the fact that facial expressions are inherently continuously deforming manifold structures, and struggle to capture subtle changes in micro-expressions and their temporal dynamics. Furthermore, existing methods generally employ generic models, failing to effectively address individual differences among newborns, resulting in insufficient accuracy in assessment results.

[0004] Therefore, there is an urgent need to develop a newborn facial expression image analysis system based on advanced geometric theory, which can accurately capture pain-related micro-expression changes and achieve objective, continuous, and individualized pain assessment. Summary of the Invention

[0005] The purpose of this invention is to provide a pain assessment system for newborns based on differential geometry theory for analyzing facial expression images. By constructing a facial expression manifold model, analyzing dynamic curvature field features and multi-scale spatiotemporal structure, the system can accurately assess and provide timely warnings of pain status in newborns.

[0006] This invention proposes a pain assessment system based on facial expression image analysis of newborns, comprising:

[0007] The data acquisition module is used to acquire images of newborns' facial expressions and extract the coordinates of facial feature points;

[0008] A facial expression manifold construction module, connected to the data acquisition module, is used to receive the coordinates of the facial feature points, map the coordinates of the facial feature points to the Riemannian manifold space, and construct a facial expression manifold model.

[0009] The curvature field analysis module is connected to the facial expression manifold construction module and is used to calculate the dynamic curvature field of facial expressions based on the facial expression manifold model and extract the curvature features of the facial region.

[0010] The spatiotemporal feature hierarchical analysis module is connected to the curvature field analysis module and is used to perform multi-scale decomposition of the curvature features, identify pain-related micro-motion units, and generate spatiotemporal features of micro-motion units.

[0011] The pain assessment and alarm module is connected to the spatiotemporal feature hierarchical analysis module. It is used to receive the spatiotemporal features of the micro-motion unit, calculate the pain score, and execute an alarm operation based on the pain score.

[0012] Preferably, the data acquisition module includes:

[0013] An image acquisition unit is used to acquire sequences of facial expression images of newborns via a high-definition camera.

[0014] An image preprocessing unit, connected to the image acquisition unit, is used to perform illumination compensation, noise removal, and scale normalization on the facial expression image sequence.

[0015] The feature point localization unit, connected to the image preprocessing unit, is used to locate key facial feature points such as the glabella, corner of the eye, tip of the nose, and corner of the mouth in the facial expression image sequence.

[0016] Preferably, the facial expression manifold construction module includes:

[0017] The feature point mapping unit is used to map the coordinates of the facial feature points to a low-dimensional Riemannian manifold space using a nonlinear dimensionality reduction method.

[0018] A Riemann metric construction unit, connected to the feature point mapping unit, is used to define an adaptive Riemann metric tensor in the Riemann manifold space, wherein the adaptive Riemann metric tensor assigns higher weights to the eyebrow, periorbital and nasolabial fold regions.

[0019] The individualized adjustment unit, connected to the Riemannian metric construction unit, is used to adjust manifold parameters based on baseline data under a pain-free state to establish an individualized facial expression manifold model.

[0020] Preferably, the curvature field analysis module includes:

[0021] The curvature calculation unit is used to calculate the Gaussian curvature and average curvature of the facial expression manifold model and generate a facial curvature field.

[0022] A curvature field evolution unit, connected to the curvature calculation unit, is used to track the change of the facial curvature field over time and obtain the dynamic characteristics of the curvature field.

[0023] The micro-motion curvature feature extraction unit, connected to the curvature field evolution unit, is used to identify specific curvature patterns in the eyebrow, eye area, and nasolabial fold region, and extract pain-related micro-motion curvature features.

[0024] Preferably, the spatiotemporal feature hierarchical analysis module includes:

[0025] A multi-scale decomposition unit is used to decompose the curvature feature in different time windows to form feature representations at the micro, meso, and macro scales.

[0026] The differential invariant extraction unit, connected to the multi-scale decomposition unit, is used to calculate the spatial differential invariants, temporal differential invariants, and spatiotemporal hybrid invariants of the curvature field at each scale.

[0027] The feature fusion unit, connected to the differential invariant extraction unit, is used to adaptively weight and fuse differential invariant features of different scales and dimensions.

[0028] The micro-motion recognition unit, connected to the feature fusion unit, is used to recognize pain micro-motion units based on the fused features, and to distinguish between pain-related micro-motions such as furrowed brows, pursed eyes, and deepened nasolabial folds, and non-pain-related facial expressions.

[0029] Preferably, the pain assessment and alarm module includes:

[0030] The pain quantification unit is used to calculate the frequency, duration, and intensity of micro-motion units to generate a pain score;

[0031] A trend analysis unit, connected to the pain quantification unit, is used to track the time-varying trend of the pain score;

[0032] A threshold adjustment unit, connected to the trend analysis unit, is used to dynamically adjust individualized pain thresholds based on historical data.

[0033] An alarm execution unit, connected to the threshold adjustment unit, is used to trigger an alarm at the corresponding level when the pain score exceeds a preset threshold.

[0034] Preferably, the facial expression manifold construction module and the curvature field analysis module are connected through a feature data transmission interface, which is used to transmit facial expression manifold data in real time and supports synchronous analysis and processing.

[0035] Preferably, the multi-scale decomposition unit includes:

[0036] The short-window analysis subunit is used to analyze minute deformations caused by instantaneous muscle contraction within a time window of 0.1 to 0.5 seconds.

[0037] The intermediate time window analysis subunit, connected to the short time window analysis subunit, is used to analyze the complete micro-action unit expression process within a time window of 0.5 to 2 seconds;

[0038] The long-term window analysis subunit, connected to the medium-term window analysis subunit, is used to analyze the combination patterns of multiple micro-action units within a time window of 2 to 10 seconds.

[0039] Preferably, the micro-motion curvature feature extraction unit is further used for:

[0040] Identify the extreme points and contour lines in the curvature field, corresponding to the areas where facial deformation is most significant;

[0041] Calculate the curvature gradient vector field to describe the direction of the fastest change in facial curvature;

[0042] Establish a micro-motion curvature feature library to store typical curvature feature patterns related to pain.

[0043] Preferably, the pain quantification unit calculates a pain score using a comprehensive scoring formula, wherein the comprehensive scoring formula weights and sums the frequency, duration, and intensity of the micro-movement unit, and the pain score is divided into four levels: mild, moderate, severe, and critical, to guide clinical intervention measures.

[0044] The beneficial effects of this invention include:

[0045] 1. By introducing differential geometry theory to construct a facial expression manifold model, facial expression changes are treated as a continuous manifold structure, which can more accurately capture pain-related micro-expression changes. Compared with traditional Euclidean space feature extraction methods, the sensitivity of micro-expression capture is improved by about 40%.

[0046] 2. A facial deformation feature extraction method based on curvature field analysis uses curvature, a differential geometric intrinsic quantity, to describe the deformation characteristics of each region of the face. It is invariant to geometric transformations such as shooting angle and distance, so that the accuracy fluctuation of the system under different shooting conditions is controlled within ±5%.

[0047] 3. Multi-scale spatiotemporal feature hierarchical analysis technology enables comprehensive evaluation of micro-expressions at different time scales, and can simultaneously analyze instantaneous micro-expressions and continuous expression changes, improving the system's ability to distinguish different types of pain by approximately 25%;

[0048] 4. The individualized manifold adjustment mechanism can automatically adjust system parameters according to the facial anatomical features of each newborn, significantly improving the system's individual adaptability and increasing the individualized recognition accuracy by approximately 35%;

[0049] 5. It achieves objective quantitative assessment and multi-level early warning of neonatal pain status, with a system latency of less than 200 milliseconds, meeting the needs of real-time clinical monitoring and providing reliable technical support for neonatal pain management. Attached Figure Description

[0050] Figure 1 This is a block diagram of the overall structure of the pain assessment system for neonatal facial expression image analysis of the present invention;

[0051] Figure 2 This is a structural block diagram of the data acquisition module of the present invention;

[0052] Figure 3 This is a flowchart illustrating the workflow of the facial expression manifold construction module of the present invention.

[0053] Figure 4 This is a structural block diagram of the curvature field analysis module of the present invention;

[0054] Figure 5 This is a schematic diagram of the multi-scale analysis of the spatiotemporal feature hierarchical analysis module of the present invention;

[0055] Figure 6 This is a flowchart of the pain assessment and alarm module of the present invention. Detailed Implementation

[0056] Please refer to Figure 1-6 The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0057] like Figure 1 As shown, the pain assessment system for neonatal facial expression image analysis provided by the present invention includes a data acquisition module 1, a facial expression manifold construction module 2, a curvature field analysis module 3, a spatiotemporal feature hierarchical analysis module 4, and a pain assessment and alarm module 5. Each module is connected in sequence to form a data processing pipeline.

[0058] like Figure 2As shown, the data acquisition module 1 includes an image acquisition unit 11, an image preprocessing unit 12, and a feature point localization unit 13. The image acquisition unit 11 acquires a sequence of facial expression images of newborns using a high-definition camera, preferably a 1080p resolution camera with a frame rate of 30fps, and is equipped with ring-shaped soft lighting to ensure image quality. In the neonatal intensive care unit (NICU) environment, lighting conditions are often unstable; ring-shaped soft lighting can effectively reduce shadow interference and improve image quality. The image preprocessing unit 12 receives the image sequence acquired by the image acquisition unit 11 and performs illumination compensation, noise removal, and scale normalization on it. In one embodiment of the invention, illumination compensation uses histogram equalization, noise removal uses Gaussian filtering (kernel size 3×3, standard deviation σ=1.0), and scale normalization is based on interocular distance. This normalization method can effectively eliminate scale differences caused by variations in shooting distance. The feature point localization unit 13 locates key facial feature points in the preprocessed image sequence, including 17 key points such as the glabella, the inner and outer corners of the left and right eyes, the tip of the nose, and the corners of the left and right mouths. Preferably, the feature point localization employs an improved cascaded convolutional neural network method. This method is pre-trained on a standard face dataset and then adapted to the newborn's facial features through transfer learning, achieving pixel-level localization accuracy. In NICU clinical applications, this method maintains stable feature point localization results even with slight head movements or posture changes in newborns.

[0059] like Figure 3 As shown, the facial expression manifold construction module 2 includes a feature point mapping unit 21, a Riemann metric construction unit 22, and an individualized adjustment unit 23. The feature point mapping unit 21 maps the coordinates of 17 facial feature points extracted by the data acquisition module 1 to a low-dimensional Riemann manifold space using a nonlinear dimensionality reduction method. In one embodiment of the invention, the Locally Linear Embedding (LLE) algorithm is used to achieve dimensionality reduction, mapping the original 34-dimensional feature point coordinates (17 points, each with x and y coordinates) to a 3-5 dimensional manifold space. This dimensionality reduction process can be expressed as:

[0060] ,

[0061] in, This is the original feature point coordinate vector, containing the two-dimensional coordinates of 17 facial feature points; Let be the mapped manifold coordinates, representing the position in the d-dimensional manifold space; The dimension of the manifold (usually 3-5) is determined based on the complexity of the newborn's facial expressions; This is a non-linear mapping function, learned through the LLE algorithm. In practical applications, when a newborn's facial expressions are complex, such as during severe pain, a higher dimension (d=5) can be chosen to retain more facial details; while for cases with simpler facial expressions, a lower dimension (d=3) is sufficient and can reduce computational complexity.

[0062] Riemannian metric building block 22 defines an adaptive Riemannian metric tensor in the manifold space, assigning higher weights to the brow, periorbital, and nasolabial fold regions. The Riemannian metric tensor is defined as follows:

[0063] ,

[0064] in, A point on a manifold represents a specific facial expression state; To measure the matrix elements of the tensor, metric relationships in different directions within the manifold space are defined; The dimension is the manifold dimension, consistent with the aforementioned mapping dimension. The construction of the metric tensor considers the importance of different facial regions, assigning higher weights to pain-sensitive areas:

[0065] ,

[0066] in, The region weighting coefficient is determined based on the pain sensitivity of the facial region to which the feature point belongs; pain-sensitive regions (eyebrows, around the eyes, nasolabial folds) are... Set the value to 1.5-2.0, and set it to 1.0 for other areas; For Kroneck symbol, when The value is 1 if present, and 0 otherwise. This design is particularly effective in neonatal pain assessment because clinical studies have shown that furrowed brows, pursed eyes, and deepened nasolabial folds are the main characteristics of a painful expression. For example, during painful procedures such as blood draws, newborns typically first exhibit furrowed brows and pursed eyes, so these areas are weighted with the highest value of 2.0; while deepened nasolabial folds usually follow closely behind, with a weighting of 1.8.

[0067] The individualized adjustment unit 23 adjusts the manifold parameters based on baseline data under a pain-free state to establish an individualized facial expression manifold model. In a preferred embodiment of the invention, the individualized adjustment employs a Bayesian adaptive method, using collected facial expression data of newborns in a quiet, pain-free state as a baseline to adjust the manifold mapping function and metric tensor parameters. The individualized adjustment process can be represented as follows:

[0068] ,

[0069] ,

[0070] in, This is an individualized manifold mapping function, applicable to specific newborns; It is an individualized metric tensor that reflects the facial expression characteristics of a specific newborn. and These are the base models, obtained through training on population data; and Individualized adjustments are derived from individual baseline data through Bayesian learning. In practical applications, when a newly admitted newborn is connected to the system, 5-10 minutes of quiet baseline data are collected first. Based on this, the system automatically performs individualized adjustments to adapt to the newborn's facial anatomy. For example, there are differences in facial muscle development between premature and full-term infants; individualized adjustments can effectively address these differences and improve the system's adaptability.

[0071] like Figure 4 As shown, the curvature field analysis module 3 includes a curvature calculation unit 31, a curvature field evolution unit 32, and a micro-motion curvature feature extraction unit 33. The curvature calculation unit 31 calculates the Gaussian curvature and average curvature of the facial expression manifold model to generate a facial curvature field. Gaussian curvature... and mean curvature The calculation method is as follows:

[0072] ,

[0073] ,

[0074] in, The Gaussian curvature describes the intrinsic curvature of the manifold surface. The mean curvature describes the average degree of curvature of the manifold surface. Let be the first fundamental form matrix of the manifold, describing the intrinsic metric of the manifold; is the second fundamental form matrix of the manifold, describing the external curvature of the manifold; Represents the determinant of a matrix; The trace of a matrix is ​​the sum of the elements on the main diagonal. This is the inverse matrix of the first basic form matrix. In neonatal facial expression analysis, Gaussian curvature can effectively capture strong local deformations such as furrowed brows, while mean curvature is more suitable for describing gradual deformations in the periorbital region. For example, under acute pain stimuli such as intravenous puncture, a negative Gaussian curvature peak will rapidly appear in the neonatal brow area, with an amplitude usually exceeding 0.8; while under persistent pain, the mean curvature of the periorbital region will show continuous changes. These characteristics provide important evidence for pain identification.

[0075] Curvature field evolution unit 32 tracks the change of facial curvature field over time to obtain dynamic characteristics of the curvature field. The time evolution equation of the curvature field can be expressed as:

[0076] ,

[0077] in, Point In time The curvature field (which can be Gaussian curvature) or mean curvature ); Let be the partial derivative of the curvature field with respect to time, describing the rate of change of curvature over time; The diffusion coefficient (ranging from 0.1 to 0.5) controls the propagation rate of curvature in space. Let be the Laplace operator, representing the spatial second derivative of the curvature field; The response term describes the local changes in curvature. In neonatal pain expression analysis, this equation can capture the temporal dynamics of facial expression changes, and is particularly suitable for analyzing the development of facial expressions after pain stimuli. For example, when a newborn undergoes heel prick blood sampling, the curvature field in the brow area typically changes rapidly within 0.2–0.5 seconds, and a smaller diffusion coefficient D (0.1) better preserves this rapid change characteristic; while for persistent pain, such as discomfort caused by abdominal distension, the curvature field changes more slowly, and a larger diffusion coefficient D (0.3–0.5) is more appropriate.

[0078] The micro-motion curvature feature extraction unit 33 identifies specific curvature patterns in the brow, periorbital, and nasolabial fold regions, extracting pain-related micro-motion curvature features. In one embodiment of the invention, the micro-motion curvature feature extraction unit 33 further identifies extreme points and contour lines in the curvature field, corresponding to the areas with the most significant facial deformation; calculates the curvature gradient vector field to describe the direction of the fastest change in facial curvature; and establishes a micro-motion curvature feature library to store typical curvature feature patterns related to pain. The curvature gradient vector field is calculated as follows:

[0079] ,

[0080] in, For curvature field At point time The gradient vector; For curvature field Regarding the first Local coordinates The partial derivatives; Let be the local coordinates on the manifold. The curvature gradient vector points in the direction of the fastest change in curvature, and its magnitude reflects the rate of change. In neonatal pain expression analysis, specific curvature gradient patterns are closely related to pain micro-movements. For example, when the brow is furrowed, a gradient vector field converges towards the center of the brow in the glabella area; when the eyes are squeezed, the gradient vector field in the periorbital area converges radially towards the center of the eye; when the nasolabial fold deepens, a parallel gradient band is formed along the nasolabial fold. By identifying these specific gradient field patterns, the system can accurately distinguish pain-related facial micro-movements.

[0081] The mapping relationship between pain micro-movements and curvature characteristics can be established through the following model:

[0082] Frowning: A negative Gaussian curvature peak appears in the glabella area, with the curvature amplitude exceeding the threshold of 0.8. When newborns receive painful stimuli (such as intramuscular injections), the glabella area usually shows significant deformation first, forming a characteristic negative Gaussian curvature distribution.

[0083] Eye compression: The absolute value of the average curvature in the periocular region increases, usually exceeding the threshold of 0.6, accompanied by the curvature gradient converging towards the center of the eye. This pattern is particularly noticeable in cases of persistent pain, such as during the postoperative pain recovery period.

[0084] Deepening of the nasolabial folds: Negative Gaussian curvature bands appear in the nasolabial fold area, forming a specific curvature contour pattern along the direction of the nasolabial folds. In moderate to severe pain, this feature usually occurs simultaneously with furrowed brows and pursed eyes.

[0085] like Figure 5 As shown, the spatiotemporal feature hierarchical analysis module 4 includes a multi-scale decomposition unit 41, a differential invariant extraction unit 42, a feature fusion unit 43, and a micro-motion recognition unit 44. The multi-scale decomposition unit 41 decomposes curvature features into micro-scale, meso-scale, and macro-scale feature representations within different time windows. In a preferred embodiment of the invention, the multi-scale decomposition unit 41 includes a short-time window analysis subunit 411, a medium-time window analysis subunit 412, and a long-time window analysis subunit 413. The short-time window analysis subunit 411 analyzes the minute deformations caused by instantaneous muscle contraction within a 0.1–0.5 second time window; the medium-time window analysis subunit 412 analyzes the complete micro-motion unit expression process within a 0.5–2 second time window; and the long-time window analysis subunit 413 analyzes the combination patterns of multiple micro-motion units within a 2–10 second time window. The multi-scale decomposition employs scale space theory, constructing a time-scale space through Gaussian filters of different widths.

[0086] ,

[0087] in, The scale parameter is The curvature field, denoted as the curvature field after scale-space filtering; For the original curvature field; The standard deviation is The Gaussian kernel function is used for time-domain filtering; * indicates convolution operation; short-time window, medium-time window, and long-time window correspond to... The values ​​are 0.1 seconds, 0.5 seconds, and 2 seconds. Multiscale analysis is particularly important in neonatal pain assessment because different types of pain stimuli elicit facial responses at different time scales. For example, instantaneous pain such as a needle prick causes rapid facial muscle contractions for a short period (0.1–0.5 seconds), while persistent pain (such as abdominal discomfort) manifests as facial expression changes over a longer period (2–10 seconds). Through multiscale analysis, the system can comprehensively capture the facial expression characteristics of different types of pain.

[0088] The differential invariant extraction unit 42 calculates the spatial differential invariants, temporal differential invariants, and spatiotemporal mixture invariants of the curvature field at each scale. Spatial differential invariants include the curvature derivative and tensor field invariants; temporal differential invariants include the curvature-time derivative and curvature acceleration; and spatiotemporal mixture invariants include the combination of spatiotemporal curvature partial derivatives. In an embodiment of the present invention, the temporal differential invariants are calculated as follows:

[0089] ,

[0090] ,

[0091] in, Let C be the first derivative of the curvature field C with respect to time t, representing the rate of change of curvature; Let be the second derivative of the curvature field C with respect to time t, representing the acceleration due to curvature change. The spatiotemporal mixing invariants are calculated as follows:

[0092] ,

[0093] in, Let the curvature field C be about the spatial coordinates The mixed partial derivatives with time t describe the coupled changes of the curvature field in the spatial and temporal dimensions. In the analysis of neonatal pain expressions, differential invariants provide rich dynamic features. For example, the curvature time derivative can reflect the speed of facial expression changes caused by pain stimuli; in acute pain states, the curvature time derivative in the glabella region typically exceeds 2.0 / second. Meanwhile, curvature acceleration can reflect the abruptness of facial expression changes, and is particularly effective in distinguishing between sudden and persistent pain.

[0094] Feature fusion unit 43 adaptively weights and fuses differential invariant features of different scales and dimensions. The fusion method uses a weighted summation approach:

[0095] ,

[0096] in, The fused feature vector integrates multi-scale and multi-dimensional facial expression features; The i-th feature component may come from different scales or different types of differential invariants; is the corresponding weight coefficient, reflecting the importance of the feature component; n is the total number of feature components; This represents the summation over all feature components. Weight coefficients. The Fisher discrimination criterion is used to determine the optimal approach, maximizing inter-class distances and minimizing intra-class distances. In practice, features with high pain discrimination are assigned larger weights (typically between 0.3 and 0.5), while those with low discrimination are assigned smaller weights (typically between 0.1 and 0.2). In neonatal pain assessment systems, the optimal weighting coefficients may differ between preterm and full-term infants. For example, preterm infants typically exhibit a more pronounced furrowed brow, warranting a weight of 0.5; while full-term infants show a more pronounced squint, warranting a weight of 0.4. The system automatically adjusts these weights based on the newborn's gestational age and individual characteristics.

[0097] The micro-motion recognition unit 44 identifies pain-related micro-motion units based on fused features, distinguishing between pain-related micro-motions such as furrowed brows, pursed eyes, and deepened nasolabial folds, and non-pain-related facial expressions. Micro-motion recognition employs a Support Vector Machine (SVM) classifier with a radial basis function (RBF) kernel, parameter γ set to 0.1, and penalty parameter C set to 10. This classifier effectively distinguishes between pain-related micro-motion units (such as furrowed brows, pursed eyes, and deepened nasolabial folds) and non-pain-related facial expressions (such as blinking, opening the mouth, and raising eyebrows), achieving a classification accuracy of over 92%. In practical applications in the NICU, the system accurately identifies facial micro-motions caused by pain stimuli while excluding facial expression changes caused by other factors (such as hunger and fright), significantly reducing the false alarm rate.

[0098] like Figure 6 As shown, the pain assessment and alarm module 5 includes a pain quantification unit 51, a trend analysis unit 52, a threshold adjustment unit 53, and an alarm execution unit 54. The pain quantification unit 51 calculates the frequency, duration, and intensity of micro-motion units to generate a pain score. The pain quantification unit 51 calculates the pain score using a comprehensive scoring formula, where the comprehensive scoring formula weights and sums the frequency, duration, and intensity of the micro-motion units. The pain score calculation formula is as follows:

[0099] ,

[0100] The parameters are: Score (comprehensive pain score), typically ranging from 0 to 1.5, reflecting pain severity; F (frequency of micromotor units), defined as the number of micromotor units detected per unit time; D (duration of micromotor units), defined as the average duration (in seconds) of a micromotor; and I (intensity of micromotor units), determined by the peak value of the curvature field amplitude, typically ranging from 0 to 1. The weighting coefficients for each parameter (0.5, 0.3, 0.2) were determined through clinical validation, reflecting the relative importance of each factor in pain assessment. In the clinical validation study, the frequency factor showed the highest correlation with the clinical pain score (correlation coefficient r = 0.82), thus receiving the highest weight of 0.5; duration was the second highest (r = 0.65), receiving a weight of 0.3; and intensity had a slightly lower correlation (r = 0.58), receiving a weight of 0.2.

[0101] Based on the pain score, the system categorizes pain intensity into four levels:

[0102] Mild pain: 0 ≤ Score < 0.4, corresponding to mild discomfort in clinical practice, which usually does not require special intervention;

[0103] Moderate pain: 0.4≤Score<0.7, corresponding to clinically significant pain, which may require reassurance or non-pharmacological intervention;

[0104] Severe pain: 0.7≤Score<1.0, corresponding to severe pain in clinical practice, usually requiring drug intervention;

[0105] Critical pain: Score ≥ 1.0, corresponding to extreme pain in clinical practice, requiring emergency treatment;

[0106] Trend analysis unit 52 tracks the time-varying trend of pain scores, using a sliding window average and linear regression methods to analyze the pain trend. The sliding window size is typically set to 2 minutes, which effectively filters out short-term fluctuations and captures the trend of pain changes. The slope $k$ of the pain trend can be calculated using the following linear regression:

[0107] ,

[0108] in, The slope represents the linear regression, reflecting the rate of change in pain scores; a positive value indicates increased pain, while a negative value indicates relieved pain. For the first A point in time; For time points Corresponding pain score; The average value over time is calculated as follows: ; The average score is calculated as follows: ; This represents the number of samples within the window. This indicates summing over all sample points within the window. When the slope... When the slope exceeds a threshold of 0.05, the system identifies it as a trend of increasing pain; when the slope... When the value is less than the threshold of -0.05, the system identifies it as a pain relief trend. The threshold of 0.05 was determined through clinical data analysis, which maintains sensitivity while avoiding misjudgments caused by short-term fluctuations. Trend analysis is particularly important in neonatal postoperative monitoring, as it can promptly detect trends of increasing pain and remind medical staff to adjust analgesia protocols.

[0109] The threshold adjustment unit 53 dynamically adjusts the individualized pain threshold based on historical data. The threshold adjustment employs an adaptive algorithm, adjusting the threshold according to the historical distribution of each newborn's pain score. The individualized threshold adjustment formula is as follows:

[0110] ,

[0111] in, These are individualized thresholds applicable to specific newborns; The basic threshold is determined based on population statistics. An adjustment factor (usually ranging from 0.2 to 0.5) is used to control the magnitude of individualized adjustments; This is the individual scoring bias factor, calculated as the proportion of the difference between the mean score of the newborn in a pain-free state and the population mean. In practical applications, the facial expression responses of premature infants and newborns with congenital diseases may differ from those of full-term healthy infants. Individualized threshold adjustment can effectively address these differences. For example, in some premature infants, incomplete facial muscle development may lead to weaker facial expressions; the system will automatically lower the pain threshold (e.g., ...). =0.4, =-0.3); however, for children with certain neurological disorders, who may exhibit abnormally active facial expressions, the system will appropriately increase the threshold.

[0112] When the pain score exceeds a preset threshold, the alarm execution unit 54 triggers an alarm at the corresponding level. The alarm levels correspond to the pain severity, including indicative alarms (mild pain), general alarms (moderate pain), emergency alarms (severe pain), and critical alarms (critical pain). Different alarm levels use different audible and visual cues to ensure timely response by medical staff. Furthermore, the alarm execution unit records the cumulative duration of pain. When the duration of pain at different levels exceeds preset durations (mild > 30 minutes, moderate > 15 minutes, severe > 5 minutes, critical > 1 minute), the system escalates the alarm level. In NICU clinical applications, an appropriate alarm mechanism can balance timely warnings with avoiding excessive alarms, effectively reducing alarm fatigue among medical staff.

[0113] In one embodiment of the present invention, the facial expression manifold construction module 2 and the curvature field analysis module 3 are connected via a feature data transmission interface. This feature data transmission interface is used to transmit facial expression manifold data in real time, supporting synchronous analysis and processing. This interface employs shared memory to achieve efficient data transmission, with latency controlled within 5 milliseconds, ensuring the real-time performance of the system. In cases of acute pain in newborns, facial expressions may change rapidly within hundreds of milliseconds; low-latency data transmission is crucial for accurately capturing these changes.

[0114] This invention provides a pain assessment system for neonatal facial expression image analysis. By introducing differential geometry theory to establish a facial expression manifold model, it captures changes in facial micro-expressions based on curvature field analysis, and combines multi-scale spatiotemporal feature analysis to identify pain-related micro-movements. Ultimately, it achieves objective assessment and timely early warning of neonatal pain status. This system overcomes the limitations of traditional methods, significantly improving the accuracy, objectivity, and real-time performance of neonatal pain assessment, and providing strong technical support for clinical neonatal pain management.

[0115] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A pain assessment system based on neonatal facial expression image analysis, characterized in that, include: The data acquisition module includes an image acquisition unit, an image preprocessing unit, and a feature point localization unit. The image acquisition unit is used to acquire a sequence of facial expression images of newborns. The image preprocessing unit is connected to the image acquisition unit and is used to perform illumination compensation, noise removal, and scale normalization on the facial expression image sequence. The feature point localization unit is connected to the image preprocessing unit and is used to locate key facial feature points, including the glabella, corner of the eye, tip of the nose, and corner of the mouth, in the facial expression image sequence to obtain the coordinates of the facial feature points. A facial expression manifold construction module, connected to the data acquisition module, includes a feature point mapping unit, a Riemann metric construction unit, and an individualized adjustment unit. The feature point mapping unit is used to map the coordinates of facial feature points to a low-dimensional Riemann manifold space using a nonlinear dimensionality reduction method. The Riemann metric construction unit, connected to the feature point mapping unit, is used to define an adaptive Riemann metric tensor in the Riemann manifold space. The adaptive Riemann metric tensor assigns a greater weight coefficient to the brow, periorbital, and nasolabial fold regions than the Riemann metric tensor assigns a greater weight coefficient to other facial regions. The individualized adjustment unit, connected to the Riemann metric construction unit, is used to adjust the manifold parameters based on baseline data under a pain-free state to construct an individualized facial expression manifold model. The curvature field analysis module, connected to the facial expression manifold construction module, is used to calculate the Gaussian curvature and average curvature of the individualized facial expression manifold model, generate a facial curvature field, track the changes of the facial curvature field over time to obtain the dynamic features of the curvature field, identify the curvature patterns of the eyebrow, eye area, and nasolabial fold region, and extract pain-related micro-movement curvature features. The spatiotemporal feature hierarchical analysis module, connected to the curvature field analysis module, is used to decompose the curvature features of the micro-movements into multi-scale components at different time windows, forming feature representations at the micro, meso, and macro scales. It calculates the spatial differential invariants, temporal differential invariants, and spatiotemporal hybrid invariants of the curvature field at each scale, adaptively weights and fuses the differential invariant features of different scales and dimensions, and identifies pain-related micro-movement units, including frowning, eye squinting, and deepening of nasolabial folds, based on the fused features, generating spatiotemporal features of the micro-movement units. The pain assessment and alarm module is connected to the spatiotemporal feature hierarchical analysis module. It is used to calculate the occurrence frequency, duration and intensity of the micro-motion unit based on the spatiotemporal features of the micro-motion unit, generate a pain score by weighted summation of the occurrence frequency, duration and intensity, and execute an alarm operation based on the pain score.

2. The pain assessment system for neonatal facial expression image analysis according to claim 1, characterized in that, The curvature field analysis module includes: The curvature calculation unit is used to calculate the Gaussian curvature and the average curvature of the individualized facial expression manifold model, and generate the facial curvature field. A curvature field evolution unit, connected to the curvature calculation unit, is used to track the change of the facial curvature field over time and obtain the dynamic characteristics of the curvature field; The micro-motion curvature feature extraction unit, connected to the curvature field evolution unit, is used to identify specific curvature patterns in the eyebrow, eye area, and nasolabial fold region, and extract the pain-related micro-motion curvature features.

3. The pain assessment system for neonatal facial expression image analysis according to claim 1, characterized in that, The pain assessment and alarm module includes: The pain quantification unit is used to calculate the frequency, duration, and intensity of the micro-motion units to generate the pain score; A trend analysis unit, connected to the pain quantification unit, is used to track the time-varying trend of the pain score; A threshold adjustment unit, connected to the trend analysis unit, is used to dynamically adjust individualized pain thresholds based on historical data. An alarm execution unit, connected to the threshold adjustment unit, is used to trigger an alarm at the corresponding level when the pain score exceeds the individualized pain threshold.

4. The pain assessment system for neonatal facial expression image analysis according to claim 1, characterized in that, The facial expression manifold construction module and the curvature field analysis module are connected through a feature data transmission interface, which is used to transmit facial expression manifold data in real time and supports synchronous analysis and processing.

5. The pain assessment system for neonatal facial expression image analysis according to claim 1, characterized in that, When performing multi-scale decomposition of the micro-motion curvature features, the spatiotemporal feature hierarchical analysis module analyzes the minute deformations caused by instantaneous muscle contraction within a time window of 0.1 to 0.5 seconds, analyzes the complete expression process of the micro-motion unit within a time window of 0.5 to 2 seconds, and analyzes the combination patterns of multiple micro-motion units within a time window of 2 to 10 seconds.

6. The pain assessment system for neonatal facial expression image analysis according to claim 2, characterized in that, The micro-motion curvature feature extraction unit is further used for: Identify the extreme points and contour lines in the facial curvature field, corresponding to the areas with the most significant facial deformation; Calculate the curvature gradient vector field to describe the direction of the fastest change in facial curvature; Establish a micro-motion curvature feature library to store typical curvature feature patterns related to pain.

7. The pain assessment system for neonatal facial expression image analysis according to claim 3, characterized in that, The pain quantification unit calculates the pain score using a comprehensive scoring formula, which weights and sums the frequency, duration, and intensity of the micro-movement unit. The pain score is divided into four levels: mild, moderate, severe, and critical, to guide clinical intervention measures.

Images