A facial dynamic video image pain assessment method based on a dynamic fusion module

Abstract

The application relates to the technical field of facial dynamic video image pain assessment, in particular to a facial dynamic video image pain assessment method based on a dynamic fusion module, which comprises the following steps: collecting data through a multi-modal sensor and generating a three-dimensional feature mapping atlas; activating an adaptive weight adjustment module to generate a configuration parameter set; performing micro-expression feature extraction and dynamic fusion operation to output a high-precision feature vector; and inputting a hierarchical assessment model to complete pain degree quantification grading. The application can accurately capture facial micro-expression changes, dynamically recover feature distortion, mine nonlinear feature components, improve feature extraction integrity and assessment accuracy, and simultaneously optimize the calculation efficiency by optimizing the dynamic fusion technology through a controlled gradient to enhance the feature capturing capability in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of medical pain assessment technology, and in particular to a method for assessing facial dynamic video images based on a dynamic fusion module. Background Technology

[0002] In the field of modern pain assessment, pain assessment methods based on dynamic facial video images have gradually become a research hotspot due to their non-contact and intuitive nature. These methods, by analyzing changes in facial expressions and combining them with machine learning or deep learning models, can achieve an objective assessment of pain intensity. However, existing related technologies still have certain shortcomings in feature extraction, data fusion, and assessment accuracy, limiting their widespread application in clinical settings. Currently, mainstream processing methods for dynamic facial video images mainly include multi-dimensional feature extraction and global and local feature fusion, but these methods exhibit certain limitations in practical applications. For example, the patent with publication number CN113116299B obtains facial images, nasal and oral sounds, motion parameters, and physiological parameters of a target person, and uses an artificial neural network to extract facial expression feature vectors and nasal and oral sound sound feature vectors, respectively, and then fuses these multi-dimensional features to assess pain intensity. However, this technical solution does not fully consider the dynamic interaction relationships between different features during feature fusion, which may lead to information loss or redundancy, affecting the accuracy of the final assessment results. Furthermore, this method lacks targeted processing of the temporal series characteristics of dynamic video images, making it difficult to effectively capture subtle changes in facial micro-expressions, thus reducing the sensitivity of the assessment. Meanwhile, patent CN114469009B proposes a pain grading assessment model based on a global and local dual-pathway approach. It extracts overall facial features related to pain expressions through a global attention mechanism and weights sub-feature maps through a local attention mechanism to ultimately achieve a graded assessment of pain intensity. However, this approach lacks a dynamic adjustment mechanism in the fusion of global and local features, making it difficult to flexibly adjust weight allocation based on feature changes at different time points in the dynamic facial video image, potentially leading to inaccurate assessment results. In addition, this method is highly dependent on training data; if there is class imbalance in the dataset, it may significantly affect the model's generalization ability. These issues indicate that existing pain assessment methods based on dynamic facial video images still have room for improvement in terms of dynamic feature fusion, capture of temporal series characteristics, and model generalization ability. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for pain assessment of facial dynamic video images based on a dynamic fusion module.

[0004] To solve the above technical problems, the following technical solution is adopted: This invention provides a method for pain assessment of facial dynamic video images based on a dynamic fusion module, comprising the following steps: S1: Acquire dynamic facial video data of the target object through a multimodal sensor array, and generate a three-dimensional feature map based on a spatiotemporal encoder; S2: Based on the dynamic changes of the 3D feature map, activate the adaptive weight adjustment module. Each module generates a set of configuration parameters including time series distribution parameters, spatial attention coefficients, and initial conditions for gradient optimization. S3: Receive the configuration parameter set, perform facial micro-expression feature extraction and dynamic fusion operation. The micro-expression feature extraction is based on the local morphological reconstruction model. The reconstruction parameters are dynamically adjusted by the time series distribution parameters. The dynamic fusion adopts the controlled gradient optimization algorithm to eliminate feature redundancy. The initial gradient of the optimization process is specified by the configuration parameter set. The output is a high-precision feature vector after fusion. S4: Input the high-precision feature vector into the grading assessment model to complete the quantitative grading of pain level.

[0005] Optionally, S1 specifically includes: S11, Multimodal sensor array deployment: Multimodal sensors are distributed across the facial region of the target object to simultaneously acquire dynamic video signals, thermal imaging signals, and texture feature signals from the environment, generating raw multimodal data sequences; S12, Spatiotemporal Feature Encoding: The original multimodal data sequence is timestamped and mapped to spatial coordinates. Temporal dependency features of dynamic video signals are extracted through a pre-trained recurrent neural network (RNN), while spatial distribution features of thermal imaging signals are extracted through a convolutional neural network (CNN). The input layer of the recurrent neural network receives time-series slice data, the input layer of the convolutional neural network receives spatial texture slice data, and the output layer generates a three-dimensional feature tensor including temporal dependency intensity, spatial distribution weights, and texture feature values. S13, Dynamic Topology Modeling: The three-dimensional feature tensor is input into the spatiotemporal encoder. Based on the attention mechanism, the feature propagation path between multimodal sensor nodes is modeled. A three-dimensional feature map is generated by updating through node embedding. The vertices of the three-dimensional feature map represent feature intensity values, and the edge weights represent feature coupling coefficients of adjacent regions. According to the real-time data stream of the sensors, the map is updated using a sliding window mechanism. By comparing the map differences between adjacent windows, resampling of abnormal regions is triggered to ensure that the map refresh rate matches the dynamic change rate of the face.

[0006] Optionally, S2 specifically includes: S21, Dynamic Change Detection and Module Triggering: Real-time monitoring of changes in the 3D feature map and activation of the adaptive weight adjustment module based on trigger conditions: **Intensity Abrupt Change Detection:** When the feature intensity of a sensor changes drastically within a short period, exceeding a preset change threshold, the system identifies the point as an abnormal region and triggers the weight adjustment module. **Coupling Anomaly Detection:** If the feature coupling between two sensors exceeds a set diffusion threshold, indicating that the feature may be propagating in space, the system will take adjustment measures for this region. **Frequency Conflict Detection:** If the main feature frequency of a sensor falls within the operating frequency range of the dynamic video signal, potentially affecting the normality of feature extraction, the system will isolate this conflict.

[0007] S22, Time series distribution parameter calculation: For the activated weight adjustment module, calculate the time series distribution parameters: By analyzing the set of sensors that are disturbed, determine the affected time series, and then generate a continuous parameter curve. The continuous parameter curve represents the time period that the current module should block, so as to avoid the subsequent feature extraction process being affected by the disturbed time period. S23, Dynamic adjustment of spatial attention coefficients: During the adjustment process, the weight allocation of spatial attention is dynamically adjusted: Historical feature data is analyzed through the traditional long short-term memory model to predict the weight coefficients required for current feature extraction. The long short-term memory model receives historical feature intensity data within a predetermined time window, calculates the weight coefficients of the current feature, and uses an activation function to constrain its output. S24, Initial conditions for gradient optimization: Based on the overall feature complexity of the current facial dynamics, initial conditions for gradient optimization are set.

[0008] Optionally, the triggering conditions include: intensity mutation detection: when the feature intensity of a sensor changes drastically in a short period of time and exceeds the intensity change threshold, it will be identified as an abnormal region and the weight adjustment module will be triggered; coupling anomaly detection: if the feature coupling degree between two sensors exceeds the set spatial diffusion threshold, it indicates that the feature is propagating in space, and adjustment measures will be taken for this region; frequency conflict detection: if the main feature frequency of a sensor falls into the working frequency band range of the dynamic video signal, this region will be isolated.

[0009] Optionally, the initial conditions for gradient optimization in S24 include calculating the global feature entropy over the entire monitoring area, evaluating the overall complexity of the features, and determining the initial conditions for the gradient optimization algorithm based on the magnitude of the global feature entropy. The more complex the features, the higher the initial conditions.

[0010] Optionally, S2 further includes parameter set encapsulation and transmission, encapsulating the time series distribution parameters, spatial attention coefficients, and gradient optimization initial conditions into a configuration parameter set, which is then distributed to the corresponding weight adjustment module via a communication bus.

[0011] Optionally, S3 specifically includes: S31, applying a local morphological reconstruction model to eliminate feature distortion caused by illumination or occlusion in dynamic video signals and reconstructing the features; S32, gradient optimization dynamic fusion, fusing the reconstructed features at multiple feature levels to eliminate redundant information, including converting the reconstructed features into a multi-dimensional feature matrix and constructing an optimized energy function to measure the rationality of the feature matrix; calculating the weight energy of each feature, considering the influence of noise, and determining the ideal weight state of the feature level; evaluating the coupling relationship between different features through spatiotemporal correlation coupling strength, and calculating the mutual influence strength between features; using a gradient optimization algorithm to optimize the energy state of the entire feature matrix based on the set initial conditions; S33, obtaining the optimal feature matrix through the gradient optimization process and converting it back into a fused high-precision feature vector.

[0012] Optionally, S31 specifically includes acquiring the original dynamic video signal and pre-extracting typical feature templates through a spatiotemporal encoder; calculating the propagation delay of the feature signal to determine the feature contribution of different sensors to the data source; dynamically adjusting the reconstruction amplitude through time series distribution parameters to construct a local morphological reconstruction model based on convolution operations, generating reconstructed features that are close to the ideal feature morphology under interference-free conditions.

[0013] Optionally, step S33 further includes dividing the fused high-precision feature vector into blocks according to a preset feature standard to form structured features that meet application requirements; calculating the error between the fused features and the ideal features, and ensuring that the error is within a preset fault tolerance threshold, and outputting the final high-precision feature vector.

[0014] Optionally, S4 includes using a graded assessment model to quantify and grade the high-precision feature vector according to pain level grading rules (such as mild, moderate, and severe).

[0015] The technical effects of this invention are as follows: By combining a spatiotemporal encoder with an adaptive weight adjustment module, this invention can accurately capture micro-expression changes in dynamic facial videos and quantify the pain level of the target object based on spatiotemporal feature extraction. In particular, feature reconstruction based on a local morphological reconstruction model can dynamically adjust the feature compensation amplitude. Combined with typical feature templates in the spatiotemporal encoder, it can accurately restore feature distortions caused by dynamic facial changes. Compared with traditional methods based on single feature extraction or static fusion, this invention can more effectively mine nonlinear feature components, ensure the integrity of feature extraction, and improve the accuracy of evaluation. This invention adopts controlled gradient optimization dynamic fusion technology. By constructing an optimized energy function containing single-point feature weight energy and spatiotemporal coupling strength, and combining the characteristics of multimodal sensors and dynamic facial changes, it achieves dynamic correction at the feature level. Compared with traditional fusion algorithms based on fixed weight allocation, this invention can find the optimal feature matrix through a global optimization strategy in complex dynamic environments, improving the robustness of feature fusion. In addition, the adaptive adjustment of initial conditions based on gradient optimization of global feature entropy can enhance the ability to capture complex feature patterns in high dynamic environments and reduce computational overhead and optimize computational efficiency in low dynamic environments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall process of a facial dynamic video image pain assessment method based on a dynamic fusion module provided in an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the three-dimensional feature mapping map generation process in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram illustrating the working principle of the adaptive weight adjustment module in an embodiment of the present invention.

[0019] Figure 4 This is a flowchart illustrating the implementation of the local morphological reconstruction model and gradient optimization dynamic fusion operation in an embodiment of the present invention.

[0020] The attached figures are labeled as follows: 1. Multimodal sensor array; 2. Spatiotemporal encoder; 3. 3D feature map; 4. Adaptive weight adjustment module; 5. Local morphological reconstruction model; 6. Gradient optimization algorithm; 7. High-precision feature vector; 8. Hierarchical evaluation model. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0022] This invention provides a method for pain assessment of facial dynamic video images based on a dynamic fusion module, which is described below in conjunction with the appendix. Figure 1 To be continued Figure 4 The specific embodiments of the present invention will be described in detail below, along with the reference numerals in the accompanying drawings. In this embodiment, the multimodal sensor array 1, the spatiotemporal encoder 2, the three-dimensional feature map 3, the adaptive weight adjustment module 4, the local morphology reconstruction model 5, the gradient optimization algorithm 6, the high-precision feature vector 7, and the hierarchical evaluation model 8 together constitute the core components for realizing the present invention. The connection relationships, positional relationships, and mutual cooperation relationships between the components will be elaborated one by one below.

[0023] First, dynamic facial video data of the target object is acquired using a multimodal sensor array 1. This array consists of multiple distributed sensors, which are used to simultaneously acquire dynamic video signals, thermal imaging signals, and texture feature signals from the environment. The dynamic video signals primarily capture changes in facial expressions, the thermal imaging signals detect changes in facial temperature distribution, and the texture feature signals record detailed information about the facial skin surface. These sensors are distributed in a matrix across the target object's facial area, ensuring coverage of key areas of the entire face, thereby generating the original multimodal data sequence. In practical applications, the sensor density can be adjusted according to the target object's facial features; for example, increasing the number of sensors in key areas such as the bridge of the nose, forehead, and corners of the mouth can improve data acquisition accuracy. The output of the multimodal sensor array 1 is connected to the input of the spatiotemporal encoder 2, and the original multimodal data sequence is transmitted to the spatiotemporal encoder 2 via a communication bus.

[0024] The spatiotemporal encoder 2 receives the raw multimodal data sequence from the multimodal sensor array 1 and performs timestamp alignment and spatial coordinate mapping. Specifically, the temporal dependence features of the dynamic video signal are extracted using a pre-trained recurrent neural network. The input layer of this network receives time-series slice data, and the output layer generates temporal dependence intensity features. Simultaneously, the spatial distribution features of the thermal imaging signal are extracted using a convolutional neural network. The input layer of this network receives spatial texture slice data, and the output layer generates spatial distribution weights and texture feature values. These two sets of features are integrated into a three-dimensional feature tensor through the internal fusion mechanism of the spatiotemporal encoder 2. This tensor contains information in three dimensions: temporal dependence intensity, spatial distribution weights, and texture feature values. Subsequently, the three-dimensional feature tensor is input into the dynamic topology modeling module, which models the feature propagation path between multimodal sensor nodes through an attention mechanism. During this process, node embedding updates generate a three-dimensional feature mapping map 3, where vertices represent feature intensity values ​​and edge weights represent feature coupling coefficients between adjacent regions. To ensure that the 3D feature map 3 can reflect facial dynamic changes in real time, a sliding window mechanism is used to update the map. When the difference between adjacent windows exceeds a preset threshold, a resampling operation for abnormal regions is triggered. The output of the spatiotemporal encoder 2 is connected to the input of the adaptive weight adjustment module 4, and the 3D feature map 3 is transmitted to the adaptive weight adjustment module 4 via a communication bus.

[0025] The adaptive weight adjustment module 4 activates corresponding adjustment operations based on the dynamic changes of the 3D feature map 3. Specifically, when the feature intensity of a sensor changes drastically within a short period of time and exceeds a preset change threshold, the system will identify the point as an abnormal region and trigger the weight adjustment module; if the feature coupling between two sensors exceeds a set spatial diffusion threshold, it indicates that the feature may be propagating in space, and the system will take adjustment measures for this region; if the main feature frequency of a sensor falls within the working frequency range of the dynamic video signal, the system will isolate this region. After the triggering conditions are met, the adaptive weight adjustment module 4 performs operations such as calculating time series distribution parameters, dynamically adjusting spatial attention coefficients, and setting initial conditions for gradient optimization. The time series distribution parameters are determined by analyzing the set of sensors affected by interference to identify the affected time series, thereby generating a continuous parameter curve. This curve represents the time period that the current module should block, avoiding the influence of the interference time period on the subsequent feature extraction process. The dynamic adjustment of spatial attention coefficients is achieved through a traditional long short-term memory model. This model receives historical feature intensity data within a predetermined time window, calculates the weight coefficient of the current feature, and constrains its output through an activation function. The initial conditions for gradient optimization are set based on the calculation results of global feature entropy. Global feature entropy is used to evaluate the overall complexity of the features; the more complex the features, the higher the initial conditions. The adaptive weight adjustment module 4 encapsulates the time series distribution parameters, spatial attention coefficients, and initial conditions for gradient optimization into a configuration parameter set, and distributes it to the corresponding weight adjustment module via the communication bus. The output of the adaptive weight adjustment module 4 is connected to the input of the local morphological reconstruction model 5, and the configuration parameter set is transmitted to the local morphological reconstruction model 5 via the communication bus.

[0026] The local morphological reconstruction model 5 receives a set of configuration parameters from the adaptive weight adjustment module 4 and performs facial micro-expression feature extraction and dynamic fusion operations based on these parameters. First, typical feature templates are pre-extracted using the spatiotemporal encoder 2, and the propagation delay of the feature signals is calculated to determine the feature contributions of different sensors to the data source. Then, the reconstruction amplitude is dynamically adjusted using time-series distribution parameters to construct a local morphological reconstruction model based on convolution operations, generating reconstructed features that approximate the ideal feature morphology under interference-free conditions. After feature reconstruction, the reconstructed features are input into the gradient optimization dynamic fusion module, eliminating redundant information through multi-level feature fusion. Specifically, the reconstructed features are converted into a multi-dimensional feature matrix, and an optimization energy function is constructed to measure the rationality of the feature matrix. The optimization energy function includes two parts: single-point feature weight energy and spatiotemporal correlation coupling strength. The single-point feature weight energy considers the influence of noise and determines the ideal weight state at the feature level, while the spatiotemporal correlation coupling strength is used to evaluate the coupling relationship between different features and calculate the mutual influence strength between features. Based on the set initial conditions, the gradient optimization algorithm 6 optimizes the energy state of the entire feature matrix, ultimately obtaining the optimal feature matrix and converting it back into a fused high-precision feature vector 7. The output of the local morphological reconstruction model 5 is connected to the input of the hierarchical evaluation model 8, and the high-precision feature vector 7 is transmitted to the hierarchical evaluation model 8 through the communication bus.

[0027] The grading assessment model 8 receives high-precision feature vectors 7 from the local morphological reconstruction model 5 and performs quantitative grading according to pain severity grading rules. Specifically, the grading assessment model 8 classifies the high-precision feature vectors 7 into three levels: mild, moderate, and severe, based on indicators such as the amplitude, frequency distribution, and spatial distribution of the feature vectors. To ensure the accuracy of the grading results, the grading assessment model 8 also incorporates preset feature standards, dividing the fused high-precision feature vectors 7 into blocks to form structured features that meet application requirements, and calculating the error between the fused features and the ideal features to ensure that the error is within a preset error tolerance threshold. The output of the grading assessment model 8 can be used in practical applications such as medical diagnosis and pain management.

[0028] In the above implementation process, the connections and positions of the various components work closely together to ensure the efficient operation of the entire system. For example, the arrangement density and position of the multimodal sensor array 1 directly affect the input data quality of the spatiotemporal encoder 2, while the three-dimensional feature map 3 generated by the spatiotemporal encoder 2 provides dynamically changing basic data for the adaptive weight adjustment module 4. The configuration parameter set generated by the adaptive weight adjustment module 4 further guides the feature reconstruction operation of the local morphological reconstruction model 5, and finally achieves feature fusion and outputs a high-precision feature vector 7 through the gradient optimization algorithm 6. The graded assessment model 8 then completes the quantitative grading of pain level based on the high-precision feature vector 7. The collaborative relationship between the components runs through the entire process, ensuring the integrity and accuracy from data acquisition to pain assessment.

[0029] To enable those skilled in the art to fully understand and implement this invention, the specific implementation principle of this invention will be further explained below in conjunction with a specific application scenario.

[0030] In practical applications, assuming the target is a patient who needs to monitor their pain level during postoperative recovery, the facial dynamic video image pain assessment method based on a dynamic fusion module provided by this invention can quantify and classify the pain level in real time and accurately.

[0031] First, during the deployment phase of the multimodal sensor array 1, technicians adjust the sensor distribution density based on the patient's facial features. For example, the number of sensors is increased in key areas such as the bridge of the nose, forehead, and corners of the mouth to ensure that subtle changes in facial expressions and temperature fluctuations can be captured. The sensors in the multimodal sensor array 1 simultaneously acquire dynamic video signals, thermal imaging signals, and texture feature signals, and transmit these data to the spatiotemporal encoder 2. During this process, the dynamic video signals record changes in the patient's facial expressions, the thermal imaging signals detect abnormal fluctuations in facial temperature distribution, and the texture feature signals capture subtle changes in skin surface details, such as deepening wrinkles or muscle tension.

[0032] Subsequently, the spatiotemporal encoder 2 performs timestamp alignment and spatial coordinate mapping on the received raw multimodal data sequence. Specifically, the temporal dependence features of the dynamic video signal are extracted through a pre-trained recurrent neural network, which receives time-series slice data and generates temporal dependence intensity features; the spatial distribution features of the thermal imaging signal are extracted through a convolutional neural network, outputting spatial distribution weights and texture feature values. These two sets of features are integrated into a three-dimensional feature tensor through the internal fusion mechanism of the spatiotemporal encoder 2, which contains information in three dimensions: temporal dependence intensity, spatial distribution weights, and texture feature values. Next, the three-dimensional feature tensor is input into the dynamic topology modeling module, which models the feature propagation path between multimodal sensor nodes through an attention mechanism, generating a three-dimensional feature mapping map 3. During this process, a sliding window mechanism updates the three-dimensional feature mapping map 3 in real time. When the difference between adjacent windows exceeds a preset threshold, an abnormal region resampling operation is triggered, thereby ensuring that the mapping map can accurately reflect the dynamic changes of the patient's face.

[0033] Next, the adaptive weight adjustment module 4 activates corresponding adjustment operations based on the dynamic changes of the 3D feature map 3. For example, when the feature intensity of a sensor changes drastically in a short period of time and exceeds a preset change threshold, the system identifies the point as an abnormal region and triggers the weight adjustment module; if the feature coupling degree between two sensors exceeds the set spatial diffusion threshold, it indicates that the feature may be propagating in space, and the system takes adjustment measures for this region; if the main feature frequency of a sensor falls into the working frequency band range of the dynamic video signal, the system will isolate this region. After the triggering conditions are met, the adaptive weight adjustment module 4 performs operations such as calculating time series distribution parameters, dynamically adjusting spatial attention coefficients, and setting initial conditions for gradient optimization. The time series distribution parameters are determined by analyzing the set of sensors affected by interference to identify the affected time series and generating continuous parameter curves to shield the interference time periods; the dynamic adjustment of spatial attention coefficients is achieved through a traditional long short-term memory model, which receives historical feature intensity data within a predetermined time window, calculates the weight coefficient of the current feature, and constrains its output through an activation function; the setting of initial conditions for gradient optimization is based on the calculation results of global feature entropy, and the more complex the feature, the higher the initial conditions. Finally, the adaptive weight adjustment module 4 encapsulates the time series distribution parameters, spatial attention coefficients, and gradient optimization initial conditions into a configuration parameter set, and transmits it to the local morphological reconstruction model 5 via the communication bus.

[0034] The local morphological reconstruction model 5 receives a set of configuration parameters from the adaptive weight adjustment module 4 and performs facial micro-expression feature extraction and dynamic fusion operations based on these parameters. First, typical feature templates are pre-extracted using the spatiotemporal encoder 2, and the propagation delay of the feature signals is calculated to determine the feature contributions of different sensors to the data source. Then, the reconstruction amplitude is dynamically adjusted using time-series distribution parameters to construct a local morphological reconstruction model based on convolution operations, generating reconstructed features that approximate the ideal feature morphology under interference-free conditions. After feature reconstruction, the reconstructed features are input into the gradient optimization dynamic fusion module, eliminating redundant information through multi-level feature fusion. Specifically, the reconstructed features are converted into a multi-dimensional feature matrix, and an optimization energy function is constructed to measure the rationality of the feature matrix. The optimization energy function includes two parts: single-point feature weight energy and spatiotemporal correlation coupling strength. The single-point feature weight energy considers the influence of noise and determines the ideal weight state at the feature level, while the spatiotemporal correlation coupling strength is used to evaluate the coupling relationship between different features and calculate the mutual influence strength between features. Based on the set initial conditions, the gradient optimization algorithm 6 optimizes the energy state of the entire feature matrix, ultimately obtaining the optimal feature matrix and converting it back into a fused high-precision feature vector 7.

[0035] Finally, the grading assessment model 8 receives the high-precision feature vector 7 from the local morphological reconstruction model 5 and completes the quantitative grading according to the pain severity grading rules. Specifically, the grading assessment model 8 classifies the high-precision feature vector 7 into three levels: mild, moderate, and severe, based on indicators such as the amplitude, frequency distribution, and spatial distribution of the feature vector. To ensure the accuracy of the grading results, the grading assessment model 8 also incorporates preset feature standards, dividing the fused high-precision feature vector 7 into blocks to form structured features that meet application requirements, and calculating the error between the fused features and the ideal features to ensure that the error is within a preset error tolerance threshold. The output of the grading assessment model 8 can be used in practical applications such as medical diagnosis and pain management.

[0036] In the above implementation process, the connections and positions of the various components work closely together to ensure the efficient operation of the entire system. For example, the arrangement density and position of the multimodal sensor array 1 directly affect the input data quality of the spatiotemporal encoder 2, while the three-dimensional feature map 3 generated by the spatiotemporal encoder 2 provides dynamically changing basic data for the adaptive weight adjustment module 4. The configuration parameter set generated by the adaptive weight adjustment module 4 further guides the feature reconstruction operation of the local morphological reconstruction model 5, and finally achieves feature fusion and outputs a high-precision feature vector 7 through the gradient optimization algorithm 6. The graded assessment model 8 then completes the quantitative grading of pain level based on the high-precision feature vector 7. The collaborative relationship between the components runs through the entire process, ensuring the integrity and accuracy from data acquisition to pain assessment.

[0037] As can be seen from the implementation steps of the specific application scenarios described above, this invention, by introducing core components such as a multimodal sensor array, a spatiotemporal encoder, an adaptive weight adjustment module, and a local morphological reconstruction model, achieves accurate extraction and dynamic fusion of pain-related features in dynamic facial video images. Particularly in the application of the local morphological reconstruction model, the design of dynamically adjusting the reconstruction amplitude and optimizing the energy function effectively eliminates feature distortion caused by changes in illumination or occlusion, thereby improving the sensitivity and accuracy of pain assessment. Furthermore, the gradient optimization initial condition adaptive adjustment strategy based on global feature entropy further enhances the system's robustness in complex dynamic environments while optimizing computational efficiency, providing reliable technical support for pain assessment in clinical settings.

[0038] The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to solve essentially the same technical problems and achieve essentially the same technical effects are all covered within the protection scope of the present invention.

Claims

1. A method for pain assessment of facial dynamic video images based on a dynamic fusion module, characterized in that, Includes the following steps: S1: Acquire dynamic facial video data of the target object through a multimodal sensor array, and generate a three-dimensional feature map based on a spatiotemporal encoder; S1 specifically includes: S11: Multimodal sensor array deployment: Multimodal sensors are distributed across the facial region of the target object to simultaneously acquire dynamic video signals, thermal imaging signals, and texture feature signals from the environment, generating raw multimodal data sequences; S12: Spatiotemporal feature encoding: The original multimodal data sequence is timestamped and mapped to spatial coordinates. The temporal dependence features of the dynamic video signal are extracted through a pre-trained recurrent neural network, and the spatial distribution features of the thermal imaging signal are extracted through a convolutional neural network. The output layer generates a three-dimensional feature tensor including temporal dependence intensity, spatial distribution weight and texture feature value. S13: Dynamic topology modeling: Input the three-dimensional feature tensor into the spatiotemporal encoder, model the feature propagation path between multimodal sensor nodes based on the attention mechanism, and generate a three-dimensional feature map through node embedding and updating. The vertices of the three-dimensional feature map represent feature intensity values, and the edge weights represent feature coupling coefficients of adjacent regions. S2: Based on the dynamic changes of the 3D feature map, activate the adaptive weight adjustment module. Each module generates a set of configuration parameters including time series distribution parameters, spatial attention coefficients, and initial conditions for gradient optimization. S3: Receive the configuration parameter set, perform facial micro-expression feature extraction and dynamic fusion operation, where micro-expression feature extraction is based on the local morphological reconstruction model, and dynamic fusion uses a controlled gradient optimization algorithm to eliminate feature redundancy, and outputs a high-precision feature vector after fusion. S4: Input the high-precision feature vector into the grading assessment model to complete the quantitative grading of pain level.

2. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 1, characterized in that, S2 specifically includes: S21: Dynamic Change Detection and Module Triggering: Real-time monitoring of changes in the 3D feature map and activation of the adaptive weight adjustment module based on triggering conditions; S22: Calculation of time series distribution parameters: For the activated weight adjustment module, analyze the set of sensors affected by the disturbance, determine the affected time series, and generate continuous parameter curves; S23: Dynamic adjustment of spatial attention coefficients: By analyzing historical feature data through traditional long short-term memory models, the weight coefficients required for current feature extraction are predicted, and the output is constrained by activation functions; S24: Initial conditions for gradient optimization: Based on the calculation results of global feature entropy, set the initial conditions for the gradient optimization algorithm.

3. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 2, characterized in that, The triggering conditions include: Intensity mutation detection: When the characteristic intensity of a sensor changes drastically in a short period of time, exceeding the intensity change threshold, it is identified as an abnormal region and the weight adjustment module is triggered; Coupling anomaly detection: If the feature coupling degree between two sensors exceeds the set spatial diffusion threshold, it indicates that the feature is propagating in space, and adjustment measures are taken for this area; Frequency conflict detection: If the main characteristic frequency of a sensor falls within the operating frequency range of a dynamic video signal, this area is isolated.

4. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 2, characterized in that, The initial conditions for gradient optimization in S24 include calculating the global feature entropy over the entire monitoring area and determining the initial conditions for the gradient optimization algorithm based on the magnitude of the global feature entropy.

5. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 2, characterized in that, The S2 also includes parameter set encapsulation and transmission, which encapsulates the time series distribution parameters, spatial attention coefficients and gradient optimization initial conditions into a configuration parameter set, and distributes the configuration parameter set to the corresponding weight adjustment module through the communication bus.

6. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 1, characterized in that, S3 specifically includes: S31: Application of Local Morphology Reconstruction Model: Acquire raw dynamic video signals, pre-extract typical feature templates through a spatiotemporal encoder, calculate the propagation delay of feature signals, determine the feature contribution of different sensors to the data source, construct a local morphology reconstruction model based on convolution operations, and generate reconstructed features; S32: Gradient optimization dynamic fusion: The reconstructed features are converted into a multi-dimensional feature matrix, an optimization energy function is constructed to measure the rationality of the feature matrix, the energy of single-point feature weights and the spatiotemporal correlation coupling strength are calculated, and the gradient optimization algorithm is used to optimize the energy state of the entire feature matrix. S33: Obtain the optimal feature matrix through the gradient optimization process and convert it back into the fused high-precision feature vector.

7. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 6, characterized in that, S33 further includes dividing the fused high-precision feature vector into blocks according to a preset feature standard to form structured features that meet application requirements, calculating the error between the fused features and the ideal features, ensuring that the error is within a preset fault tolerance threshold, and outputting the final high-precision feature vector.

8. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 1, characterized in that, The S4 includes using a graded assessment model to classify high-precision feature vectors according to pain level grading rules. The classification criteria include the amplitude, frequency distribution, and spatial distribution indicators of the feature vectors.

9. The facial dynamic video image pain assessment method based on a dynamic fusion module according to claim 1, characterized in that, The multimodal sensor array includes dynamic video signal sensors, thermal imaging signal sensors, and texture feature signal sensors. Each sensor is distributed in a matrix on the facial region of the target object, and the sensor density is adjusted according to the facial features of the target object.

Images