Face dynamic video image pain assessment method based on dynamic fusion module

Abstract

The invention relates to the technical field of facial dynamic video image pain assessment, in particular to a facial dynamic video image pain assessment method based on a dynamic fusion module, and the method comprises the steps: collecting data through a multi-modal sensor, and generating a three-dimensional feature mapping map; activating an adaptive weight adjustment module to generate a configuration parameter set; executing micro-expression feature extraction and dynamic fusion operation to output a high-precision feature vector; inputting a grading evaluation model to complete pain degree quantitative grading. According to the method, facial micro-expression changes can be accurately captured, feature distortion can be dynamically recovered, nonlinear feature components can be mined, feature extraction integrity and evaluation accuracy can be improved, meanwhile, the feature capture capability in a complex environment can be enhanced through a controlled gradient optimization dynamic fusion technology, and the calculation efficiency can be optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical pain assessment, and in particular to a facial dynamic video image pain assessment method based on a dynamic fusion module. BACKGROUND

[0002] In the field of modern pain assessment, pain assessment methods based on facial dynamic video images have gradually become a research hotspot due to their non-contact and intuitive nature. Such methods can achieve objective assessment of pain intensity by analyzing facial expression changes combined with machine learning or deep learning models. However, existing related technical solutions still have certain deficiencies in feature extraction, data fusion, and evaluation accuracy, limiting their widespread application in clinical environments. Currently, the mainstream processing methods for facial dynamic video images mainly include multi-dimensional feature extraction, global and local feature fusion, etc., but these methods have certain limitations in practical application: for example, the patent with publication number CN113116299B obtains facial images, mouth and nose sounds, action parameters, and physiological parameters of the target person, uses an artificial neural network to extract facial image expression feature vectors and mouth and nose sound feature vectors, and fuses these multi-dimensional features to assess pain intensity. However, this technical solution does not fully consider the dynamic interaction between different features during feature fusion, which may result in information loss or redundancy, affecting the accuracy of the final evaluation results. In addition, this method lacks targeted processing of the time sequence characteristics of dynamic video images, making it difficult to effectively capture subtle changes in facial micro-expressions, thereby reducing the sensitivity of the evaluation. Meanwhile, the patent with publication number CN114469009B proposes a pain grading evaluation model based on global and local dual channels, which extracts overall features related to facial and pain expressions through a global attention mechanism, and weights the sub-feature maps through a local attention mechanism, finally realizing the grading evaluation of pain intensity. However, this technical solution lacks a dynamic adjustment mechanism in the fusion process of global and local features, making it difficult to flexibly adjust the weight distribution according to the feature changes at different time points in the facial dynamic video image, resulting in evaluation results that may not be accurate enough. In addition, this method is highly dependent on training data, and if there is a class imbalance problem in the data set, it may significantly affect the generalization ability of the model. The above problems show that the existing pain assessment methods based on facial dynamic video images still have certain room for improvement in feature dynamic fusion, time sequence characteristic capture, and model generalization ability. SUMMARY

[0003] The present application aims to overcome the deficiencies of the prior art and provide a facial dynamic video image pain assessment method based on a dynamic fusion module.

[0004] To solve the above technical problems, the following technical solutions are adopted: The application provides a face dynamic video image pain assessment method based on a dynamic fusion module, comprising the following steps: S1: acquiring dynamic face video data of a target object through a multi-modal sensor array, and generating a three-dimensional feature mapping atlas based on a space-time encoder; S2: activating an adaptive weight adjustment module according to the dynamic changes of the three-dimensional feature mapping atlas, and each module generating a configuration parameter set comprising a time series distribution parameter, a spatial attention coefficient and a gradient optimization initial condition; S3: receiving the configuration parameter set, and performing face micro-expression feature extraction and dynamic fusion operation, wherein the micro-expression feature extraction is based on a local morphology reconstruction model, the reconstruction parameter is dynamically adjusted by the time series distribution parameter, the dynamic fusion adopts a controlled gradient optimization algorithm to eliminate feature redundancy, the initial gradient of the optimization process is specified by the configuration parameter set, and a high-precision feature vector after fusion is output; S4: inputting the high-precision feature vector into a hierarchical evaluation model to complete quantitative grading of the pain degree.

[0005] Optionally, the S1 specifically comprises: S11, multi-modal sensor array deployment: distributing multi-modal sensors in the face area of the target object, synchronously collecting dynamic video signals, thermal imaging signals and texture feature signals in the environment, and generating original multi-modal data sequences; S12, space-time feature coding: performing time stamp alignment and space coordinate mapping on the original multi-modal data sequences, extracting time-dependent features of the dynamic video signals through a pre-trained recurrent neural network (RNN), and extracting spatial distribution features of the thermal imaging signals through a convolutional neural network (CNN), wherein the input layer of the recurrent neural network receives time series slice data, the input layer of the convolutional neural network receives spatial texture slice data, and the output layer generates a three-dimensional feature tensor comprising time-dependent intensity, spatial distribution weight and texture feature value; S13, dynamic topology modeling: inputting the three-dimensional feature tensor into the space-time encoder, modeling the feature propagation path between multi-modal sensor nodes based on an attention mechanism, generating a three-dimensional feature mapping atlas through node embedding update, wherein the vertex of the three-dimensional feature mapping atlas represents the feature intensity value, and the edge weight represents the feature coupling coefficient of adjacent regions; according to the real-time data stream of the sensor, the mapping atlas is updated by using a sliding window mechanism, the abnormal area resampling is triggered by comparing the atlas difference degrees of adjacent windows, and the mapping atlas refresh rate is matched with the face dynamic change rate.

[0006] Optionally, the S2 specifically comprises: S21, dynamic change detection and module triggering: real-time monitoring of the change of the three-dimensional feature mapping, and activating the adaptive weight adjustment module according to the triggering condition: intensity mutation detection: when the feature intensity of a certain sensor changes dramatically in a short time, exceeding the preset change threshold, the system will identify the point as an abnormal area and trigger the weight adjustment module. Coupling anomaly detection: if the coupling degree between two sensors exceeds the set diffusion threshold, indicating that the feature may be spreading in space, the system will take adjustment measures for this area. Frequency conflict detection: if the main feature frequency of a certain sensor falls within the working frequency range of the dynamic video signal, it may affect the normality of feature extraction, and the system will isolate this area.

[0007] S22, time series distribution parameter calculation: for the activated weight adjustment module, calculate the time series distribution parameter: by analyzing the disturbed sensor set, determine the affected time series, and then generate a continuous parameter curve, which represents the time period that the current module should shield, to avoid the influence of the disturbance period on the subsequent feature extraction process; S23, dynamic adjustment of spatial attention coefficient: during the adjustment process, dynamically adjust the weight distribution of spatial attention: analyze historical feature data through a traditional long short-term memory model to predict the weight coefficient required for current feature extraction. The long short-term memory model receives historical feature intensity data within a predetermined time window, calculates the weight coefficient of the current feature, and uses an activation function to constrain its output; S24, setting of initial conditions for gradient optimization: based on the overall feature complexity of the current face dynamics, set the initial conditions for gradient optimization.

[0008] Optionally, the triggering condition includes: intensity mutation detection: when the feature intensity of a certain sensor changes dramatically in a short time, exceeding the intensity change threshold, it will be identified as an abnormal area and trigger the weight adjustment module; coupling anomaly detection: if the coupling degree between two sensors exceeds the set spatial diffusion threshold, indicating that the feature is spreading in space, that is, taking adjustment measures for this area; frequency conflict detection: if the main feature frequency of a certain sensor falls within the working frequency range of the dynamic video signal, isolate this area for processing.

[0009] Optionally, the initial conditions for gradient optimization in S24 include calculating the global feature entropy in the entire monitoring area, evaluating the overall complexity of the feature, and determining the initial conditions for the gradient optimization algorithm according to the size of the global feature entropy. The more complex the feature, the higher the initial condition.

[0010] Optionally, S2 further includes parameter set packaging and transmission, which packages the time series distribution parameter, the spatial attention coefficient and the gradient optimization initial condition into a configuration parameter set, and distributes the configuration parameter set to the corresponding weight adjustment module through the communication bus.

[0011] Optionally, the S3 specifically includes: S31, a local morphological reconstruction model application, eliminating feature distortion caused by light or occlusion in the dynamic video signal, and reconstructing the feature; S32, gradient optimization dynamic fusion, fusing the reconstructed features at multiple feature levels to eliminate redundant information, including converting the reconstructed features into a multi-dimensional feature matrix, and constructing an optimization energy function to measure the rationality of the feature matrix; calculating the weight energy of each feature, considering the influence of noise, and determining the ideal weight state of the feature level; evaluating the coupling relationship between different features through the spatiotemporal correlation coupling strength, and calculating the mutual influence strength between features; using a gradient optimization algorithm, based on the set initial conditions, to optimize the energy state of the entire feature matrix; S33, through the gradient optimization process, obtaining the optimal feature matrix, and converting it back to the fused high-precision feature vector.

[0012] Optionally, the S31 specifically includes collecting the original dynamic video signal, and pre-extracting the typical feature template through the spatiotemporal encoder; calculating the propagation time delay of the feature signal, determining the feature contribution of different sensors to the data source; adjusting the reconstruction amplitude dynamically through the time series distribution parameter, constructing a local morphological reconstruction model based on convolution operation, generating the reconstructed feature, and making it close to the ideal feature morphology under the condition of no interference.

[0013] Optionally, the S33 further includes dividing the fused high-precision feature vector into blocks according to the pre-set feature standard, forming a structured feature that meets the application requirements; calculating the error between the fused feature and the ideal feature, and ensuring that the error is within the pre-set fault tolerance threshold, outputting the final high-precision feature vector.

[0014] Optionally, the S4 includes using a hierarchical evaluation model to quantitatively grade the high-precision feature vector according to the pain degree grading rules (such as mild, moderate, and severe).

[0015] The technical effect of the present application: the present application can accurately capture the micro-expression changes in the facial dynamic video by combining the spatio-temporal encoder with the adaptive weight adjustment module, and quantitatively evaluate the pain degree of the target object based on the spatio-temporal feature extraction, especially the feature reconstruction based on the local morphological reconstruction model, which can dynamically adjust the feature compensation amplitude, and combine the typical feature template in the spatio-temporal encoder to realize the accurate recovery of the feature distortion caused by the facial dynamic change. Compared with the traditional method based on single feature extraction or static fusion, the present application can more effectively mine the nonlinear feature components, ensure the integrity of feature extraction, and improve the accuracy of evaluation. The present application adopts a controlled gradient optimization dynamic fusion technology, constructs an optimization energy function containing single-point feature weight energy and spatio-temporal coupling strength, combines the characteristics of multi-modal sensors and facial dynamic changes, realizes dynamic correction at the feature level, and compared with the traditional fusion algorithm based on fixed weight distribution, the present application can find the optimal feature matrix through a global optimization strategy in a complex dynamic environment, improve the robustness of feature fusion, and further, based on the gradient optimization initial condition adaptive adjustment of global feature entropy, can enhance the capture ability of complex feature mode in a high dynamic environment, and reduce the calculation overhead in a low dynamic environment, and optimize the calculation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The overall flowchart of the facial dynamic video image pain assessment method based on the dynamic fusion module provided by the embodiment of the present application is shown.

[0017] Figure 2 The structure diagram of the three-dimensional feature mapping atlas generation process in the embodiment of the present application is shown.

[0018] Figure 3 The working principle diagram of the adaptive weight adjustment module in the embodiment of the present application is shown.

[0019] Figure 4 The implementation flowchart of the local morphological reconstruction model and the gradient optimization dynamic fusion operation in the embodiment of the present application is shown.

[0020] The reference signs are as follows: 1, multi-modal sensor array; 2, spatio-temporal encoder; 3, three-dimensional feature mapping atlas; 4, adaptive weight adjustment module; 5, local morphological reconstruction model; 6, gradient optimization algorithm; 7, high-precision feature vector; 8, hierarchical evaluation model. DETAILED DESCRIPTION

[0021] For the purposes of the present invention, the technical solutions and advantages will be more clearly and specifically understood, the following drawings and examples will be used to further illustrate the present invention. However, it should be understood that the specific examples described herein are only used to explain the present invention, and are not used to limit the scope of the present invention. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0022] The present application provides a dynamic fusion module-based facial dynamic video image pain assessment method. The following will combine the drawings and examples to further illustrate the present invention. However, it should be understood that the specific examples described herein are only used to explain the present invention, and are not used to limit the scope of the present invention. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention. Figure 1 to the drawings Figure 4 The specific embodiments of the present application are described in detail in the drawings and the reference numerals in the drawings. In the present embodiment, the multi-modal sensor array 1, the space-time encoder 2, the three-dimensional feature mapping atlas 3, the adaptive weight adjustment module 4, the local morphology reconstruction model 5, the gradient optimization algorithm 6, the high-precision feature vector 7 and the hierarchical evaluation model 8 together constitute the core components of the present application. The connection relationship, position relationship and mutual cooperation relationship between the components will be described one by one in the following.

[0023] First, the multi-modal sensor array 1 collects dynamic facial video data of the target object. The array is composed of a plurality of distributed sensors, which are used to synchronously collect dynamic video signals, thermal imaging signals and texture feature signals in the environment. The dynamic video signal mainly captures the change of facial expression, the thermal imaging signal is used to detect the change of facial temperature distribution, and the texture feature signal records the detailed information of the surface of the facial skin. These sensors are distributed in the form of a matrix in the facial region of the target object, ensuring that the key parts of the entire face are covered, thereby generating a raw multi-modal data sequence. In actual application, the arrangement density of the sensors can be adjusted according to the facial features of the target object, for example, the number of sensors in the key areas such as the nose bridge, forehead and corners of the mouth can be increased to improve the data acquisition accuracy. The output end of the multi-modal sensor array 1 is connected to the input end of the space-time encoder 2, and the raw multi-modal data sequence is transmitted to the space-time encoder 2 through the communication bus.

[0024] The spatio-temporal encoder 2 receives the raw multi-modal data sequence from the multi-modal sensor array 1 and performs timestamp alignment and spatial coordinate mapping. Specifically, the time-dependent features of the dynamic video signal are extracted by a pre-trained recurrent neural network, whose input layer receives the time-series slice data and output layer generates the time-dependent intensity features. Meanwhile, the spatial distribution features of the thermal imaging signal are extracted by a convolutional neural network, whose input layer receives the spatial texture slice data and output layer generates the spatial distribution weights and texture feature values. The above two parts of features are integrated into a three-dimensional feature tensor through the internal fusion mechanism of the spatio-temporal encoder 2, which contains information of three dimensions of time-dependent intensity, spatial distribution weight and texture feature value. Subsequently, the three-dimensional feature tensor is input into the dynamic topology modeling module, and the feature propagation path between the nodes of the multi-modal sensor is modeled through the attention mechanism. In this process, the node embedding update generates a three-dimensional feature mapping atlas 3, whose vertices represent the feature intensity values and edge weights represent the feature coupling coefficients of adjacent regions. In order to ensure that the three-dimensional feature mapping atlas 3 can reflect the dynamic changes of the face in real time, a sliding window mechanism is used to update the atlas, and when the atlas difference of adjacent windows exceeds the preset threshold, the abnormal region resampling operation is triggered. The output end of the spatio-temporal encoder 2 is connected with the input end of the adaptive weight adjustment module 4, and the three-dimensional feature mapping atlas 3 is transmitted to the adaptive weight adjustment module 4 through the communication bus.

[0025] The adaptive weight adjustment module 4 activates the corresponding adjustment operation according to the dynamic changes of the three-dimensional feature mapping atlas 3. Specifically, when the feature intensity of a certain sensor changes dramatically in a short time and exceeds the preset change threshold, the system will identify this point as an abnormal area and trigger the weight adjustment module; if the coupling degree of features between two sensors exceeds the set spatial diffusion threshold, it indicates that the features may be spreading in space, and the system will take adjustment measures for this area; if the main feature frequency of a certain sensor falls within the working frequency range of the dynamic video signal, the system will isolate this area. After the trigger condition is met, the adaptive weight adjustment module 4 performs the operations of time series distribution parameter calculation, spatial attention coefficient dynamic adjustment, and gradient optimization initial condition setting. The time series distribution parameter determines the affected time series by analyzing the set of disturbed sensors, and then generates a continuous parameter curve, which represents the time period that the current module should shield to avoid the influence of the interference period on the subsequent feature extraction process. The dynamic adjustment of the spatial attention coefficient is realized through the traditional long short-term memory model, which receives the historical feature intensity data within the predetermined time window, calculates the weight coefficient of the current feature, and constrains its output through the activation function. The setting of the gradient optimization initial condition is based on the calculation result of the global feature entropy, which is used to evaluate the overall complexity of the feature. The more complex the feature, the higher the initial condition. The adaptive weight adjustment module 4 encapsulates the time series distribution parameter, the spatial attention coefficient, and the gradient optimization initial condition into a configuration parameter set, and distributes it to the corresponding weight adjustment module through the communication bus. The output end of the adaptive weight adjustment module 4 is connected to the input end of the local morphological reconstruction model 5, and the configuration parameter set is transmitted to the local morphological reconstruction model 5 through the communication bus.

[0026] The local morphological reconstruction model 5 receives the configuration parameter set from the adaptive weight adjustment module 4 and performs facial micro-expression feature extraction and dynamic fusion operation based on this. First, the typical feature template is pre-extracted by the spatio-temporal encoder 2, and the propagation time delay of the feature signal is calculated to determine the feature contribution of different sensors to the data source. Then, the reconstruction amplitude is dynamically adjusted using the time series distribution parameter, and the local morphological reconstruction model based on convolution operation is constructed to generate the reconstructed features close to the ideal feature morphology under the condition of no interference. After the feature reconstruction is completed, the reconstructed features are input into the gradient optimization dynamic fusion module to eliminate redundant information through multi-level feature level fusion. Specifically, the reconstructed features are converted into a multi-dimensional feature matrix, and an optimization energy function is constructed to measure the rationality of the feature matrix. The optimization energy function includes single-point feature weight energy and spatio-temporal correlation coupling strength, wherein the single-point feature weight energy considers the influence of noise and determines the ideal weight state of the feature level, and the spatio-temporal correlation coupling strength is used to evaluate the coupling relationship between different features and calculate the mutual influence strength between features. Based on the set initial conditions, the gradient optimization algorithm 6 is used to optimize the energy state of the entire feature matrix, and finally the optimal feature matrix is obtained and converted back to the fused high-precision feature vector 7. The output end of the local morphological reconstruction model 5 is connected to the input end of the hierarchical evaluation model 8, and the high-precision feature vector 7 is transmitted to the hierarchical evaluation model 8 through the communication bus.

[0027] The hierarchical evaluation model 8 receives the high-precision feature vector 7 from the local morphological reconstruction model 5 and completes the quantitative grading according to the pain degree grading rules. Specifically, the hierarchical evaluation model 8 classifies the high-precision feature vector 7 into three levels of mild, moderate and severe according to the amplitude, frequency distribution and spatial distribution of the feature vector. In order to ensure the accuracy of the grading results, the hierarchical evaluation model 8 also combines the pre-set feature standard to form a structured feature that meets the application requirements by blocking the fused high-precision feature vector 7, and calculates the error between the fused feature and the ideal feature to ensure that the error is within the pre-set fault tolerance threshold. The output result of the hierarchical evaluation model 8 can be used in practical application scenarios such as medical diagnosis and pain management.

[0028] In the above implementation process, the connection relationship and position relationship between each component are closely matched to ensure the efficient operation of the entire system. For example, the arrangement density and position of the multi-modal sensor array 1 directly affect the input data quality of the spatio-temporal encoder 2, while the three-dimensional feature mapping atlas 3 generated by the spatio-temporal encoder 2 provides dynamic changing basic data for the adaptive weight adjustment module 4. The configuration parameter set generated by the adaptive weight adjustment module 4 further guides the feature reconstruction operation of the local morphology reconstruction model 5, and finally realizes feature fusion and outputs high-precision feature vector 7 through the gradient optimization algorithm 6. The hierarchical evaluation model 8 completes the quantitative grading of the pain degree based on the high-precision feature vector 7. The cooperative relationship between each component runs through the entire process, ensuring the integrity and accuracy from data acquisition to pain assessment.

[0029] In order to better enable relevant persons in the art to fully understand and implement the present application, the specific implementation principles of the present application are supplemented in the following in conjunction with one specific application scenario.

[0030] In actual application, it is assumed that the target object is a patient who needs to monitor the pain degree for postoperative recovery. Through the facial dynamic video image pain assessment method based on the dynamic fusion module provided by the present application, the pain degree of the patient can be quantitatively graded in real time and accurately.

[0031] Firstly, in the deployment stage of the multi-modal sensor array 1, the technical personnel adjusts the distribution density of the sensors according to the facial features of the patient. For example, the number of sensors is increased in key areas such as the nose bridge, forehead and corners of the mouth to ensure that subtle expression changes and temperature fluctuations can be captured. The sensors in the multi-modal sensor array 1 synchronously collect dynamic video signals, thermal imaging signals and texture feature signals, and transmit these data to the spatio-temporal encoder 2. In this process, the dynamic video signals record the changes of the patient's facial expressions, the thermal imaging signals detect abnormal fluctuations in the temperature distribution of the face, and the texture feature signals capture changes in the details of the skin surface, such as wrinkle deepening or muscle tension.

[0032] Subsequently, the spatio-temporal encoder 2 performs timestamp alignment and spatial coordinate mapping processing on the received original multi-modal data sequence. Specifically, the time-dependent features of the dynamic video signal are extracted by a pre-trained recurrent neural network, which receives time series slice data and generates time-dependent intensity features; the spatial distribution features of the thermal imaging signal are extracted by a convolutional neural network, outputting spatial distribution weights and texture feature values. The above two parts of features are integrated into a three-dimensional feature tensor through the internal fusion mechanism of the spatio-temporal encoder 2, which contains information of three dimensions of time-dependent intensity, spatial distribution weight and texture feature value. Then, the three-dimensional feature tensor is input into the dynamic topology modeling module to model the feature propagation path between multi-modal sensor nodes through the attention mechanism, generating a three-dimensional feature mapping atlas 3. In this process, the sliding window mechanism updates the three-dimensional feature mapping atlas 3 in real time, and when the atlas difference of adjacent windows exceeds the preset threshold, the abnormal region resampling operation is triggered, so as to ensure that the mapping atlas can accurately reflect the dynamic changes of the patient's face.

[0033] Next, the adaptive weight adjustment module 4 activates the corresponding adjustment operation according to the dynamic changes of the three-dimensional feature mapping atlas 3. For example, when the feature intensity of a certain sensor changes dramatically in a short time and exceeds the preset change threshold, the system identifies this point as an abnormal region and triggers the weight adjustment module; if the feature coupling degree between two sensors exceeds the set spatial diffusion threshold, it indicates that the feature may be spatially propagated, and the system takes adjustment measures for this region; if the main feature frequency of a certain sensor falls within the working frequency range of the dynamic video signal, the system will isolate this region. After the trigger condition is met, the adaptive weight adjustment module 4 performs time series distribution parameter calculation, spatial attention coefficient dynamic adjustment, and gradient optimization initial condition setting operations. The time series distribution parameter determines the affected time series by analyzing the set of disturbed sensors, generates a continuous parameter curve to shield the interference time period; the dynamic adjustment of the spatial attention coefficient is realized through the traditional long short-term memory model, which receives the historical feature intensity data within the predetermined time window, calculates the weight coefficient of the current feature and constrains its output through the activation function; the setting of the gradient optimization initial condition is based on the calculation result of the global feature entropy, the more complex the feature, the higher the initial condition. Finally, the adaptive weight adjustment module 4 encapsulates the time series distribution parameter, the spatial attention coefficient and the gradient optimization initial condition into a configuration parameter set, and transmits it to the local morphology reconstruction model 5 through the communication bus.

[0034] The local morphological reconstruction model 5 receives the configuration parameter set from the adaptive weight adjustment module 4 and performs facial micro-expression feature extraction and dynamic fusion operations based on it. First, the typical feature template is pre-extracted by the space-time encoder 2, and the propagation time delay of the feature signal is calculated to determine the feature contribution of different sensors to the data source. Then, the reconstruction amplitude is dynamically adjusted using the time series distribution parameters, and a local morphological reconstruction model based on convolution operation is constructed to generate the reconstructed features close to the ideal feature morphology under the condition of no interference. After the feature reconstruction is completed, the reconstructed features are input into the gradient optimization dynamic fusion module to eliminate redundant information through multi-level feature level fusion. Specifically, the reconstructed features are converted into a multi-dimensional feature matrix, and an optimization energy function is constructed to measure the rationality of the feature matrix. The optimization energy function includes single-point feature weight energy and space-time correlation coupling strength, where the single-point feature weight energy considers the influence of noise and determines the ideal weight state of the feature level, and the space-time correlation coupling strength is used to evaluate the coupling relationship between different features and calculate the mutual influence strength between features. Based on the set initial conditions, the gradient optimization algorithm 6 is used to optimize the energy state of the entire feature matrix, and finally the optimal feature matrix is obtained and converted back to the fused high-precision feature vector 7.

[0035] Finally, the hierarchical evaluation model 8 receives the high-precision feature vector 7 from the local morphological reconstruction model 5 and completes the quantitative grading according to the pain degree grading rules. Specifically, the hierarchical evaluation model 8 classifies the high-precision feature vector 7 into three levels of mild, moderate and severe, and the classification criteria include amplitude, frequency distribution and spatial distribution of the feature vector. In order to ensure the accuracy of the grading results, the hierarchical evaluation model 8 also combines the pre-set feature standards to form structured features that meet the application requirements by blocking the fused high-precision feature vector 7, and calculates the error between the fused features and the ideal features to ensure that the error is within the pre-set fault tolerance threshold. The output results of the hierarchical evaluation model 8 can be used in practical application scenarios such as medical diagnosis and pain management.

[0036] In the above implementation process, the connection relationship and position relationship between the components are closely matched to ensure the efficient operation of the entire system. For example, the arrangement density and position of the multi-modal sensor array 1 directly affect the input data quality of the space-time encoder 2, and the three-dimensional feature mapping atlas 3 generated by the space-time encoder 2 provides dynamic changing basic data for the adaptive weight adjustment module 4. The configuration parameter set generated by the adaptive weight adjustment module 4 further guides the feature reconstruction operation of the local morphological reconstruction model 5, and finally realizes feature fusion through the gradient optimization algorithm 6 and outputs the high-precision feature vector 7. The hierarchical evaluation model 8 completes the quantitative grading of the pain degree based on the high-precision feature vector 7. The cooperation relationship between the components runs through the entire process, ensuring the integrity and accuracy from data acquisition to pain assessment.

[0037] As can be seen from the implementation steps of the above specific application scenarios, the present application realizes the accurate extraction and dynamic fusion of pain-related features in facial dynamic video images by introducing core components such as a multi-modal sensor array, a space-time encoder, an adaptive weight adjustment module, and a local morphological reconstruction model. In particular, in the application of the local morphological reconstruction model, by dynamically adjusting the reconstruction amplitude and optimizing the design of the energy function, the feature distortion problem caused by changes in illumination or occlusion is effectively eliminated, thereby improving the sensitivity and accuracy of pain assessment. In addition, the gradient optimization initial condition adaptive adjustment strategy based on global feature entropy further enhances the robustness of the system in complex dynamic environments, while optimizing the computational efficiency, providing reliable technical support for pain assessment in clinical environments.

[0038] The above is only a specific embodiment of the present application, but the technical features of the present application are not limited thereto. Any simple change, equivalent replacement or modification made on the basis of the present application to solve the basically same technical problem and realize the basically same technical effect is covered within the protection scope of the present application.

Claims

1. A dynamic facial video image pain assessment method based on a dynamic fusion module, characterized in that, The method comprises the following steps: S1: acquiring dynamic facial video data of a target object by a multi-modal sensor array, and generating a three-dimensional feature mapping atlas based on a space-time encoder; S2: activating an adaptive weight adjustment module according to the dynamic changes of the three-dimensional feature mapping atlas, each module generating a configuration parameter set comprising time series distribution parameters, spatial attention coefficients and gradient optimization initial conditions; S3: receiving the configuration parameter set, performing facial micro-expression feature extraction and dynamic fusion operation, wherein the micro-expression feature extraction is based on a local morphological reconstruction model, and the dynamic fusion adopts a controlled gradient optimization algorithm to eliminate feature redundancy, and outputs a fused high-precision feature vector; S4: inputting the high-precision feature vector into a hierarchical evaluation model to complete the quantitative grading of the pain degree.

2. The dynamic fusion module based facial dynamic video image pain assessment method according to claim 1, wherein, The S1 specifically comprises: S11: multi-modal sensor array deployment: distributing multi-modal sensors in the facial region of the target object, synchronously acquiring dynamic video signals, thermal imaging signals and texture feature signals in the environment, and generating original multi-modal data sequences; S12: space-time feature coding: time stamp alignment and space coordinate mapping are performed on the original multi-modal data sequences, time-dependent features of the dynamic video signals are extracted through a pre-trained recurrent neural network, spatial distribution features of the thermal imaging signals are extracted through a convolutional neural network, and a three-dimensional feature tensor comprising time-dependent intensity, spatial distribution weight and texture feature value is generated in the output layer; S13: dynamic topology modeling: inputting the three-dimensional feature tensor into the space-time encoder, modeling the feature propagation path between the multi-modal sensor nodes based on the attention mechanism, and generating a three-dimensional feature mapping atlas through node embedding update, wherein the vertices of the three-dimensional feature mapping atlas represent feature intensity values, and the edge weights represent feature coupling coefficients of adjacent regions. 3.The dynamic fusion module based facial dynamic video image pain assessment method according to claim 1, wherein, The S2 specifically comprises: S21: dynamic change detection and module triggering: real-time monitoring of the change of the three-dimensional feature mapping atlas, and activating the adaptive weight adjustment module according to the triggering condition; S22: time series distribution parameter calculation: for the activated weight adjustment module, analyzing the disturbed sensor set, determining the affected time series, and generating a continuous parameter curve; S23: dynamic adjustment of spatial attention coefficients: analyzing historical feature data through a traditional long short-term memory model, predicting the weight coefficients required for current feature extraction, and constraining the output thereof by using an activation function; S24: setting of gradient optimization initial conditions: setting the initial conditions of the gradient optimization algorithm based on the calculation results of the global feature entropy.

4. The dynamic fusion module based facial dynamic video image pain assessment method according to claim 3, wherein, The triggering conditions comprise: intensity mutation detection: when the feature intensity of a sensor changes dramatically within a short time and exceeds the intensity change threshold, it is identified as an abnormal region and the weight adjustment module is triggered; coupling anomaly detection: if the feature coupling degree between two sensors exceeds the set spatial diffusion threshold, it indicates that the feature is propagating in space, and adjustment measures are taken for this region; frequency conflict detection: if the main feature frequency of a sensor falls within the working frequency range of the dynamic video signal, the region is isolated.

5. The dynamic fusion module based facial dynamic video image pain assessment method according to claim 3, wherein, The gradient optimization initial condition in the S24 comprises calculating a global feature entropy in the whole monitoring area, and determining the initial condition of the gradient optimization algorithm according to the size of the global feature entropy.

6. The dynamic fusion module based facial dynamic video image pain assessment method according to claim 3, wherein, The S2 further comprises parameter set packaging and transmission, packaging the time series distribution parameters, the spatial attention coefficient and the gradient optimization initial condition into a configuration parameter set, and distributing the configuration parameter set to the corresponding weight adjustment module through a communication bus.

7. The dynamic fusion module based facial dynamic video image pain assessment method according to claim 1, wherein, The S3 specifically comprises: S31: local morphological reconstruction model application: collecting an original dynamic video signal, pre-extracting a typical feature template through a space-time encoder, calculating a propagation time delay of the feature signal, determining a feature contribution of different sensors to a data source, constructing a local morphological reconstruction model based on convolution operation, and generating a reconstructed feature; S32: gradient optimization dynamic fusion: converting the reconstructed feature into a multi-dimensional feature matrix, constructing an optimization energy function to measure the rationality of the feature matrix, calculating a single-point feature weight energy and a space-time correlation coupling strength, and optimizing the energy state of the whole feature matrix by using a gradient optimization algorithm; S33: obtaining an optimal feature matrix through the gradient optimization process, and converting the optimal feature matrix back into a fused high-precision feature vector.

8. The dynamic fusion module based facial dynamic video image pain assessment method according to claim 7, wherein, The S33 further comprises blocking the fused high-precision feature vector according to a preset feature standard, forming a structured feature meeting the application requirements, calculating an error between the fused feature and an ideal feature, and ensuring that the error is within a preset fault tolerance threshold, and outputting a final high-precision feature vector. 9.The dynamic fusion module based facial dynamic video image pain assessment method according to claim 1, wherein, The S4 comprises using a hierarchical evaluation model to classify the high-precision feature vector according to a pain degree grading rule, and the classification basis comprises an amplitude, a frequency distribution and a spatial distribution index of the feature vector.

10. The dynamic fusion module based facial dynamic video image pain assessment method according to claim 1, wherein, The multi-modal sensor array comprises a dynamic video signal sensor, a thermal imaging signal sensor and a texture feature signal sensor, each sensor is distributed in a face area of a target object in a matrix form, and the arrangement density of the sensor is adjusted according to the face features of the target object.

Images