Biological characteristic time sequence monitoring system based on micro expression-motion collaborative analysis

Abstract

The invention discloses a biological characteristic time sequence monitoring system based on micro expression-motion collaborative analysis, and relates to the technical field of biological characteristic time sequence information processing. The biological characteristic time sequence monitoring system based on micro expression-motion collaborative analysis comprises a biological characteristic signal quality monitoring module; a multi-modal biological characteristic association monitoring module; and a multi-mode time sequence characteristic accuracy monitoring module. According to the method, whether biological characteristic signal quality monitoring optimization is adopted or not is judged by performing biological characteristic signal quality primary monitoring, then multi-modal biological characteristic correlation analysis and multi-modal time sequence characteristic fusion are performed, and finally whether timestamp alignment adjustment is adopted or not is determined based on a multi-modal time sequence characteristic accuracy judgment result. The effect of improving the accuracy of biological characteristic time sequence monitoring based on micro-expression and motion collaborative analysis is achieved, and the problem that in the prior art, the accuracy of biological characteristic time sequence monitoring based on micro-expression and motion collaborative analysis is low is solved.

Description

Technical Field

[0001] This invention relates to the field of biometric temporal information processing technology, and in particular to a biometric temporal monitoring system based on micro-expression-motion co-analysis. Background Technology

[0002] With the aging population, the incidence of neurodegenerative diseases (such as Parkinson's disease) is rising year by year. Early symptoms of these diseases are often subtle and insidious (e.g., fine tremors in the hands in early Parkinson's). Traditional monitoring methods rely on single-modal data and primarily on physician observation, making it difficult to comprehensively reflect an individual's true physiological and emotional state. This leads to biases in the results of biometric time-series monitoring. The overall process of biometric time-series monitoring is as follows: First, multimodal dynamic biometric data (such as facial micro-expression sequences, eye fixation trajectories, and blink frequency) is collected through the coordinated deployment of multimodal sensors (e.g., high-definition cameras to capture facial micro-expression images, eye-tracking devices to record eye movements). Next, the collected raw data is preprocessed (e.g., non-local mean filtering, data denoising, data synchronization, and standardization) to obtain standardized multimodal data. Subsequently, feature extraction is performed on the preprocessed multimodal standardized data to obtain a multidimensional modal feature set. Then, a hybrid strategy combining feature layer fusion and decision layer fusion is adopted to input the multidimensional modal feature set (such as micro-expression features, motion features, etc.) into a preset recognition model (such as a neural network, attention mechanism fusion model, etc.) for collaborative analysis to obtain a multimodal feature association map and early disease risk probability (such as early warning probability of Parkinson's disease, etc.). Finally, dynamic modeling is performed on the fused feature data based on time series models (such as hidden Markov models, conditional random fields, etc.) to identify the trend and abnormal patterns of biometrics over time. The monitoring results are then presented through intuitive charts (such as time series feature curves, etc.) and visualization interfaces, and based on the monitoring results, early warning, dynamic tracking of disease progression, and quantitative evaluation of treatment effectiveness are carried out.

[0003] In the process of video micro-expression recognition, existing technologies first filter out image frames containing micro-expressions from user video data, then select a number of consecutive frames equal to the empirical frame values ​​to form a micro-expression sequence, then call the weight calculation layer to calculate the image feature vector of each frame in the sequence combined with the weight value, sum these feature vectors to obtain a comprehensive image feature vector, and then input it into a preset micro-expression recognition model (such as a convolutional neural network) to obtain the micro-expression recognition result, and finally call the event processing micro-expression strategy to obtain the corresponding event processing flow information.

[0004] For example, Chinese invention patent application CN120600318A discloses a multimodal artificial intelligence-based psychological assessment method and device, which includes: First, acquiring a video of a user's facial micro-expressions, quantifying the level of privacy interference through regional pixel segmentation, and blurring non-sensitive areas while retaining the features of key micro-expression regions; simultaneously, acquiring the user's speech spectrum waveform and reconstructing the voiceprint pitch and fundamental frequency waveform; then, extracting emotional keywords from the text, constructing a cross-stage emotional association chain to repair semantic breaks caused by local desensitization; subsequently, dynamically adjusting the fusion weight of video and speech according to the level of privacy interference, generating stage evaluation parameters and time series labels; finally, constructing a psychological state evolution map based on the above data, and outputting a psychological assessment report.

[0005] For example, the Chinese invention patent with announcement number CN119418915B discloses a training method, system, and assessment method for a Parkinson's disease assessment model, which includes: first, acquiring multi-dimensional training data related to Parkinson's disease (such as clinical indicators, imaging data, physiological signals, etc.); then, preprocessing the acquired training data (such as data cleaning, labeling, and format standardization) to remove noisy data to ensure data quality; then, based on the preprocessed training data, constructing the network architecture of the Parkinson's disease assessment model (such as a deep learning model), and iteratively training the model using training data; finally, inputting relevant data of the object to be assessed into the Parkinson's disease assessment model, and outputting assessment results such as the risk of Parkinson's disease and the stage of the disease.

[0006] The above-mentioned technology has at least the following technical problems:

[0007] In the process of early biometric time-series monitoring, existing technologies rely heavily on clinical symptom assessment (such as physician observation), imaging examinations, and gene monitoring, resulting in low accuracy in the collaborative correlation and fusion of multi-dimensional data. For example, the lack of effective collaborative correlation and fusion of multi-dimensional data such as micro-expression features (e.g., facial stiffness), speech signal features (e.g., speech rate), and gait features (e.g., reduced stride length) leads to low accuracy in the results of multi-dimensional data fusion. Consequently, when conducting early biometric time-series monitoring based on low-accuracy multi-dimensional data fusion results, it may be unable to capture the potential correlations between multi-modal features (e.g., the synchronicity of increased micro-movement frequency, speech pauses, and reduced stride length in early Parkinson's disease). Due to the lack of potential correlations among multi-modal features, the accuracy of disease assessment is low when conducting early biometric time-series monitoring of neurodegenerative diseases with subtle and hidden early symptoms (e.g., Parkinson's disease). There is also the problem of low accuracy in biometric time-series monitoring based on micro-expression and motor synergistic analysis. Summary of the Invention

[0008] To address the low accuracy of biometric temporal monitoring based on micro-expression and motion co-analysis in existing technologies, this invention provides a biometric temporal monitoring system based on micro-expression-motion co-analysis, comprising: a biometric signal quality monitoring module, a multimodal biometric association monitoring module, and a multimodal temporal feature accuracy monitoring module. The biometric signal quality monitoring module performs primary biometric signal quality monitoring during early biometric temporal monitoring to assess the quality of biometric signal acquisition. Based on the results of the primary biometric signal quality monitoring, it determines whether biometric signal quality monitoring optimization is necessary. Biometric signal quality monitoring optimization improves the clarity of key regions in micro-expression images and the accuracy of motion feature acquisition, reduces noise interference and data fluctuations, and ensures the accuracy of biometric signals. To ensure the reliability of the signal, the multimodal biometric correlation monitoring module is used to perform multimodal biometric correlation analysis to mine potential correlations between biometrics and non-motor biometrics after the initial monitoring of biometric signal quality is qualified. After the multimodal biometric correlation analysis is completed, multimodal time series feature fusion is performed to integrate the time series features of various biometrics and reduce information redundancy. The multimodal time series feature accuracy monitoring module is used to perform multimodal time series feature accuracy judgment to verify the temporal consistency, stability and modal correlation of the fused multimodal time series features after the multimodal time series feature fusion is completed. Based on the multimodal time series feature accuracy judgment result, it is determined whether timestamp alignment adjustment is needed. The timestamp alignment adjustment is used to correct the acquisition time deviation of different modal time series features and improve the temporal consistency of multimodal biometrics.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0010] 1. By conducting primary monitoring of biosignal signal quality, and determining whether further optimization is needed based on the results, it helps to screen qualified biosignal signals from the source, reducing interference from low-quality and invalid data in the subsequent collaborative fusion process. This lays the foundation for accurate correlation and fusion of multi-dimensional data. After passing the primary monitoring, multimodal biosignal correlation analysis helps to uncover potential correlations between multi-dimensional features such as biological temporal features and biological physiological function temporal features, reducing the problem of missing correlations caused by ineffective collaborative correlation of multi-dimensional data in existing technologies. After the multimodal biosignal correlation analysis, multimodal temporal feature fusion helps to integrate the advantages of each modality, reduce information redundancy, and improve the accuracy of multi-dimensional data collaborative fusion results. After multimodal temporal feature fusion, multimodal temporal feature accuracy judgment is performed. Based on the results, it is determined whether timestamp alignment adjustment is needed, which helps to correct the time deviation of different modal temporal features, ensure the temporal consistency of fused features, and improve the accuracy of disease assessment during early biosignal temporal monitoring.

[0011] 2. By selectively using micro-expression image sequence signal-to-noise ratio, micro-motion signal quality quantification value, and dynamic equilibrium signal quality quantification value as multi-dimensional biometric signal quality quantification indicators, this approach comprehensively captures signal acquisition quality status across different dimensions, including the intensity of effective micro-expression signals, the proportion of effective micro-motion information, and the acquisition accuracy of dynamic equilibrium signals. This reduces the limitations of existing technologies that rely solely on single-dimensional indicators, making it difficult to comprehensively reflect the quality fluctuations of biometric signals under different acquisition scenarios. Furthermore, the lack of continuous monitoring and screening mechanisms for low-quality signals leads to their direct entry into subsequent analysis processes. By using multi-dimensional quantification indicators to determine the suitability of biometric signal quality monitoring, combined with signal quality monitoring counter over-limit discrimination, this approach helps to specifically address the subsequent analysis bias caused by low-quality biometric signals. It reduces the number of persistently low-quality signals entering subsequent analysis stages, screening qualified biometric signals from the source, providing a high-quality and stable data foundation for subsequent analysis stages, and ensuring the accuracy and reliability of the final disease assessment results.

[0012] 3. By selectively choosing a multimodal temporal matching index, a multimodal temporal stability index, and a modal correlation correction coefficient to construct a multimodal temporal feature consistency evaluation index, this approach comprehensively captures the core quality dimensions of multimodal temporal features. This reduces the limitations of existing technologies that rely solely on a single dimension for evaluation, which can only cover a portion of quality issues. It comprehensively considers the combined effects of multiple factors on biometric quality, avoiding evaluation biases caused by single parameters or unweighted calculations. This makes the multimodal temporal feature consistency evaluation index more objectively reflect the overall accuracy of multimodal temporal features. It determines whether the multimodal temporal feature consistency evaluation index exceeds a preset temporal consistency threshold. If so, a biometric temporal anomaly alarm is issued; otherwise, timestamp alignment adjustments are performed. This helps to specifically address the issues of asynchronous multimodal feature temporal data and insufficient correlation caused by timestamp discrepancies, preventing misjudgments or missed judgments caused by using low-accuracy features directly for anomaly alarms, and ensuring the accuracy of subsequent biometric temporal anomaly alarms. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the structure of the biometric time-series monitoring system based on micro-expression-motion co-analysis provided in an embodiment of the present invention;

[0015] Figure 2 This is a flowchart summarizing the overall process of the biometric time-series monitoring system based on micro-expression-motion co-analysis provided in this embodiment of the invention.

[0016] Figure 3 This is an optimization logic diagram for biosignal signal quality monitoring of a biosignal time-series monitoring system based on micro-expression-motion co-analysis provided in this embodiment of the invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] like Figure 1 The diagram shown is a structural schematic of a biometric time-series monitoring system based on micro-expression-motion co-analysis provided in an embodiment of the present invention. The biometric time-series monitoring system based on micro-expression-motion co-analysis includes: a biometric signal quality monitoring module, a multimodal biometric association monitoring module, and a multimodal time-series feature accuracy monitoring module.

[0019] The biometric signal quality monitoring module is used during early biometric time-series monitoring to perform primary biometric signal quality monitoring. This monitoring assesses the quality of biometric signal acquisition and reduces the strong interference of low-quality signals on subsequent analysis. Based on the results of the primary biometric signal quality monitoring, it determines whether biometric signal quality monitoring optimization is needed. Biometric signal quality monitoring optimization improves the clarity of key areas in micro-expression images and the accuracy of motion feature acquisition, reduces noise data interference and data fluctuations, and ensures the reliability of biometric signals. By evaluating the results of primary biometric signal quality monitoring, it helps to screen qualified biometric signals from the source, reduce the impact of noise and data fluctuations on subsequent analysis, and provide high-quality and highly reliable basic data input for multimodal biometric correlation analysis and fusion.

[0020] The multimodal biometric correlation monitoring module is used to perform multimodal biometric correlation analysis to uncover potential correlations between biometrics and non-motor biometrics after the initial biometric signal quality monitoring is qualified. After the multimodal biometric correlation analysis is completed, multimodal temporal feature fusion is performed to integrate the temporal features of various biometrics and reduce information redundancy. Through multimodal biometric correlation analysis and multimodal temporal feature fusion, it is helpful to uncover the synergistic change patterns between the temporal features of biometrics and the temporal features of biophysiological functions, improve the fusion accuracy of multidimensional data, and provide a fusion feature foundation with correlation value for subsequent accurate multimodal temporal feature discrimination and biometric anomaly alarm.

[0021] The multimodal temporal feature accuracy monitoring module is used to determine the accuracy of multimodal temporal features after the fusion of multimodal temporal features. Based on the determination result, it determines whether timestamp alignment adjustment is needed. Timestamp alignment adjustment is used to correct the time deviation of different modal temporal features and improve the temporal consistency of multimodal biometric features. By monitoring the multimodal temporal feature accuracy determination result, it helps to correct the time synchronization problem of multimodal features in a timely manner, reduce the insufficient feature accuracy caused by time deviation, ensure the accuracy of subsequent biometric time anomaly alarms, and reduce misjudgments or missed judgments caused by biometric quality problems.

[0022] It should be understood that the biometric time-series monitoring system based on micro-expression-motion co-analysis provided in this application pre-constructs a database storing various preset data during the design phase. The data sources of this database include basic preset values ​​such as preset image sequence signal-to-noise ratio thresholds, preset micro-motion thresholds, and preset time-series consistency thresholds directly configured by technicians, as well as reference datasets compiled through historical biometric monitoring cases (such as multimodal feature data and monitoring records of patients with different neurodegenerative diseases), clinical trial data (such as threshold verification results in multi-center biometric simulation monitoring), and expert experience calibration data (such as suggested values ​​of biometric abnormality alarm thresholds by medical experts). This provides clinical practice support for the rationality of the preset values. Its storage adopts an encrypted distributed architecture, combining a relational database to store structured preset parameters (such as the specific values ​​of various thresholds and the range of modality weight coefficients) and a non-relational database to store unstructured reference data (such as historical biometric time-series maps and micro-expression image sequences). Technicians can dynamically calibrate the preset values ​​based on newly accumulated biometric monitoring data and clinical feedback, so that the stored data always adapts to the needs of actual biometric time-series monitoring scenarios.

[0023] In this embodiment, a comprehensive monitoring system is constructed, encompassing biosignal quality control, multimodal biosignal association monitoring, and multimodal temporal feature accuracy monitoring. This system extends from biosignal signal quality control to multimodal association fusion and temporal accuracy verification. This enhances the robustness of the early biosignal temporal monitoring system and the accuracy of assessing latent diseases. Specifically, the biosignal signal quality monitoring module provides high-quality, low-noise basic biosignal signals to the multimodal biosignal association monitoring module, reducing the impact of low-quality data on association analysis and fusion results. The multimodal biosignal association monitoring module mines and integrates features to output fusion features with collaborative value to the multimodal temporal feature accuracy monitoring module. The judgment results of the multimodal temporal feature accuracy monitoring module ensure the accuracy of subsequent biosignal anomaly alarms, forming a closed-loop optimization mechanism. This ensures the reliability and timeliness of early biosignal temporal monitoring in assessing subtle and latent neurodegenerative diseases (such as Parkinson's disease).

[0024] like Figure 2 The diagram shown is a general overview flowchart of the biometric time-series monitoring system based on micro-expression-motion co-analysis provided in this embodiment of the invention. Figure 2It can be seen that during the early biometric time-series monitoring process, primary monitoring of biometric signal quality is performed, and quantitative indicators of biometric signal quality are obtained to determine whether the biometric signal quality monitoring results are qualified. If so, multimodal biometric correlation analysis is performed; otherwise, signal quality monitoring counter over-limit judgment is performed to determine whether the signal quality monitoring counter value exceeds the preset maximum threshold. If not, primary monitoring of biometric signal quality continues; otherwise, biometric signal quality monitoring optimization is implemented. After the biometric signal quality monitoring optimization is completed, the biometric signal quality monitoring results are determined to be qualified. If not, a biometric signal quality monitoring report is sent. If the optimization fails, an early warning is issued; otherwise, multimodal biometric correlation analysis is performed. After the multimodal biometric correlation analysis, multimodal temporal feature fusion is performed. After the multimodal temporal feature fusion, the accuracy of the multimodal temporal features is judged, and the consistency evaluation index of the multimodal temporal features is obtained. It is determined whether the consistency evaluation index of the multimodal temporal features is greater than the preset temporal consistency threshold. If it is, a biometric temporal anomaly alarm is issued; otherwise, timestamp alignment adjustment is performed. After the timestamp alignment adjustment is completed, it is determined whether the accuracy judgment of the multimodal temporal features is qualified. If it is, a biometric temporal anomaly alarm is issued; otherwise, a timestamp alignment adjustment failure warning is sent.

[0025] Furthermore, the specific process of primary monitoring of biometric signal quality is as follows: Obtaining biometric signal quality quantification indicators for evaluating the quality of biometric signal acquisition. These indicators include the signal-to-noise ratio (SNR) of the micro-expression image sequence, reflecting the intensity of effective feature signals in the micro-expression image sequence; the quality quantification value of the micro-motion signal, reflecting the proportion of effective information and data reliability in the micro-motion signal; and the quality quantification value of the dynamic equilibrium signal, reflecting the acquisition accuracy and data fluctuation amplitude of the biometric dynamic equilibrium signal. Based on these biometric signal quality quantification indicators, a qualification judgment is made regarding the biometric signal quality monitoring. The SNR of the micro-expression image sequence is calculated by dividing the average pixel intensity of the preset key feature region in the micro-expression image frame by the average pixel intensity of the background noise region within the image frame during a preset feature acquisition time period. After quantization, the results are accumulated and represented by the ratio quantization of the number of micro-expression image frames. Among them, the preset key feature regions are pre-set by preset personnel. The average pixel intensity is represented by quantizing the ratio of the gray values ​​of all pixels monitored by the high-definition image acquisition camera in the preset key feature regions (such as the eye area and the corner of the mouth) to the total number of pixels in the preset key feature regions. The average pixel intensity of the background noise region of the image frame is represented by quantizing the ratio of the gray values ​​of all pixels in the background region without biological feature information (such as the blank area of ​​the cheek and the image edge region) in the image frame obtained by the adaptive threshold segmentation algorithm (such as the adaptive Gaussian threshold method) to the total number of pixels in the background region. The ratio quantization is then performed by ratio calculation.

[0026] Specifically, the micro-motion signal quality quantization value is represented by the result of weighted coupling processing of the effective micro-motion signal quality quantization parameters and the effective micro-motion signal quality influence parameters within a preset feature acquisition time period. The weighted coupling processing involves multiplication. The effective micro-motion signal quality quantization parameters represent the micro-motion signal quality quantization parameters that meet the micro-motion feature correlation verification conditions. These parameters include the micro-motion tremor signal signal-to-noise ratio (SNR), which reflects the degree of separation between the effective tremor features and environmental noise in the micro-motion tremor signal, and the micro-motion force signal sampling accuracy, which reflects the accuracy of the micro-motion force signal sampling data. Effective micro-motion signal quality impact parameters include the vibration signal-to-noise ratio (SNR) impact coefficient, which reflects the influence of the micro-motion vibration signal SNR on the quantized value of the micro-motion signal quality, and the force signal sampling accuracy impact coefficient, which reflects the influence of the micro-motion sensor sampling accuracy on the quantized value of the micro-motion signal quality. The micro-motion vibration signal SNR is represented by the ratio of the peak voltage of the micro-motion vibration signal monitored by the micro-skin-attached accelerometer to the peak voltage of the environmental noise signal within a preset characteristic acquisition time period. The micro-motion force signal sampling accuracy is represented by the ratio of the peak voltage of the micro-motion force signal monitored by the micro-skin-attached accelerometer to the peak voltage of the environmental noise signal within a preset characteristic acquisition time period. The result of quantifying the ratio of the total number of valid sampled values ​​to the total number of micro-motion force signal sampled values ​​is represented. The valid sampled value of the micro-motion force signal represents the micro-motion force signal monitored by the pressure sensor that meets the valid discrimination condition of the micro-motion force signal. The valid discrimination condition of the micro-motion force signal means that the absolute value of the deviation between the force signal sampled value monitored by the pressure sensor and the preset force signal threshold is less than the preset force signal deviation threshold. The preset force signal threshold is represented by the average value of the micro-motion force signal sampled values ​​over a historical time period. The preset force signal deviation threshold is set in advance by preset personnel. The dynamic balance signal quality quantification value is represented by the result of weighted coupling processing of the dynamic balance signal quality quantification parameters and the dynamic balance signal quality influence parameters within a preset feature acquisition time period. The dynamic balance signal quality quantification parameters include the step frequency sampling stability coefficient, which reflects the stable fluctuation of the step frequency sampling data in the time dimension, and the proportion of valid gait signal frames, which reflects the effectiveness of gait signal acquisition. The dynamic balance signal quality influence parameters include the step frequency sampling stability influence coefficient, which reflects the degree of influence of the step frequency sampling stability coefficient on the dynamic balance signal quality quantification value, and the gait signal valid frame influence coefficient, which reflects the degree of influence of the proportion of valid gait signal frames on the dynamic balance signal quality quantification value.The gait frequency sampling stability coefficient is represented by the proportion of the standard deviation and mean gait frequency data (such as the interval between each step during walking) monitored by the gait sensor within a preset feature acquisition period. The effective frame proportion of the gait signal is represented by the proportion of the number of gait data frames monitored by the gait sensor that meet the gait signal quality discrimination criteria to the total number of gait data frames within the preset feature acquisition period. The mean gait frequency is represented by the arithmetic mean of all gait frequency data within the preset feature acquisition period.

[0027] Specifically, the gait signal quality judgment condition indicates that the gait signal intensity monitored by the gait sensor is greater than a preset gait signal intensity threshold, where the preset gait signal intensity threshold is represented by the average gait signal intensity over a historical time period; the micro-motion feature correlation verification condition indicates that when the micro-skin-attached accelerometer detects a tremor signal and the pressure sensor should simultaneously show a corresponding gripping force fluctuation signal, the micro-motion feature delay index is less than a preset delay threshold and the consistency of the micro-motion feature change trend is greater than a preset consistency threshold, where the preset delay threshold is represented by the average micro-motion feature delay index over a historical time period, and the preset consistency threshold is represented by the average consistency of the micro-motion feature change trend over a historical time period; the micro-motion feature delay index is represented by the average micro-motion feature change trend consistency over a preset feature acquisition time period. Within a given time interval, the absolute value deviation between the occurrence time of the peak tremor signal monitored by the miniature skin-attached accelerometer and the occurrence time of the peak grip strength fluctuation signal monitored by the pressure sensor, which is closest to the current peak tremor signal, is calculated, and then the sum of these results is represented by an arithmetic average. The preset feature acquisition time period represents the time period for primary monitoring of biometric signal quality, and within the preset feature acquisition time period, there are micro-expression image sequences, effective micro-motion signal quality quantification parameters, peak tremor signal, and peak grip strength fluctuation signal. The consistency of micro-motion feature change trends is represented by the correlation coefficient between the tremor signal time series and the grip strength fluctuation signal time series obtained based on the Pearson correlation analysis algorithm within the preset feature acquisition time period.

[0028] Specifically, the biometric signal quality monitoring pass / fail judgment is used to reduce over-adjustment caused by a single, accidental failure in biometric signal quality monitoring, providing high-quality and stable data input for multimodal fusion and biometric status assessment. The specific process is as follows: A biometric signal quality monitoring pass / fail judgment is performed. If the biometric signal quality monitoring result is qualified, multimodal biometric correlation analysis is performed; otherwise, the signal quality monitoring counter value is incremented, and a signal quality monitoring counter over-limit judgment is performed. The specific process for the signal quality monitoring counter over-limit judgment is as follows: It is determined whether the signal quality monitoring counter value is greater than the preset maximum threshold. If so, biometric signal quality monitoring optimization is implemented, and the signal quality monitoring counter value is reset to its initial value; otherwise, primary biometric signal quality monitoring continues. The maximum threshold of the signal counter is preset by a designated person. The specific process for judging the pass / fail status of biometric signal quality monitoring is as follows: when the biometric signal quality quantification index meets the signal quality monitoring discrimination conditions, the biometric signal quality monitoring result is judged to be qualified; otherwise, the biometric signal quality monitoring result is judged to be unqualified. The signal quality monitoring discrimination conditions indicate that the signal-to-noise ratio of the micro-expression image sequence is greater than the preset image sequence signal-to-noise ratio threshold, the quantification value of the micro-motion signal quality is greater than the preset micro-motion threshold, and the quantification value of the dynamic balance signal quality is greater than the preset dynamic balance threshold. The preset image sequence signal-to-noise ratio threshold is represented by the average value of the signal-to-noise ratio of the micro-expression image sequence over a historical time period, the preset micro-motion threshold is represented by the average value of the quantification value of the micro-motion signal quality over a historical time period, and the preset dynamic balance threshold is represented by the average value of the quantification value of the dynamic balance signal quality over a historical time period.

[0029] It should be understood that, in the initial monitoring of biometric signal quality, a set of parameter mapping tables is involved to clarify the correlation between effective micro-motion signal quality influence parameters, dynamic equilibrium signal quality influence parameters, and corresponding signal quality quantification values. These tables are pre-set by technical personnel and stored in a database, providing a basis for matching and calculating the two types of signal quality influence parameters with the final quantification results. For example, a large amount of historical data from initial biometric signal quality monitoring scenarios is extracted, covering parameter combinations such as the signal-to-noise ratio of micro-motion tremor signals, micro-motion sensor sampling accuracy, gait frequency sampling stability coefficient, effective frame ratio of gait signals, and corresponding influence coefficients under different acquisition environments (e.g., changes in lighting, motion interference). Each combination is assigned a weighted quantification value based on its influence on the signal quality quantification value, while simultaneously recording the actual effective values ​​of the four types of influence coefficients in each scenario. The data is then analyzed using correlation analysis (such as Pearson correlation coefficient) to remove abnormal correlation data caused by temporary equipment failures (such as accuracy drift of miniature skin-attached accelerometers) or sudden interference in the acquisition environment (such as instantaneous electromagnetic noise). Statistically significant parameter combinations and their corresponding influence coefficients are retained. Finally, all valid data are integrated to form a parameter mapping table containing multiple sets of mapping relationships. The association rules in the table use a numerical range of 0-1 to represent the proportion of each influence coefficient, thereby achieving accurate matching (including one-to-one correspondence or multi-parameter collaborative adaptation) of basic parameters and corresponding influence coefficients in the calculation of micro-motion signal quality quantification and dynamic equilibrium signal quality quantification. When the system conducts primary monitoring of biometric signal quality, the matching influence coefficients can be quickly retrieved from this mapping table, ensuring the objectivity and reliability of the signal quality quantification results.

[0030] In this embodiment, primary monitoring of biometric signal quality helps control data quality from the source of early biometric time-series monitoring. By using multi-dimensional signal quality quantification indicators to screen high-quality data, the system reduces invalid signals caused by background noise and drastic data fluctuations. This reduces the interference of low-quality data on subsequent stages such as multimodal biometric correlation analysis and time-series feature fusion, improving the reliability, integrity, and temporal consistency of biometric data entering subsequent processes. This achieves pre-emptive control of data quality across the entire chain from the initial data collection stage to subsequent analysis stages, providing stable and high-quality data support for the accurate integration and discrimination of multimodal time-series features.

[0031] like Figure 3 The diagram shown is an optimization logic diagram for biometric signal quality monitoring in a biometric time-series monitoring system based on micro-expression-motor co-analysis provided in an embodiment of the present invention. Figure 3It can be seen that: biometric signal quality monitoring and optimization includes micro-expression feature signal quality monitoring and optimization and micro-motion feature signal quality monitoring and optimization. After the micro-expression feature signal quality monitoring and optimization is completed, it is determined whether the clarity of the micro-expression image is greater than the preset clarity threshold. If not, a micro-expression monitoring and optimization failure warning is sent. Otherwise, micro-motion feature signal quality monitoring and optimization is performed, and the angle data of both legs is obtained. It is determined whether the angle data of both legs is less than the preset angle data. If so, the biometric signal quality monitoring and optimization effect is verified. Otherwise, the angle measurement calculator value exceeds the limit judgment. It is determined whether the angle measurement counter value is greater than the preset maximum threshold of the angle counter. If not, micro-motion feature signal quality monitoring and optimization continues. Otherwise, the step frequency measurement interval is adjusted. After the step frequency measurement interval is adjusted, the biometric signal quality monitoring and optimization effect is verified. It is determined whether the biometric signal quality monitoring result is qualified. If not, a biometric signal quality monitoring and optimization failure warning is sent. Otherwise, multimodal biometric correlation analysis is performed.

[0032] Furthermore, the optimization of biometric signal quality monitoring includes optimization of micro-expression feature signal quality monitoring and micro-motion feature signal quality monitoring. Optimization of micro-expression feature signal quality monitoring reduces noise and blurring interference from micro-expression changes, improving the clarity of micro-expression images. The specific process is as follows: The signal-to-noise ratio of the micro-expression image sequence and the proportion of effective micro-expression image frames are entered into a preset focal length mapping table in the database for querying to obtain the focal length coefficient. The proportion of effective micro-expression image frames is represented by the quantified result of comparing the number of micro-expression image frames monitored by the image acquisition camera that meet the micro-expression image quality discrimination criteria within a preset feature acquisition time period with the total number of micro-expression image frames. Micro-expression image quality discrimination... The condition indicates that the edge sharpness of the key region of the image is greater than a preset image sharpness threshold. The preset image sharpness threshold is represented by the average edge sharpness of the key region of the image over a historical time period. The edge sharpness of the key region is represented by the average gray-level gradient of the pixels at the edge of the preset micro-expression key region (such as the eye contour and mouth corner lines), obtained based on an adaptive thresholding segmentation algorithm (such as adaptive Gaussian thresholding). The average gray-level gradient is represented by the result of summing the gray-level gradient values ​​of all pixels at the edge of the key region based on the Sobel operator and then quantizing the proportion with the total number of edge pixels. The adjustment range corresponding to the focal length coefficient is used as the adjustment... The camera focal length is adjusted incrementally in the direction of increasing micro-expression image sharpness (after each adjustment, the micro-expression image sharpness is recalculated; if the sharpness is still below a preset threshold, the adjusted focal length is used as the initial value for the next adjustment, continuing to incrementally adjust in the direction of increasing micro-expression image sharpness). This helps reduce overshooting or undershooting caused by large single adjustments, ensuring that each adjustment effectively improves image quality. Micro-expression image sharpness is represented by the average pixel density of micro-expression images monitored by the image acquisition camera within a preset feature acquisition time period. The camera focal length is within the preset focal length range. Within the specified range, the preset focal length range is pre-set by a preset team and includes both endpoints of the preset focal length range. The system continuously monitors the clarity of micro-expression images. When the clarity of the micro-expression image exceeds a preset clarity threshold, micro-motion feature signal quality monitoring and optimization are performed. Conversely, if the clarity is below a preset threshold, the micro-expression feature signal quality monitoring and optimization continues. If the number of micro-expression feature signal quality monitoring and optimization executions exceeds the preset maximum number of micro-expression signal executions, and the micro-expression image clarity is still not greater than the preset clarity threshold, a micro-expression monitoring and optimization failure warning is sent. The preset clarity threshold is represented by the average clarity of micro-expression images over a historical time period, and the preset maximum number of micro-expression signal executions is pre-set by a preset team.

[0033] Specifically, the micro-motion feature signal quality monitoring and optimization is used to reduce gait data fluctuations and correct tremor signal delay deviations, thereby improving the acquisition accuracy of dynamic balance and micro-motion timing features. The specific process is as follows: Obtain bi-leg angle data reflecting the degree of abnormality in knee joint movement function. The bi-leg angle data is represented by the average value of the bi-leg knee flexion and extension angles measured by angle sensors within a preset angle acquisition time period. The preset angle acquisition time period represents the time period for micro-motion feature signal quality monitoring and optimization, and it is shorter than the preset feature acquisition time period. Determine whether the bi-leg angle data is shorter than the preset angle data. If so, ... The angle measurement counter value is accumulated, and the angle measurement calculator value is judged to be out of limit. Otherwise, the biometric signal quality monitoring optimization effect is verified. The preset angle data is represented by the average value of the angle data of both legs over a historical time period. The specific process of judging the angle measurement calculator value to be out of limit is as follows: it is determined whether the angle measurement counter value is greater than the preset maximum threshold of the angle counter. If so, the step frequency measurement interval is adjusted and the angle measurement counter value is reset to the initial value. Otherwise, the micro-motion feature signal quality monitoring optimization continues. The preset maximum threshold of the angle counter is set in advance by preset personnel.

[0034] Specifically, the gait frequency measurement interval adjustment is used to improve the accuracy of the gait frequency stability assessment index and ensure the quality of motion feature time-series data acquisition. The specific process is as follows: The dynamic balance signal quality quantization value and the leg angle data are input into a preset gait frequency measurement interval mapping table in the database for querying to obtain the gait frequency measurement interval coefficient; using the adjustment range corresponding to the gait frequency measurement interval coefficient as the adjustment step size, the gait frequency measurement interval of the gait sensor is adjusted step by step in the direction of increasing dynamic balance signal quality quantization value (i.e., the direction of increasing gait frequency measurement interval coefficient). (After each gait frequency measurement interval adjustment, the dynamic balance signal quality quantization value is recalculated. If the dynamic balance signal quality quantization value is still not greater than the preset dynamic balance threshold, the adjusted gait frequency measurement interval is used as the initial value for the next adjustment, and the adjustment continues step by step in the direction of increasing dynamic balance signal quality quantization value.) This helps reduce data redundancy caused by excessively short measurement intervals or feature loss caused by excessively long intervals; the dynamic balance signal quality quantization value is continuously monitored. When the dynamic balance signal quality quantization value is greater than the preset dynamic balance threshold, the biometric signal quality monitoring optimization effect is verified. Otherwise, the step frequency measurement interval adjustment continues. If the number of step frequency measurement interval adjustments is still greater than the preset maximum number of step frequency measurement interval adjustments, and the dynamic balance signal quality quantization value is still not greater than the preset dynamic balance threshold, a micro-motion feature signal quality monitoring optimization failure warning is sent. The specific process for verifying the biometric signal quality monitoring optimization effect is as follows: After the biometric signal quality monitoring optimization is completed, the biometric monitoring data for the next preset feature acquisition time period is reacquired, and the biometric signal quality monitoring qualification is judged. If the biometric signal quality monitoring result is qualified, multimodal biometric correlation analysis is performed based on bio-temporal features and bio-physiological function temporal features. Otherwise, a biometric signal quality monitoring optimization failure warning is sent. The biometric monitoring data includes biometric signal quality quantization indicators, bio-temporal features, and bio-physiological function temporal features.

[0035] It should be understood that the preset focal length mapping table, preset gait frequency measurement interval mapping table, and preset multimodal timestamp compensation mapping table involved in the operation of the biometric time-series monitoring system based on micro-expression-motion co-analysis are all preset by professional technicians and stored in the database. They provide accurate basis for matching the signal-to-noise ratio of micro-expression image sequences, the proportion of effective micro-expression image frames and focal length coefficients; the dynamic balance signal quality quantization value; the leg angle data and gait frequency measurement interval coefficients; and the multimodal time-series consistency index and timestamp compensation coefficients.

[0036] Specifically, a large amount of historical data from biometric monitoring scenarios is extracted first: For a preset focal length mapping table, parameter combinations covering different micro-expression image sequences, signal-to-noise ratios, effective image frame ratios, and corresponding effective focal length coefficients are assigned adaptive quantization values ​​based on image clarity optimization effects (e.g., matching a larger focal length coefficient to improve detail capture when the signal-to-noise ratio is low, and matching an adaptive focal length coefficient to reduce blurred frames when the effective frame ratio is insufficient); For a preset gait frequency measurement interval mapping table, combinations of different dynamic balance signal quality quantization values, leg angle data, and corresponding gait frequency measurement interval coefficients are collected, and quantization values ​​are assigned according to the stability requirements of gait data (e.g., matching a smaller measurement interval to increase data density when the dynamic balance quality is low). For the timestamp compensation mapping table, the combination of consistency evaluation indicators and corresponding compensation coefficients of different multimodal temporal characteristics is summarized, and quantitative values ​​are assigned according to the temporal synchronization effect. At the same time, the actual effective values ​​of the three types of coefficients in each scenario are recorded. Then, through correlation analysis (such as Pearson correlation coefficient), abnormal correlation data caused by temporary equipment failure (such as camera focal length drift) and interference from the collection environment (such as sudden changes in illumination) are eliminated, and the correspondence between parameter combinations and coefficients with statistical significance is retained. Finally, the three types of mapping tables are integrated. When the system performs parameter adjustment, the matching coefficients can be quickly retrieved from the corresponding mapping table to ensure the clarity of micro-expression images, the stability of gait data, and the accuracy of multimodal temporal synchronization adjustment.

[0037] In this embodiment, optimization through biometric signal quality monitoring helps to implement scenario-based and precise adjustments for micro-expression image acquisition and gait signal acquisition, adapting to the signal acquisition needs under different acquisition environments. This reduces low-quality signal problems such as blurring of key areas in micro-expression images and drastic fluctuations in gait data, as well as the interference of poor-quality data on multimodal biometric correlation analysis and temporal feature fusion. It improves the detail sharpness of micro-expression images and the temporal stability of gait frequency data, while enhancing the overall accuracy and data integrity of biometric signal acquisition. This achieves closed-loop management from biometric signal acquisition optimization to pre-data quality assurance, ensuring that biometric signals entering subsequent analysis processes always maintain high reliability, and laying a high-quality data foundation for the efficient integration and accurate discrimination of multimodal temporal features.

[0038] Furthermore, multimodal biometric correlation analysis is used to uncover the synergistic changes between bio-temporal features and bio-physiological function temporal features over time, identify cross-modal anomalous correlation patterns, and reduce information redundancy between different modalities. The specific process is as follows: Bio-temporal features represent bio-temporal features collected chronologically within a preset feature acquisition period, including micro-expression temporal features (such as the frequency change sequence of facial muscle tremor signals within the preset feature acquisition period), micro-movement temporal features (such as the amplitude change sequence of grip force signals within the preset feature acquisition period), and movement temporal features (such as the gait stride stability change sequence within the preset feature acquisition period). Among these, facial... The sequence of muscle tremor signal frequency changes represents a continuous data sequence of tremor signal frequency changes in preset micro-expression key areas (such as the periorbital contour) monitored by a miniature skin-attached accelerometer within a preset feature acquisition time period. The sequence of grip force signal amplitude changes represents a dynamic data sequence of pressure value fluctuation range during gripping movements monitored by a pressure sensor within a preset feature acquisition time period. The sequence of gait stride stability changes represents a continuous data sequence of gait stride deviation values ​​within a preset feature acquisition time period. The gait stride deviation value is represented by the difference between the actual gait stride measurement value monitored by the gait sensor and a preset stride threshold. The preset stride threshold is determined by gait data from historical time periods. The average stride measurement value represents the biophysiological function temporal characteristics collected chronologically within a preset feature acquisition period. These include olfactory temporal characteristics (such as the olfactory recognition reaction time sequence) and heart rate temporal characteristics (such as the heart rate value sequence). The olfactory recognition reaction time sequence represents the continuous data sequence of time intervals between different olfactory stimulus signals monitored by the olfactory stimulus response monitor within the preset feature acquisition period. The heart rate value sequence represents the dynamic data sequence of heartbeat signals monitored by the heart rate sensor within the preset feature acquisition period. The data will be collected chronologically within the preset feature acquisition period. The collected biological temporal features and corresponding biological physiological function temporal features are input into a preset multimodal feature fusion model (such as an attention mechanism fusion model). The model outputs a single biological temporal feature correlation index (such as the dynamic correlation index between the facial muscle tremor frequency change sequence and the heart rate value change sequence, which is used to quantify the temporal coordination between the frequency fluctuation of facial muscle tremors and the rise and fall trend of heart rate values ​​within the preset feature collection time period). The model also performs non-motor temporal feature correlation discrimination. The preset multimodal feature fusion model is set in advance by preset personnel to explore the potential correlation between biological temporal features and biological physiological function temporal features, suppress irrelevant feature interference, and achieve effective integration of multi-dimensional temporal features.The specific process for non-motor temporal feature correlation determination is as follows: It is determined whether the correlation index of a single biological temporal feature is greater than a preset correlation threshold. If so, the corresponding biological physiological function temporal feature is classified as a valid biological physiological function temporal feature, and multimodal temporal feature fusion is performed. Conversely, if the correlation index is less than the preset threshold, the corresponding biological physiological function temporal feature is classified as an invalid biological physiological function temporal feature and is removed. The preset correlation threshold is represented by the average correlation index of a single biological temporal feature over a historical time period.

[0039] Specifically, the process of multimodal temporal feature fusion is as follows: The original acquisition timestamps of biological temporal features are aligned with the original acquisition timestamps of effective biological physiological function temporal features using a linear interpolation algorithm; the temporal correlation coefficients of each modality feature are obtained, reflecting the degree of synergistic change between each biological temporal feature and non-motor temporal feature in the time dimension; the temporal correlation coefficients of each modality feature represent the Pearson correlation coefficients between multimodal feature subsequences within a preset time window, obtained using a Pearson correlation analysis algorithm, where the preset time window represents the time period for multimodal temporal feature fusion, pre-set by designated personnel; the temporal correlation coefficients of each modality feature are input into a preset correlation coefficient weight allocation table in the database for querying, obtaining the weight coefficients of the multimodal feature subsequences; after standardizing each multimodal feature subsequence using the Z-score normalization algorithm, the weights are then added... The process involves weighting to obtain multimodal fusion feature vectors. Weighting involves multiplying the data at each time point in the standardized modal feature subsequences with the corresponding modal multimodal feature subsequence weight coefficients, and then concatenating the weighted data from each modality at the same time point in a preset order. Each multimodal feature subsequence includes biological temporal feature subsequences and biological physiological function temporal feature subsequences. Biological temporal feature subsequences represent biological temporal feature segments arranged chronologically within a preset time window; biological physiological function temporal feature subsequences represent biological physiological function temporal feature segments arranged chronologically within a preset time window. All multimodal fusion feature vectors within the preset time window are arranged chronologically to obtain a multidimensional modal temporal feature set. After multimodal temporal feature fusion, the accuracy of the multimodal temporal features is determined based on the multidimensional modal temporal feature set.

[0040] In this embodiment, multimodal biometric correlation analysis and multimodal temporal feature fusion help to deeply explore the implicit synergistic relationship between biological temporal features and biological physiological function temporal features in the time dimension (such as the synchronization pattern between micro-movement fluctuations and heart rate changes). This breaks through the limitations of single-modal feature analysis, reduces the interference of information silos and redundant overlaps between different modal data on subsequent analysis, improves the integration efficiency and correlation accuracy of multi-dimensional feature information, strengthens the comprehensive representation ability of the fused feature vector on changes in biological state, and realizes the transformation from scattered single-modal temporal data to structured multimodal fused features. This provides more valuable core data support for subsequent multimodal temporal feature accuracy judgment and biological feature state assessment.

[0041] Furthermore, the accuracy discrimination of multimodal temporal features is used to evaluate the temporal consistency, stability, and modal correlation of multi-dimensional modal temporal feature sets. The specific process is as follows: Obtain the multimodal temporal feature consistency evaluation index to reflect the accuracy of multimodal temporal features; the multimodal temporal feature consistency evaluation index is represented by the result of the collaborative fusion processing of the multimodal temporal matching index used to quantify multimodal temporal consistency, the multimodal temporal stability index used to reflect multimodal temporal stability, and the modal correlation correction coefficient used to correct the correlation deviation between modes. The collaborative fusion processing represents the product operation.

[0042] Specifically, the formula for the multimodal time series matching index is as follows:

[0043] ;

[0044] Among them, S m The multimodal temporal matching index is represented by m, which represents the index of the m-th modal temporal feature, ranging from 1 to M, where M represents the total number of modal temporal features, R represents the preset temporal constant, and W represents the multimodal temporal matching index. m The modal weight coefficient represents the temporal feature of the m-th modality, reflecting its influence on the multimodal temporal matching index (DTW). m The dynamic time warping distance of the m-th mode is represented by the preset time constant, which is set in advance by preset personnel to avoid the multimodal time matching index being meaningless. The dynamic time warping distance of the m-th mode is obtained by aligning the time feature subsequence of the current m-th mode with the preset time feature subsequence of the mode. It is based on the minimum cumulative distance between the two sequences obtained by the dynamic time planning algorithm and is used to quantify the temporal consistency of the temporal features of a single mode.

[0045] Specifically, the process of the multimodal time series stability index is as follows:

[0046] ;

[0047] Where T represents the multimodal temporal stability index, σ T The time series stability coefficient is represented by r within a preset time window, and the stability adjustment factor is represented by the standard deviation of the dynamic time warping distance of each multimodal within the preset time window. It is used to reflect the degree of consistency fluctuation of multimodal time series characteristics in the time dimension. The stability adjustment factor is used to reflect the degree of influence of the time series stability coefficient on the consistency evaluation index of multimodal time series characteristics.

[0048] Specifically, the formula for the consistency evaluation index of multimodal temporal features is as follows:

[0049] ;

[0050] Where C represents the consistency evaluation index of multimodal time series features, and q represents the modal correlation correction coefficient, which is represented by the average value of the temporal correlation coefficients of each modal feature within a preset time window. It is used to correct the consistency evaluation index of multimodal time series features caused by insufficient correlation between modal time series features.

[0051] If the consistency evaluation index of multimodal time series features is greater than the preset time series consistency threshold, a biometric time series anomaly alarm is issued; otherwise, timestamp alignment adjustment is performed. The preset time series consistency threshold is represented by the average value of the consistency evaluation index of multimodal time series features over a historical time period.

[0052] It should be understood that in the embodiments of this application, there is a set of modal weight coefficients, temporal stability coefficients, and stability adjustment factors. These three types of parameters are all generated through statistical analysis and scenario adaptation verification based on historical monitoring data and stored in the system database, providing a reliable basis for the accurate calculation of multimodal temporal matching index and consistency evaluation index.

[0053] Specifically, firstly, a large amount of historical data from multimodal biometric monitoring scenarios is extracted, covering modal feature data and corresponding parameter value samples under different modality combinations and different acquisition environments (such as indoor steady state and outdoor dynamic interference). For modality weight coefficients, based on the actual impact of each modality (such as micro-expression modality and motion modality) on the temporal matching results in historical monitoring, differentiated weight quantification values ​​are assigned to different modalities (e.g., when micro-expression modality has a greater impact on emotion-related temporal matching, its weight coefficient is higher). For temporal stability coefficients, the standard deviation of the multimodal dynamic time regularization distance within a preset time window under each historical scenario is calculated to form a stable... A sample library of stability coefficients is established. For the stability adjustment factor, based on the influence of the time series stability coefficient on the final consistency assessment result in historical data, the effective value range of the adjustment factor is determined. After simultaneously recording the actual effective values ​​of the three types of parameters in each scenario, abnormal parameter samples caused by temporary equipment failures (such as data interruption of a certain modal sensor) and sudden interference (such as instantaneous motion) are removed through correlation analysis (such as Pearson correlation coefficient). The correspondence between statistically significant parameters and monitoring scenarios is retained. Finally, through data integration and expert experience calibration, the modal weight coefficient, the benchmark value of the time series stability coefficient, and the value of the stability adjustment factor applicable to multiple scenarios are determined.

[0054] In this embodiment, by judging the accuracy of multimodal temporal features, it is helpful to identify problems such as timestamp deviation and insufficient temporal stability in multimodal fusion features in a timely manner (such as the asynchronous timing of micro-expression and heart rate). This reduces the risk of analysis deviation and misjudgment caused by insufficient accuracy of temporal features, improves the consistency, reliability and matching degree of multimodal temporal features with actual biological state changes in the time dimension, and ensures that the temporal feature data entering the final biometric abnormality alarm stage remains high quality.

[0055] Furthermore, the specific process of timestamp alignment adjustment is as follows: The multimodal time series feature consistency evaluation index is input into the preset multimodal timestamp compensation mapping table in the database for querying, and the timestamp compensation coefficient is obtained; using the magnitude corresponding to the timestamp compensation coefficient as the adjustment step size, the timestamp compensation coefficient of each modality's time series data is adjusted level by level in the direction of increasing the multimodal time series feature consistency evaluation index (i.e., the direction of increasing the timestamp compensation coefficient). (After each timestamp compensation coefficient adjustment, the multimodal time series feature consistency evaluation index is recalculated. If the multimodal time series feature consistency evaluation index is still not greater than the preset time series consistency threshold, then the adjusted timestamp compensation coefficient is used as...) The initial value for the next adjustment is continuously adjusted step by step in the direction of increasing the multimodal temporal feature consistency evaluation index. This helps reduce the time deviation between different modalities caused by excessive or insufficient compensation in a single instance, dynamically improves the temporal synchronization of multimodal temporal features, ensures the coordinated matching of data from each modality in the time dimension, and reduces the distortion of feature correlation caused by time misalignment. The timestamp compensation coefficient is superimposed on the original acquisition timestamp of each modality's temporal data to re-acquire the multi-dimensional modal temporal feature set, and the multimodal temporal feature consistency evaluation index is continuously monitored. When the multimodal temporal feature consistency evaluation index is greater than the preset temporal consistency threshold, a biometric temporal anomaly alarm is issued; otherwise, it is not. Then, the timestamp alignment adjustment continues. When the number of timestamp alignment adjustments exceeds the preset maximum number of timestamp alignment adjustments, if the multimodal temporal feature consistency evaluation index is still not greater than the preset temporal consistency threshold, a timestamp alignment adjustment failure warning is sent. The preset maximum number of timestamp alignment adjustments is set in advance by preset personnel. The specific process of biometric anomaly alarms used to promptly identify abnormal patterns in biometrics and trigger corresponding warning levels is as follows: The historical multimodal temporal feature set is divided into a historical multimodal temporal feature training set and a historical multimodal temporal feature test set. The preset recognition model (as shown in the figure, neural network, note) is tested using the historical multimodal temporal feature training set. The parameters of the model are iteratively optimized using a fusion model of intention and force mechanisms, and the model accuracy is verified using a test set of historical multi-dimensional modal temporal features to complete the training of the preset recognition model. The multi-dimensional modal temporal feature set is input into the trained preset recognition model, and the biometric abnormality correlation coefficient is output. The biometric abnormality correlation coefficient is input into the preset biometric abnormality database to obtain the corresponding level of biometric abnormality warning. The preset recognition model is used to mine the hidden abnormal correlation patterns in the multi-dimensional modal temporal feature set, quantitatively evaluate the degree of biometric abnormality and the correlation strength between each modality feature, and provide accurate quantitative basis for biometric abnormality warning.

[0056] In this embodiment, by adjusting timestamp alignment and issuing biometric anomaly alerts, the time deviation of multimodal temporal features (such as the asynchrony between micro-expression time series and heart rate time series) can be accurately corrected, ensuring the coordinated matching of data from each modality in the time dimension and reducing the distortion of feature correlation caused by time misalignment. At the same time, relying on the biometric anomaly correlation coefficient output by the preset recognition model to trigger the corresponding level of warning, the hidden abnormal correlation patterns in multimodal features can be captured in a timely manner, reducing the missed or false judgment of potential biometric anomalies, and realizing the accurate identification and graded warning of early biometric anomalies.

[0057] In summary, conducting primary monitoring of biosignal signal quality and determining whether further optimization is needed based on the results helps to screen qualified biosignal signals from the source, reducing interference from low-quality and invalid data in subsequent collaborative fusion processes. This lays the foundation for accurate correlation and fusion of multi-dimensional data. After passing primary monitoring, multimodal biosignal correlation analysis helps to uncover potential correlations between multi-dimensional features such as biological temporal features and biological physiological function temporal features, reducing the correlation loss problem caused by ineffective multi-dimensional data correlation in existing technologies. After multimodal biosignal correlation analysis, multimodal temporal feature fusion helps to integrate the advantages of each modality, reduce information redundancy, and improve the accuracy of multi-dimensional data collaborative fusion results. After multimodal temporal feature fusion, multimodal temporal feature accuracy judgment is performed. Based on the accuracy judgment results, it is determined whether timestamp alignment adjustment is needed, which helps to correct time deviations of different modal temporal features, ensure the temporal consistency of fused features, and improve the accuracy of disease assessment during early biosignal temporal monitoring.

[0058] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A biometric time-series monitoring system based on micro-expression-motion co-analysis, characterized in that, include: Biometric signal quality monitoring module, multimodal biometric correlation monitoring module, and multimodal temporal feature accuracy monitoring module: The biometric signal quality monitoring module is used to perform primary biometric signal quality monitoring during early biometric time-series monitoring to evaluate the quality of biometric signal acquisition. Based on the results of the primary biometric signal quality monitoring, it determines whether biometric signal quality monitoring optimization is needed. The biometric signal quality monitoring optimization is used to improve the clarity of key areas of micro-expression images and the accuracy of motion feature acquisition, reduce noise data interference and data fluctuations, and ensure the reliability of biometric signals. The multimodal biofeature association monitoring module is used to perform multimodal biofeature association analysis to mine potential associations between biofeatures and non-motor biofeatures after the initial monitoring of biofeature signal quality is qualified. After the multimodal biofeature association analysis is completed, multimodal time series feature fusion is performed to integrate the time series features of each biomodality and reduce information redundancy. The multimodal temporal feature accuracy monitoring module is used to perform multimodal temporal feature accuracy judgment after the multimodal temporal feature fusion is completed, which is used to verify the temporal consistency, stability and modal correlation of the fused multimodal temporal features. Based on the multimodal temporal feature accuracy judgment result, it determines whether timestamp alignment adjustment is needed. The timestamp alignment adjustment is used to correct the acquisition time deviation of different modal temporal features and improve the temporal consistency of multimodal biometric features.

2. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 1, characterized in that, The specific process for the primary monitoring of biometric signal quality is as follows: Obtain biometric signal quality quantification indicators for evaluating the quality of biometric signal acquisition. The biometric signal quality quantification indicators include the micro-expression image sequence signal-to-noise ratio (SNR) for reflecting the intensity of effective feature signals in the micro-expression image sequence, the micro-motion signal quality quantification value for reflecting the proportion of effective information and data reliability in the micro-motion signal, and the dynamic balance signal quality quantification value for reflecting the acquisition accuracy and data fluctuation amplitude of the biometric dynamic balance signal. Judging the compliance of biosignal quality monitoring based on quantitative indicators of biosignal quality; The micro-motion signal quality quantization value is represented by the result of weighted coupling processing of the effective micro-motion signal quality quantization parameter and the effective micro-motion signal quality influence parameter within a preset feature acquisition time period. The effective micro-motion signal quality quantization parameter represents the micro-motion signal quality quantization parameter that meets the micro-motion feature correlation verification condition. The micro-motion signal quality quantization parameter includes the micro-motion tremor signal signal-to-noise ratio, which reflects the degree of separation between the effective tremor feature and the environmental noise in the micro-motion tremor signal, and the micro-motion force signal sampling accuracy, which reflects the degree of agreement between the micro-motion force signal sampling data and the actual force value. The effective micro-motion signal quality influence parameters include the vibration signal signal-to-noise ratio influence coefficient, which reflects the influence of the micro-motion vibration signal signal-to-noise ratio on the quantized value of the micro-motion signal quality, and the force signal sampling accuracy influence coefficient, which reflects the influence of the sampling accuracy of the micro-motion sensor on the quantized value of the micro-motion signal quality. The dynamic equilibrium signal quality quantization value is represented by the result of weighted coupling processing of the dynamic equilibrium signal quality quantization parameter and the dynamic equilibrium signal quality influence parameter within a preset feature acquisition time period. The dynamic balance signal quality quantization parameters include a step frequency sampling stability coefficient, which reflects the stable fluctuation of step frequency sampling data in the time dimension, and a gait signal effective frame ratio, which reflects the effectiveness of gait signal acquisition. The dynamic balance signal quality influence parameters include a step frequency sampling stability influence coefficient, which reflects the influence of the step frequency sampling stability coefficient on the dynamic balance signal quality quantization value, and a gait signal effective frame influence coefficient, which reflects the influence of the gait signal effective frame ratio on the dynamic balance signal quality quantization value. The micro-motion feature correlation verification condition means that when a vibration signal is detected and the pressure sensor should synchronously produce a corresponding gripping force fluctuation signal, the micro-motion feature delay index is less than a preset delay threshold and the consistency of the micro-motion feature change trend is greater than a preset consistency threshold. The consistency of the micro-motion feature change trend is represented by the correlation coefficient between the time series of tremor signals and the time series of grip strength fluctuation signals obtained within a preset feature acquisition period.

3. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 2, characterized in that, The biometric signal quality monitoring pass / fail judgment is used to reduce over-adjustment caused by a single, accidental failure of biometric signal quality monitoring results. The specific process is as follows: The biometric signal quality monitoring is qualified. If the biometric signal quality monitoring result is qualified, multimodal biometric correlation analysis is performed. Otherwise, the signal quality monitoring counter value is accumulated and the signal quality monitoring counter exceeds the limit. The specific process for determining the signal quality monitoring counter when it exceeds the limit is as follows: Determine whether the signal quality monitoring counter value is greater than the preset maximum threshold of the signal counter. If so, optimize the biometric signal quality monitoring and reset the signal quality monitoring counter value to the initial value. Otherwise, continue to perform the primary biometric signal quality monitoring. The specific process for determining the pass / fail status of biometric signal quality monitoring is as follows: When the quantitative indicators of biometric signal quality meet the criteria for signal quality monitoring and discrimination, the biometric signal quality monitoring result is deemed qualified; otherwise, the biometric signal quality monitoring result is deemed unqualified. The signal quality monitoring and discrimination conditions indicate that the signal-to-noise ratio of the micro-expression image sequence is greater than a preset image sequence signal-to-noise ratio threshold, the quantization value of the micro-motion signal quality is greater than a preset micro-motion threshold, and the quantization value of the dynamic balance signal quality is greater than a preset dynamic balance threshold.

4. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 2, characterized in that, The optimization of biometric signal quality monitoring includes optimization of micro-expression feature signal quality monitoring and optimization of micro-motion feature signal quality monitoring; The micro-expression feature signal quality monitoring and optimization is used to reduce noise and blurring interference from micro-expression changes, and improve the clarity of micro-expression images. The specific process is as follows: The signal-to-noise ratio of the micro-expression image sequence and the proportion of effective micro-expression image frames are entered into the database for querying to obtain the focal length coefficient; Using the adjustment range corresponding to the focal length coefficient as the adjustment step size, the camera focal length is adjusted step by step in the direction of increasing the clarity of the micro-expression image; The system continuously monitors the clarity of micro-expression images. When the clarity of the micro-expression image exceeds a preset clarity threshold, it performs micro-motion feature signal quality monitoring and optimization. Otherwise, it continues to perform micro-expression feature signal quality monitoring and optimization. If the number of micro-expression feature signal quality monitoring and optimization executions exceeds the preset maximum number of micro-expression signal executions, and the clarity of the micro-expression image still does not exceed the preset clarity threshold, it sends a micro-expression monitoring and optimization failure warning.

5. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 4, characterized in that, The optimization of micro-motion feature signal quality monitoring is used to improve the acquisition accuracy of dynamic balance and micro-motion temporal features. The specific process is as follows: Obtain leg angle data to reflect the degree of abnormality in the movement function of the knee joints; Determine whether the angle data of both legs is less than the preset angle data. If so, the angle measurement counter value is incremented and the angle measurement calculator value exceeds the limit. Otherwise, the biometric signal quality monitoring optimization effect is verified. The specific process for determining if the angle measurement calculator's numerical values ​​exceed limits is as follows: If the angle measurement counter value is greater than the preset maximum threshold, the step frequency measurement interval is adjusted to improve the stability and timing consistency of step frequency data sampling, and the angle measurement counter value is reset to the initial value. Otherwise, the micro-motion feature signal quality monitoring and optimization continues.

6. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 5, characterized in that, The adjustment of the gait frequency measurement interval is used to improve the accuracy of the gait frequency stability assessment index and ensure the acquisition quality of motion feature time series data. The specific process is as follows: The dynamic balance signal quality quantization value and the leg angle data are input into the database for querying to obtain the step frequency measurement interval coefficient; The step frequency measurement interval of the gait sensor is adjusted step by step in the direction of increasing the quantization value of the dynamic balance signal quality, using the adjustment amplitude corresponding to the step frequency measurement interval coefficient as the adjustment step size. The dynamic balance signal quality quantization value is continuously monitored. When the dynamic balance signal quality quantization value is greater than the preset dynamic balance threshold, the biometric signal quality monitoring optimization effect is verified. Otherwise, the step frequency measurement interval adjustment is continued. When the number of step frequency measurement interval adjustments is greater than the preset maximum number of step frequency measurement interval adjustments, if the dynamic balance signal quality quantization value is still not greater than the preset dynamic balance threshold, a micro-motion feature signal quality monitoring optimization failure warning is sent. The specific process for verifying the optimization effect of biometric signal quality monitoring is as follows: After the optimization of biosignal signal quality monitoring is completed, biosignal monitoring data for the next preset feature acquisition time period is reacquired, and the qualification of biosignal signal quality monitoring is judged. If the biosignal signal quality monitoring result is qualified, multimodal biosignal correlation analysis is performed based on biotemporal features and biophysiological function temporal features; otherwise, a biosignal signal quality monitoring optimization failure warning is sent.

7. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 4, characterized in that, The multimodal biomarker correlation analysis is used to uncover the synergistic change patterns between biological temporal features and biological physiological function temporal features over time. The specific process is as follows: The biological temporal features and corresponding biological physiological function temporal features collected in chronological order within the preset feature collection time period are input into the preset multimodal feature fusion model, which outputs the correlation index of a single biological temporal feature and performs non-motor temporal feature correlation discrimination. The specific process for determining the correlation of non-moment temporal features is as follows: If the correlation index of a single biological time-series feature is greater than a preset correlation threshold, the corresponding biological physiological function time-series feature is determined as a valid biological physiological function time-series feature and multimodal time-series feature fusion is performed. Otherwise, the corresponding biological physiological function time-series feature is determined as an invalid biological physiological function time-series feature and is removed.

8. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 7, characterized in that, The specific process of multimodal temporal feature fusion is as follows: The original timestamps of biological temporal features are aligned with the original timestamps of effective biological physiological function temporal features. Obtain temporal correlation coefficients for each modality feature that reflect the degree of coordinated change between biological temporal features and non-motor temporal features in the temporal dimension; The temporal correlation coefficient of each modal feature represents the Pearson correlation coefficient between multimodal feature subsequences within a preset time window; Input the temporal correlation coefficients of each modality feature into the preset correlation coefficient weight allocation table in the database for querying, and obtain the weight coefficients of the multimodal feature subsequences; After standardizing each multimodal feature subsequence, a weighted sum is performed to obtain the multimodal fusion feature vector. Each of the multimodal feature subsequences includes biological time-sequence feature subsequences and biological physiological function time-sequence feature subsequences; Arrange all multimodal fusion feature vectors within the preset time window in chronological order to obtain a multidimensional modal temporal feature set; After the multimodal temporal feature fusion is completed, the accuracy of the multimodal temporal features is judged based on the multidimensional modal temporal feature set.

9. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 7, characterized in that, The accuracy determination of the multimodal temporal features is used to evaluate the temporal consistency, stability, and modal correlation of the multi-dimensional modal temporal feature set. The specific process is as follows: Obtain a consistency evaluation index for multimodal temporal features to reflect the accuracy of multimodal temporal features; The multimodal temporal feature consistency evaluation index is represented by the result of a collaborative fusion of the multimodal temporal matching index used to quantify multimodal temporal consistency, the multimodal temporal stability index used to reflect multimodal temporal stability, and the modal correlation correction coefficient used to correct the correlation deviation between modes. Determine whether the consistency evaluation index of multimodal temporal features is greater than the preset temporal consistency threshold. If so, issue a biometric temporal anomaly alarm; otherwise, take timestamp alignment adjustment measures.

10. The biometric time-series monitoring system based on micro-expression-motion co-analysis according to claim 9, characterized in that, The specific process for timestamp alignment adjustment is as follows: Input the multimodal temporal feature consistency evaluation index into the preset multimodal timestamp compensation mapping table for querying and obtain the timestamp compensation coefficient; Using the magnitude corresponding to the timestamp compensation coefficient as the adjustment step size, the timestamp compensation coefficient of each modality time series data is adjusted step by step in the direction of increasing the consistency evaluation index of multimodal time series features; The timestamp compensation coefficient is superimposed on the original acquisition timestamp of each modality time series data to re-acquire the multi-dimensional modality time series feature set. The consistency evaluation index of multi-modality time series features is continuously monitored. When the consistency evaluation index of multi-modality time series features is greater than the preset time series consistency threshold, a biometric time series anomaly alarm is issued. Otherwise, the timestamp alignment adjustment continues to be performed. When the number of timestamp alignment adjustment executions is greater than the preset maximum number of timestamp alignment adjustments, if the consistency evaluation index of multi-modality time series features is still not greater than the preset time series consistency threshold, a timestamp alignment adjustment failure warning is sent. The biometric anomaly alarm is used to promptly identify abnormal biometric patterns and trigger corresponding warning levels. The specific process is as follows: Input the multi-dimensional modal temporal feature set into the preset recognition model, output the biometric anomaly correlation coefficient, and input the biometric anomaly correlation coefficient into the preset biometric anomaly database to obtain the corresponding level of biometric anomaly warning.

Images