Emotion event recognition method and system based on multi-modal physiological parameters

Abstract

This invention relates to the field of emotion recognition technology, specifically to a method and system for emotion event recognition based on multimodal physiological parameters. The method determines a baseline time window based on the changing trend of the respiratory rate time series; it determines the heart rate analysis interval and blood pressure analysis interval based on the hysteresis response characteristics of the heart rate and blood pressure time series within the baseline time window; the baseline time window, heart rate, and blood pressure analysis intervals constitute the current representation feature; the feature matching degree is determined based on the similarity between the current representation feature and a preset set of historical representation features; the current representation feature is weighted and fused for optimization based on the feature matching degree to determine the optimized representation feature; the optimized representation feature is input into a pre-trained classification model, which outputs a category label corresponding to the emotional event. This makes the representation feature more closely resemble the real emotion-physiological linkage, so that the classification result can take into account the differences in individual physiological characteristics.

Description

Technical Field

[0001] This invention relates to the field of emotion recognition technology, specifically to a method and system for emotion event recognition based on multimodal physiological parameters. Background Technology

[0002] In scenarios such as health management and daily condition monitoring, sensor technology or wearable devices continuously collect multimodal physiological parameters of the human body (such as respiration, heart rate, and blood pressure). These massive amounts of time-series physiological data contain rich information reflecting changes in physiological state related to emotions.

[0003] In existing technologies, the analysis of physiological parameters often focuses on the static threshold judgment of a single indicator or the simple parallel processing of multiple parameters. However, this approach ignores the significant differences in the response speed and dynamic characteristics of different physiological parameters as physiological states change with emotional triggers, as well as the inherent regulatory relationships between physiological parameters. This leads to a high probability of introducing noise or losing physiological parameters that are still changing, making it difficult to fully capture the process of coordinated changes between physiological parameters. Furthermore, since different individuals have different physiological responses to the same emotion, this approach is difficult to adapt to the physiological regulatory patterns of different individuals. Summary of the Invention

[0004] To address the technical problem that neglecting the significant differences in response speed to changes in physiological state due to different physiological parameters, and ignoring the inherent regulatory relationships between physiological parameters, affects the accuracy of emotional event recognition, this invention provides an emotional event recognition method and system based on multimodal physiological parameters. The specific technical solution adopted is as follows: This invention proposes a method for identifying emotional events based on multimodal physiological parameters, the method comprising: Real-time acquisition of multimodal physiological parameter data of target objects to generate time series of physiological parameters, wherein the multimodal physiological parameter data includes at least respiratory rate, heart rate and blood pressure; Based on the changing trend of the respiratory rate time sequence, a baseline time window corresponding to the emotional event is determined; Based on the hysteresis response characteristics of the heart rate time series and blood pressure time series relative to the respiratory rate series within the baseline time window, the heart rate analysis interval and blood pressure analysis interval are determined respectively; the baseline time window, heart rate analysis interval, and blood pressure analysis interval together constitute the current representation characteristics of the current emotional event; The feature matching degree is determined based on the similarity between the current representation features and the preset set of historical representation features. Based on the feature matching degree, the baseline time window, heart rate analysis interval and blood pressure analysis interval corresponding to the current emotional event are weighted and fused for optimization to determine the optimized representation features. The optimized representation features are input into the pre-trained classification model, which outputs category labels corresponding to emotional events; based on the frequency of occurrence of category labels within a preset time period, prompt information is triggered.

[0005] Furthermore, the process of determining the reference time window includes: Based on the respiratory rate time series, the baseline value of respiratory rate is obtained through statistical analysis; The moment when the respiratory rate first exceeds the first floating threshold in the time sequence is taken as the starting point of the reference time window, wherein the first floating threshold is determined based on the respiratory rate baseline value being floated up by a preset first proportion; The moment when the respiratory rate first falls below the second floating threshold in the time series is taken as the end point of the reference time window, wherein the second floating threshold is determined based on a preset second proportion of the respiratory rate baseline value. The reference time window is defined by the start point and the end point.

[0006] Furthermore, the step of statistically obtaining the baseline value of respiratory rate based on the respiratory rate time series includes: The respiratory rate time sequence is divided into multiple continuous time periods according to a preset unit of time; For each time period, calculate the mode of the respiratory rate within that time period; Calculate the arithmetic mean of the modes for all time periods, and use it as the baseline for respiratory rate.

[0007] Furthermore, the multimodal physiological parameter data also includes heart rate variability parameters, and the process of determining the heart rate analysis interval and blood pressure analysis interval includes: Based on the instantaneous change trend of respiratory rate time sequence within the baseline time window, identify slow-rhythm breathing periods where the respiratory rate shows a rhythmic decrease. Calculate the standard deviation of respiratory rate during the slow-rhythm breathing period, and normalize the reciprocal of the standard deviation to obtain the respiratory rhythm parameter; calculate the ratio of the duration of the slow-rhythm breathing period to the duration of the baseline time window as the time proportion. The product of the breathing rhythm parameter and the time percentage is normalized to obtain the regulatory participation parameter. Based on the baseline time window and regulatory participation parameters, the heart rate analysis interval is determined; Based on the baseline time window, regulatory participation parameters, and heart rate variability parameters, the blood pressure analysis interval is determined.

[0008] Furthermore, the identification of slow-rhythm breathing periods where the respiratory rate exhibits a rhythmic decrease, based on the instantaneous change trend of the respiratory rate time sequence within the reference time window, includes: Calculate the first-order difference sequence of the respiratory rate time series; Traverse the respiratory rate time sequence and identify sub-intervals that meet the rhythmic slowing condition. The rhythmic slowing condition is that the respiratory rate is continuously lower than the respiratory rate baseline value within the sub-interval, the first difference sequence is continuously negative within the sub-interval, and the absolute value of the first difference is maintained within a preset amplitude range. All identified sub-intervals that meet the criteria for rhythmic slowing are collectively considered as slow-rhythm breathing periods.

[0009] Furthermore, determining the heart rate analysis interval based on the baseline time window and the regulation participation parameter includes: The time intervals between adjacent peak times in the respiratory rate time series and the heart rate time series are extracted to generate respiratory time interval sequences and heart rate time interval sequences, respectively. The least squares method is used to fit the respiratory time interval sequences and heart rate time interval sequences to construct a respiratory-heart rate correlation model. Based on the time interval between the last adjacent peak times in the respiratory rate time series, the corresponding heart rate time interval is predicted through the respiratory-heart rate correlation model, and the corresponding heart rate time interval is used as the heart rate time series lag. Add the participation parameter to a positive integer 1 to obtain the expansion coefficient; calculate the product of the heart rate lag and the expansion coefficient as the heart rate expansion duration; By overlaying the baseline time window with the extended heart rate duration, the heart rate analysis interval is obtained.

[0010] Furthermore, the determination of the blood pressure analysis interval based on the baseline time window, regulatory participation parameter, and heart rate variability parameter includes: The ratio of low-frequency power to high-frequency power in the heart rate variability parameters was calculated as an indicator of sympathetic nerve activity. If the sympathetic nerve activity index exceeds the preset threshold, the difference between the sympathetic nerve activity index and the preset threshold is normalized to obtain a correction coefficient. Calculate blood pressure expansion duration using the same method as heart rate expansion duration; Calculate the product of the correction factor and the blood pressure expansion duration as the actual blood pressure expansion duration; The baseline time window is superimposed with the actual extended duration of blood pressure to obtain the blood pressure analysis interval.

[0011] Furthermore, the process of determining the preset set of historical representation features includes: Historical representation features corresponding to multiple historical emotional events are extracted from historical physiological parameter data to form a historical representation feature sample set. Cluster analysis was performed on the historical characteristic feature sample set to obtain multiple feature clusters; Calculate the arithmetic mean of all historical representation features within each feature cluster, and use it as the typical feature representing the feature cluster. The set of historical representation features is composed of the typical features of all feature clusters.

[0012] Furthermore, the feature matching degree determination process includes: For the typical features of each feature cluster in the historical feature set, calculate the absolute difference between the current feature and the corresponding baseline time window, heart rate analysis interval and blood pressure analysis interval duration in the typical features; normalize each absolute difference to obtain each normalized difference. The product of all normalized differences and preset weight coefficients is added together to obtain the feature difference between the current representation feature and the typical feature; the feature cluster corresponding to the minimum feature difference is taken as the target cluster. Calculate the arithmetic mean of the feature differences between all pairs of historical representation features within the target cluster, and use it as the mean difference. Calculate the ratio of the minimum feature difference to the mean difference, and use it as the first ratio; normalize the absolute difference between the positive integer 1 and the first ratio to obtain the feature matching degree.

[0013] An emotion event recognition system based on multimodal physiological parameters is disclosed. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of an emotion event recognition method based on multimodal physiological parameters.

[0014] The present invention has the following beneficial effects: This invention determines the baseline analysis interval through a respiratory rate time series and divides the analysis interval based on the dynamic response characteristics of heart rate and blood pressure relative to respiratory rate. This not only adapts to the response time differences of different physiological parameters but also fully captures the coordinated changes in physiological parameters triggered by emotions, making the subsequently determined representational features more consistent with the real emotional-physiological linkage. A pre-set set of historical representational features is introduced, and through similarity matching between current and historical representational features, as well as weighted fusion optimization of current representational features, features that better fit individual emotional-physiological responses are output, significantly improving the recognition accuracy of different individuals. This allows subsequent classification results to consider both the stability and dynamism of individual physiological characteristics. By constructing a respiratory-heart rate correlation model and combining heart rate variability parameter analysis with blood pressure changes, the inherent synergistic relationship of multimodal physiological parameters under emotional triggering is deeply explored, overcoming the limitations of single-parameter analysis and enhancing the richness of feature dimensions. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 The flowchart illustrates an emotion event recognition method based on multimodal physiological parameters, as provided in one embodiment of the present invention. Figure 2 This is an example diagram illustrating the process of determining heart rate analysis zones according to an embodiment of the present invention; Figure 3 This is an example diagram illustrating the process of determining blood pressure analysis intervals according to an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an emotion event recognition method and system based on multimodal physiological parameters proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details the specific scheme of the emotional event recognition method and system based on multimodal physiological parameters provided by this invention.

[0020] Please see Figure 1 The diagram illustrates a flowchart of an emotion event recognition method based on multimodal physiological parameters according to an embodiment of the present invention. The method includes: S101: Real-time acquisition of multimodal physiological parameter data of the target object, generating a time series of physiological parameters, wherein the multimodal physiological parameter data includes at least respiratory rate, heart rate and blood pressure.

[0021] It is important to understand that respiratory rate, heart rate, and blood pressure have a clear synergistic regulatory relationship in the physiological response to emotions: changes in respiratory rate affect heart rate variability through neuro-humoral regulation, while heart rate and vasomotor status jointly determine blood pressure fluctuations. The dynamic correlation characteristics of the three can comprehensively reflect the physiological response pattern triggered by emotions. Therefore, respiratory rate, heart rate, and blood pressure are selected as core physiological parameters.

[0022] It should be noted that physiological parameter data can be collected using existing non-invasive wearable devices. For example, respiratory motion signals can be collected using chest and abdominal strain sensors, and the respiratory rate can be calculated after filtering and peak detection processing.

[0023] It should be noted that the sampling frequency of the device is dynamically adjusted according to the physiological parameters. The sampling frequency of respiratory rate and heart rate is no less than 1Hz (i.e., 1 sample per second) to ensure the continuity of temporal changes, while blood pressure data adopts a periodic acquisition mode according to the physiological response characteristics.

[0024] It should be noted that, in order to ensure the alignment of different physiological parameters in the time dimension, so as to lay the foundation for subsequent correlation analysis of different physiological parameters, the raw physiological parameter data collected are arranged in the order of timestamps during the generation of physiological parameter time series, forming a structured physiological parameter time series. That is, the respiratory rate, heart rate and blood pressure time series use the same data structure, and record the dynamic changes of the corresponding physiological parameters in the time dimension.

[0025] S102: Determine the baseline time window corresponding to the emotional event based on the changing trend of the respiratory rate time sequence.

[0026] It is important to understand that, from a physiological regulatory perspective, respiration is regulated by the brainstem, limbic system, and cortex, and is the only physiological signal that runs through the three levels of "reflex → autonomy → consciousness." When an emotional event is triggered, changes in respiratory rate usually precede parameters such as heart rate and blood pressure. The temporal characteristics of respiratory rate (such as frequency value, rhythm stability, and fluctuation amplitude) show significant regular changes during emotional events. In contrast, changes in heart rate and blood pressure may be affected by non-emotional factors such as exercise and body position, and need to be indirectly achieved through neuro-humoral regulation, resulting in a delayed response. Individual differences in respiratory rate are relatively stable, and baseline values ​​are easily obtained in a calm state. When an emotional event occurs, the respiratory rate time series deviates from the baseline and shows a clear trend change. Therefore, choosing the respiratory rate time series as a benchmark to divide the time window of emotional events can not only objectively define the complete cycle of emotional events, but also provide a clear time boundary for subsequent analysis of the delayed responses of parameters such as heart rate and blood pressure.

[0027] Emotional events refer to the physiological response processes related to emotions triggered by internal and external environmental stimuli, which can be captured by physiological parameters. During this process, physiological parameters such as respiration, heart rate, and blood pressure change in tandem. Therefore, this embodiment focuses only on the fluctuation patterns of objective physiological parameters triggered by emotions.

[0028] For example, when the target is in a state of tension or excitement, their breathing rate may first show a rhythmic increase or decrease, followed by adaptive changes in heart rate and blood pressure. This complete process of coordinated changes in physiological parameters is defined as an emotional event.

[0029] In this embodiment, a baseline respiratory rate value is obtained through statistical analysis based on a respiratory rate time series. The moment when the respiratory rate time series first exceeds a first floating threshold is taken as the starting point of a reference time window, wherein the first floating threshold is determined based on a preset first proportion upwards from the baseline respiratory rate value. The moment when the respiratory rate time series first falls below a second floating threshold is taken as the ending point of the reference time window, wherein the second floating threshold is determined based on a preset second proportion downwards from the baseline respiratory rate value. The reference time window is defined by the starting point and the ending point.

[0030] The starting point is the first time point at which an emotional event causes the respiratory rate to deviate significantly from the baseline respiratory rate value.

[0031] It should be noted that the threshold is obtained by floating the baseline respiratory rate upwards by a preset first percentage (e.g., 10%). For example, if the baseline respiratory rate is 17 breaths / minute and the preset first percentage is 10%, then the first floating threshold is 17 × (1 + 10%) = 18.7 breaths / minute.

[0032] It should be noted that, in order to avoid momentary fluctuations (such as an occasional rapid breath) being misjudged as the initiation of an emotional event, the breathing frequency time sequence is traversed. When the breathing frequency of multiple consecutive sampling points (such as 5 sampling points) exceeds the first floating threshold for the first time, the first moment of the multiple consecutive sampling points that exceeds the first floating threshold is determined as the starting point.

[0033] The termination point corresponds to the time when the emotional event ends and the breathing rate returns to normal.

[0034] It should be noted that the second floating threshold is obtained by adjusting the baseline respiratory rate by a preset second percentage (e.g., 5%). For example, if the baseline respiratory rate is 17 breaths / minute and the preset second percentage is 5%, then the second floating threshold is 17 × (1 - 5%) = 16.15 breaths / minute.

[0035] It should be noted that, considering that respiratory rate usually gradually decreases from above the baseline value when emotions subside, when the respiratory rate stabilizes again, that is, slightly below the baseline value, it indicates that breathing has returned to near normal. Therefore, when multiple consecutive sampling points (e.g., 5 sampling points) in the respiratory rate time series first fall below the second floating threshold and remain stable thereafter, the moment when the first of these multiple consecutive sampling points falls below the second floating threshold is determined as the termination point.

[0036] To accurately obtain the baseline value of respiratory rate, one possible implementation is to determine the baseline value of respiratory rate based on the mode of multiple consecutive time periods obtained by dividing the respiratory rate time sequence.

[0037] As an example, the respiratory rate time series is divided into multiple consecutive time periods according to a preset unit time; for each time period, the mode of the respiratory rate within the time period is calculated; the arithmetic mean of the modes of all time periods is calculated as the baseline value of the respiratory rate.

[0038] It should be noted that the specific value of the preset unit time is determined according to the actual situation, and this embodiment does not make a specific limitation. For example, the respiratory rate time sequence is divided into multiple continuous time periods in segments of 5 minutes each.

[0039] The mode is the respiratory rate value that occurs most frequently.

[0040] S103: Based on the hysteresis response characteristics of the heart rate time series and blood pressure time series relative to the respiratory rate series within the baseline time window, the heart rate analysis interval and blood pressure analysis interval are determined respectively; the baseline time window, heart rate analysis interval, and blood pressure analysis interval together constitute the current representation characteristics of the current emotional event.

[0041] It is important to understand that changes in respiratory rate are directly driven by neural regulation triggered by emotional stimuli, resulting in the fastest response. Heart rate is influenced by both respiratory rhythm and neurohumoral regulation, and its changes typically lag behind respiration (for example, heart rate only shows a significant upward trend about 10-30 seconds after respiratory rate increases). Changes in blood pressure depend on the coordinated adjustment of heart rate and vascular resistance, exhibiting an even more pronounced lag. Therefore, changes in physiological parameters triggered by emotional events do not occur synchronously, but rather follow a progressive response chain of respiration → heart rate → blood pressure. This time difference is known as the lag response characteristic.

[0042] In this embodiment, based on the instantaneous change trend of the respiratory rate time sequence within a baseline time window, slow-rhythm breathing periods with rhythmic slowing of respiratory rate are identified; the standard deviation of respiratory rate within the slow-rhythm breathing period is calculated, and the reciprocal of the standard deviation is normalized to obtain the respiratory rhythm parameter; the ratio of the duration of the slow-rhythm breathing period to the duration of the baseline time window is calculated as the time proportion; the product of the respiratory rhythm parameter and the time proportion is normalized to obtain the regulatory participation parameter; based on the baseline time window and the regulatory participation parameter, the heart rate analysis interval is determined; based on the baseline time window, the regulatory participation parameter, and the heart rate variability parameter, the blood pressure analysis interval is determined.

[0043] Heart rate analysis intervals are used to mark key periods in the heart rate response to respiration, reflecting intermediate stages of neural regulation.

[0044] Blood pressure analysis intervals are used to mark key periods in the blood pressure response to heart rate, reflecting the final effect of physiological regulation.

[0045] It should be noted that in normal life scenarios, the target's emotions will always fluctuate slightly, and their breathing rate will also fluctuate to some extent. Therefore, the duration of the baseline time window cannot be zero.

[0046] It is important to understand that if the standard deviation within a certain slow-paced breathing period is smaller, it indicates that the rhythm of slow-paced breathing is more stable and the possibility of active regulation is higher; if the proportion of a certain time is larger, it indicates that the "duration of active regulation is longer", that is, the longer the duration of the slow-paced breathing period, the more significant the regulatory behavior is.

[0047] In this embodiment, to accurately distinguish slow-rhythm breathing periods: the first-order difference sequence of the respiratory rate time series is calculated; the respiratory rate time series is traversed to identify sub-intervals that meet the rhythmic slowing condition, wherein the rhythmic slowing condition is that the respiratory rate is continuously lower than the respiratory rate baseline value within the sub-interval, the first-order difference sequence is continuously negative within the sub-interval, and the absolute value of the first-order difference is maintained within a preset amplitude range; all identified sub-intervals that meet the rhythmic slowing condition are collectively regarded as slow-rhythm breathing periods.

[0048] It is important to understand that when target subjects are affected by emotional events, they often exhibit emotional breathing (passive), i.e., spontaneously regulated breathing patterns. However, target subjects may also consciously and actively regulate their breathing patterns, indicating that there are differences in breathing regulation in response to emotional events among different target subjects (for example, some people experience a more pronounced slowing of breathing when relaxing). Furthermore, since the activation of the vagus nerve requires a certain amount of time for emotional events involving the target subject's active breathing, it can lead to delayed physiological responses. Therefore, further analysis of the respiratory-heart rate-blood pressure coordinated regulation mechanism during slow-paced breathing periods is necessary to appropriately extend the time interval of other physiological parameters in order to capture the complete physiological change process.

[0049] It should be noted that the calculation method of the first-order difference is a well-known technique in the art, and will not be described in detail in this embodiment.

[0050] It can be understood that the sign of the first-order difference represents the direction of change in respiratory rate, for example, This represents the first-order difference result of the i-th sampling point in the respiratory rate time sequence. A value greater than 0 indicates that the respiratory rate shows an increasing trend from the i-th sampling point to the (i+1)-th sampling point; if A value less than 0 indicates that the respiratory rate shows a decreasing trend from the i-th sampling point to the (i+1)-th sampling point.

[0051] It is understandable that the absolute value of the first-order difference represents the magnitude of the change. The larger the absolute value, the more drastic the change in respiratory rate between two adjacent sampling points (i.e., the i-th sampling point and the (i+1)-th sampling point).

[0052] It should be noted that slow-paced breathing essentially means that the respiratory rate is lower than the target subject's normal level (i.e., the baseline respiratory rate). This condition ensures that what is being identified is a slowdown rather than normal fluctuations. The negative value of the first difference indicates that the respiratory rate decreases point by point within the sub-interval, rather than fluctuating randomly or decreasing briefly, ensuring that a continuous slowing trend is captured. The stability of the absolute value of the first difference is the core manifestation of rhythmicity, indicating that the rate of slowing of the respiratory rate is uniform, rather than a sudden drop or fluctuating speed.

[0053] It should be noted that the specific value of the preset amplitude range is determined according to the actual situation, and this embodiment does not impose a specific limitation. For example, the preset amplitude range can be 0.2-0.8 times / minute.

[0054] The process of determining the heart rate analysis zone is as follows: Figure 2 As shown, it includes: S103-1: Extract the time intervals between adjacent peak times in the respiratory rate time series and the heart rate time series, and generate respiratory time interval sequences and heart rate time interval sequences respectively; use the least squares method to fit the respiratory time interval sequences and heart rate time interval sequences to construct a respiratory-heart rate correlation model.

[0055] For a respiratory rate time series, the peak moment refers to the moment when the respiratory rate is the highest. For heart rate time series, the peak moment refers to the moment when the amplitude of the heart rate waveform is the largest.

[0056] It should be noted that locating the peak moments in the time series is an existing technical method, which will not be elaborated in this embodiment. For example, all peak moments can be located by using peak detection algorithms (such as the sliding window maximum method).

[0057] It should be noted that the specific method of fitting using the least squares method is a technique well-known to those skilled in the art, and will not be elaborated upon in this embodiment. For example, using the respiratory time interval sequence... Heart rate time interval sequence as independent variable As the dependent variable, a linear relationship is established between the independent and dependent variables. Then, the sum of squared errors between the actual and predicted values ​​is minimized using the least squares method to solve for the optimal parameters k and b. Finally, a respiration-heart rate correlation model is obtained through fitting. The model can be expressed as follows: Where, k represents the model parameter, reflecting the strength of the correlation between respiratory rate and heart rate. The larger the value of k, the more significant the effect of changes in the respiratory cycle on the cardiac cycle (for example, when emotionally stressed, a 1-second shortening of the respiratory cycle may lead to a 0.2-second shortening of the cardiac cycle, k=0.2); the b value represents the theoretical value when the respiratory cycle is 0, which can be understood as the baseline value unaffected by breathing. This represents the respiratory time interval in a respiratory time interval sequence; This represents the heart rate time interval in the heart rate time interval sequence.

[0058] S103-2: Based on the time interval between the last adjacent peak moments in the respiratory rate time series, the corresponding heart rate time interval is predicted through the respiratory-heart rate correlation model, and the corresponding heart rate time interval is used as the heart rate time series lag.

[0059] It is understandable that, since the heart rate response lags behind the respiratory rate response, by selecting the time interval between the last adjacent peak moments in the respiratory rate time series, i.e. the latest state of the respiratory cycle, the heart rate response cycle corresponding to the latest respiratory cycle can be predicted through the respiratory-heart rate correlation model, and this heart rate response cycle can be used as the heart rate time series lag.

[0060] S103-3: Add the regulation participation parameter to the positive integer 1 to obtain the expansion coefficient; calculate the product of the heart rate time lag and the expansion coefficient as the heart rate expansion duration.

[0061] It is important to understand that if the regulatory participation parameter is higher, it reflects a stronger emotion, and correspondingly, the activity of heart rate regulation in emotional events is also higher; if the heart rate time lag is greater, it reflects a longer lag time of heart rate relative to respiratory rate. Therefore, it is necessary to adapt to increasing the duration of the time interval corresponding to heart rate.

[0062] S103-4: Overlay the baseline time window with the heart rate extension duration to obtain the heart rate analysis interval.

[0063] It should be noted that, based on the starting point of the baseline time window, the heart rate timing lag is shifted backward (to ensure that the heart rate analysis interval begins at the moment the heart rate response begins); based on the ending point of the baseline time window, the heart rate extension duration is superimposed backward (to ensure that the heart rate analysis interval covers the duration and decay of the heart rate response).

[0064] For example, if the baseline time window is [10s, 100s], the heart rate timing lag is 0.9 seconds, and the extended duration is 1.17 seconds, then the heart rate analysis interval is [10+0.9=10.9s, 100+1.17=101.17s].

[0065] It is important to understand that sympathetic nerve activity is the core regulatory factor in the changes in blood pressure caused by emotions: when the sympathetic nervous system is excited (such as when tense or excited), it will cause the heart rate to increase and blood vessels to constrict, which will lead to an increase in blood pressure. Moreover, the lag in this change in blood pressure is more obvious and the duration is longer. Conversely, when the sympathetic nervous system is inhibited (such as when relaxed), the change in blood pressure is gradual and the duration is shorter. Therefore, further analysis of the degree of sympathetic nerve activity can serve as a key basis for judging the intensity and duration of blood pressure regulation.

[0066] The process of determining blood pressure analysis intervals is as follows: Figure 3 As shown, it includes: S103-5: Calculate the ratio of low-frequency power to high-frequency power in the heart rate variability parameters, as an indicator of sympathetic nerve activity.

[0067] It should be noted that the physiological parameters also include heart rate variability parameters, which include at least low-frequency power and high-frequency power.

[0068] It should be noted that high-frequency power represents the vagus nerve's regulatory activity on the heart. In actual working conditions, a situation where high-frequency power is zero is not the normal physiological state.

[0069] It is important to understand that low-frequency power is mainly regulated by both the sympathetic and vagus nerves, with the sympathetic nerves playing a dominant role (for example, low-frequency power increases significantly when emotionally stressed); high-frequency power is regulated only by the vagus nerve (for example, high-frequency power increases when relaxed). Therefore, the larger the ratio of low-frequency power to high-frequency power, the stronger the relative activity of the sympathetic nerves (autonomic nervous system balance leans towards sympathetic excitation).

[0070] S103-6: If the sympathetic nerve activity index exceeds the preset threshold, the difference between the sympathetic nerve activity index and the preset threshold is normalized to obtain a correction coefficient.

[0071] It should be noted that the specific value of the preset threshold is determined according to the actual situation, and this embodiment does not impose a specific limitation. For example, when analyzing the correlation data between sympathetic activity and blood pressure response in historical emotional events, the preset threshold can be set to 2.5.

[0072] It is important to understand that the correction coefficient is used to adjust the duration of blood pressure extension based on the strength of sympathetic nerve activity, avoiding insufficient or redundant interval coverage due to differences in sympathetic activity. Therefore, if the sympathetic nerve activity index exceeds the preset threshold, the larger the sympathetic nerve activity index, the greater the difference between the sympathetic nerve activity index and the preset threshold, reflecting significant sympathetic excitation, a larger increase in blood pressure, and a longer duration. The extension duration needs to be extended by the correction coefficient.

[0073] In this embodiment, when the sympathetic nerve activity index does not exceed the preset threshold, the correction coefficient is set to 1. In this case, there is no need to extend the blood pressure extension time. The blood pressure extension time can be calculated according to the calculation method of the reference heart rate extension time. In this case, the baseline time window is directly superimposed with the blood pressure extension time to obtain the blood pressure analysis interval.

[0074] S103-7: Calculate blood pressure extension duration by referring to the method for calculating heart rate extension duration.

[0075] S103-8: Calculate the product of the correction factor and the duration of blood pressure expansion, which is taken as the actual duration of blood pressure expansion.

[0076] S103-9: Overlay the baseline time window with the actual extended duration of blood pressure to obtain the blood pressure analysis interval.

[0077] It should be noted that, based on the starting point of the baseline time window, the heart rate time lag is shifted backward (to ensure that the blood pressure analysis interval begins at the moment when the blood pressure response begins); based on the ending point of the baseline time window, the actual duration of blood pressure extension is superimposed backward (to ensure that the blood pressure analysis interval covers the duration and decay of the blood pressure response).

[0078] S104: Determine the feature matching degree based on the similarity between the current representation feature and the preset set of historical representation features; based on the feature matching degree, perform weighted fusion optimization on the baseline time window, heart rate analysis interval and blood pressure analysis interval corresponding to the current emotional event to determine the optimized representation feature.

[0079] In this embodiment, historical representation features corresponding to multiple historical emotional events are extracted from historical physiological parameter data to form a historical representation feature sample set; cluster analysis is performed on the historical representation feature sample set to obtain multiple feature clusters; the arithmetic mean of all historical representation features in each feature cluster is calculated as the typical feature representing the feature cluster; the typical features of all feature clusters together constitute the historical representation feature set.

[0080] The historical representation feature set is a historical database that has been collected in advance and clearly labeled with the category of emotional events. For example, it records the heart rate time series data, blood pressure time series data, and respiratory time series data for each emotional event such as "happy, anxious, calm" recorded in the past month, as well as the corresponding baseline time window and heart rate and blood pressure analysis interval.

[0081] To accurately cluster multiple historical representation features reflecting similar historical emotional events into the same cluster, as an example, for any two historical representation features, the feature difference degree between any two historical representation features is calculated. The feature difference degree between each historical representation feature is used as the neighborhood radius, and the DBSCAN clustering method is used for clustering.

[0082] It should be noted that calculating the arithmetic mean of all historical representation features within each feature cluster is to calculate the arithmetic mean of the same historical representation feature within each feature cluster. For example, it is to calculate the arithmetic mean of the duration of all baseline time windows / heart rate analysis intervals / blood pressure analysis intervals within each feature cluster.

[0083] In this embodiment, for the typical features of each feature cluster in the historical representation feature set, the absolute difference between the current representation feature and the corresponding baseline time window, heart rate analysis interval, and blood pressure analysis interval duration in the typical features is calculated; each absolute difference is normalized to obtain a normalized difference; the product of all normalized differences and preset weight coefficients is added to obtain the feature difference degree between the current representation feature and the typical features; the feature cluster corresponding to the minimum feature difference degree is taken as the target cluster; the arithmetic mean of the feature difference degrees between all pairwise historical representation features in the target cluster is calculated as the difference mean; the ratio of the minimum feature difference degree to the difference mean is calculated as the first ratio; the absolute difference between the positive integer 1 and the first ratio is normalized to obtain the feature matching degree.

[0084] It should be noted that in this embodiment, the calculation methods for the feature difference between any two historical representation features, the feature difference between the current representation feature and the typical feature, and the feature difference between any two historical representation features are all the same, only the input parameters are slightly different, which will not be repeated in this embodiment.

[0085] It should be noted that this embodiment only focuses on three core indicators: the duration of the baseline time window, the duration of the heart rate analysis interval, and the duration of the blood pressure analysis interval.

[0086] It should be noted that the specific values ​​of the preset weight coefficients are determined according to actual needs, and this embodiment does not impose specific limitations. For example, they can be determined based on the contribution of each interval in historical data to the distinction of the category label of emotional events (the higher the contribution, the greater the weight). The weight of the baseline time window is 0.3, the weight of the heart rate analysis interval is 0.4, the weight of the blood pressure analysis interval is 0.3, and the sum of the weights is 1.

[0087] It is important to understand that if the mean difference is smaller, it indicates that the typical features within the target cluster are more consistent, and the reference value of the target cluster is higher. However, if the ratio of the minimum feature difference to the mean difference is much greater than 1, it means that the current representation feature may deviate from the typical features within the target cluster, and the feature matching degree will be lower.

[0088] It should be noted that in practical applications, physiological indicators such as heart rate and blood pressure are affected by factors such as individual state (e.g., breathing, muscle tension), environment (e.g., temperature, noise), and measurement errors (e.g., sensor accuracy, sampling frequency). Even for the same type of emotional event, the corresponding physiological parameter data cannot be completely replicated, so the mean difference will not be zero.

[0089] In this embodiment, typical features of the target cluster are extracted as historical reference features. Based on the feature matching degree, the durations of the baseline time window, heart rate analysis interval, and blood pressure analysis interval corresponding to the current representation feature and the historical reference feature are weighted and fused. The baseline time window, heart rate analysis interval, and blood pressure analysis interval obtained after weighted fusion together constitute the optimized representation feature.

[0090] It should be noted that the weighted fusion rules are as follows: the feature matching degree is used as the contribution weight of the historical reference feature duration, and the feature matching degree is positively correlated with the contribution weight of the historical reference feature duration; the difference between the positive integer 1 and the feature matching degree is used as the contribution weight of the current representation feature duration, and the difference between the positive integer 1 and the feature matching degree is positively correlated with the contribution weight of the current representation feature duration.

[0091] It's important to understand that when determining the contribution weight of historical reference feature duration, the feature matching degree is directly used as the weight, assuming... This is represented as the feature matching degree (within the range of 0-1). A higher feature matching degree indicates a better match. The larger the value, the more closely the historical reference features match the current emotional event, and the higher the weight should be assigned; conversely, the lower the feature matching degree, the better. The smaller the size, the lower the reference value of historical characteristics.

[0092] It's important to understand that when determining the contribution weight of the current feature representation duration, the difference between the positive integer 1 and the feature matching degree is used as the weight. Assuming... This represents the contribution weight of the current representation feature duration (i.e.) ), where the higher the feature matching degree, The smaller the value, the smaller the difference between the current representation characteristics and the typical characteristics within the target cluster, so we can refer more to historical patterns.

[0093] S105: Input the optimized representation features into the pre-trained classification model and output the category label corresponding to the emotional event; trigger the prompt message based on the frequency of occurrence of the category label within a preset time period.

[0094] It should be noted that the pre-trained classification model is a lightweight neural network model trained on a historical representation feature sample set and corresponding emotion labels. It has learned the mapping rules between physiological representation features and emotional event categories. For example, the feature of "long baseline time window and extended heart rate analysis interval" often corresponds to the category type of anxiety event (such as "anxiety-001"), and the feature of "short baseline time window and small fluctuation in blood pressure analysis interval" often corresponds to the category type of calm event (such as "REL-000").

[0095] It should be noted that the specific training process of the neural network model is a technique well known to those skilled in the art, and will not be described in detail in this embodiment.

[0096] Category labels are standardized identifiers / symbols used to precisely define the specific type of emotional event. Category labels directly correspond to emotional events; for example, "REL-000" represents a "calm and relaxing event".

[0097] For example, within a preset time period (e.g., 1 hour), the model outputs the total number of times the same category label (e.g., "anxiety-001") is generated, i.e., the frequency of occurrence; if the frequency of occurrence exceeds a preset trigger threshold, a prompt message is sent.

[0098] It should be noted that different trigger thresholds can be preset for different category labels. The specific value of the preset trigger threshold is determined based on prior experience. This embodiment does not make specific limitations. For example, for the category label "anxiety-001", the preset trigger threshold can be 2 times.

[0099] It should be noted that the specific form of the prompt message is not limited in this embodiment. For example, a prompt box can be used to send the prompt message.

[0100] An emotion event recognition system based on multimodal physiological parameters includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the emotion event recognition method based on multimodal physiological parameters.

[0101] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0102] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for emotion event recognition based on multi-modal physiological parameters, characterized in that, The method includes: Real-time acquisition of multimodal physiological parameter data of target objects to generate time series of physiological parameters, wherein the multimodal physiological parameter data includes at least respiratory rate, heart rate and blood pressure; Based on the changing trend of the respiratory rate time sequence, a baseline time window corresponding to the emotional event is determined; Based on the hysteresis response characteristics of the heart rate time series and blood pressure time series relative to the respiratory rate series within the baseline time window, the heart rate analysis interval and blood pressure analysis interval are determined respectively; the baseline time window, heart rate analysis interval, and blood pressure analysis interval together constitute the current representation characteristics of the current emotional event; The feature matching degree is determined based on the similarity between the current representation features and the preset set of historical representation features. Based on the feature matching degree, the baseline time window, heart rate analysis interval and blood pressure analysis interval corresponding to the current emotional event are weighted and fused for optimization to determine the optimized representation features. The optimized representation features are input into the pre-trained classification model, which outputs category labels corresponding to emotional events; based on the frequency of occurrence of category labels within a preset time period, prompt information is triggered. 2.The emotion event recognition method based on multi-modal physiological parameters according to claim 1, characterized in that, The process of determining the reference time window includes: Based on the respiratory rate time series, the baseline value of respiratory rate is obtained through statistical analysis; The moment when the respiratory rate first exceeds the first floating threshold in the time sequence is taken as the starting point of the reference time window, wherein the first floating threshold is determined based on the respiratory rate baseline value being floated up by a preset first proportion; The moment when the respiratory rate first falls below the second floating threshold in the time series is taken as the end point of the reference time window, wherein the second floating threshold is determined based on a preset second proportion of the respiratory rate baseline value. The reference time window is defined by the start point and the end point.

3. The emotion event recognition method based on multi-modal physiological parameters according to claim 2, characterized in that, The baseline value of respiratory rate, obtained statistically based on the respiratory rate time series, includes: The respiratory rate time sequence is divided into multiple continuous time periods according to a preset unit of time; For each time period, calculate the mode of the respiratory rate within that time period; Calculate the arithmetic mean of the modes for all time periods, and use it as the baseline for respiratory rate.

4. The emotional event recognition method based on multimodal physiological parameters according to claim 1, characterized in that, Multimodal physiological parameter data also includes heart rate variability parameters. The process for determining the heart rate analysis interval and blood pressure analysis interval includes: Based on the instantaneous change trend of respiratory rate time sequence within the baseline time window, identify slow-rhythm breathing periods where the respiratory rate shows a rhythmic decrease. Calculate the standard deviation of respiratory rate during the slow-rhythm breathing period, and normalize the reciprocal of the standard deviation to obtain the respiratory rhythm parameter; calculate the ratio of the duration of the slow-rhythm breathing period to the duration of the baseline time window as the time proportion. The product of the breathing rhythm parameter and the time percentage is normalized to obtain the regulatory participation parameter. Based on the baseline time window and regulatory participation parameters, the heart rate analysis interval is determined; Based on the baseline time window, regulatory participation parameters, and heart rate variability parameters, the blood pressure analysis interval is determined.

5. The emotional event recognition method based on multimodal physiological parameters according to claim 4, characterized in that, The method of identifying slow-rhythm breathing periods where the respiratory rate rhythmically decreases based on the instantaneous change trend of the respiratory rate time sequence within a reference time window includes: Calculate the first-order difference sequence of the respiratory rate time series; Traverse the respiratory rate time sequence and identify sub-intervals that meet the rhythmic slowing condition. The rhythmic slowing condition is that the respiratory rate is continuously lower than the respiratory rate baseline value within the sub-interval, the first difference sequence is continuously negative within the sub-interval, and the absolute value of the first difference is maintained within a preset amplitude range. All identified sub-intervals that meet the criteria for rhythmic slowing are collectively considered as slow-rhythm breathing periods.

6. The emotional event recognition method based on multimodal physiological parameters according to claim 4, characterized in that, The determination of heart rate analysis intervals based on the baseline time window and regulatory participation parameters includes: The time intervals between adjacent peak times in the respiratory rate time series and the heart rate time series are extracted to generate respiratory time interval sequences and heart rate time interval sequences, respectively. The least squares method is used to fit the respiratory time interval sequences and heart rate time interval sequences to construct a respiratory-heart rate correlation model. Based on the time interval between the last adjacent peak times in the respiratory rate time series, the corresponding heart rate time interval is predicted through the respiratory-heart rate correlation model, and the corresponding heart rate time interval is used as the heart rate time series lag. Add the participation parameter to a positive integer 1 to obtain the expansion coefficient; calculate the product of the heart rate lag and the expansion coefficient as the heart rate expansion duration; By overlaying the baseline time window with the extended heart rate duration, the heart rate analysis interval is obtained.

7. The emotional event recognition method based on multimodal physiological parameters according to claim 6, characterized in that, The determination of the blood pressure analysis interval based on the baseline time window, regulatory participation parameters, and heart rate variability parameters includes: The ratio of low-frequency power to high-frequency power in the heart rate variability parameters was calculated as an indicator of sympathetic nerve activity. If the sympathetic nerve activity index exceeds the preset threshold, the difference between the sympathetic nerve activity index and the preset threshold is normalized to obtain a correction coefficient. Calculate blood pressure expansion duration using the same method as heart rate expansion duration; Calculate the product of the correction factor and the blood pressure expansion duration as the actual blood pressure expansion duration; The baseline time window is superimposed with the actual extended duration of blood pressure to obtain the blood pressure analysis interval.

8. The emotional event recognition method based on multimodal physiological parameters according to claim 1, characterized in that, The process of determining the preset set of historical representation features includes: Historical representation features corresponding to multiple historical emotional events are extracted from historical physiological parameter data to form a historical representation feature sample set. Cluster analysis was performed on the historical characteristic feature sample set to obtain multiple feature clusters; Calculate the arithmetic mean of all historical representation features within each feature cluster, and use it as the typical feature representing the feature cluster. The set of historical representation features is composed of the typical features of all feature clusters.

9. The emotional event recognition method based on multimodal physiological parameters according to claim 8, characterized in that, The feature matching degree determination process includes: For the typical features of each feature cluster in the historical feature set, calculate the absolute difference between the current feature and the corresponding baseline time window, heart rate analysis interval and blood pressure analysis interval duration in the typical features; normalize each absolute difference to obtain each normalized difference. The product of all normalized differences and preset weight coefficients is added together to obtain the feature difference between the current representation feature and the typical feature; the feature cluster corresponding to the minimum feature difference is taken as the target cluster. Calculate the arithmetic mean of the feature differences between all pairs of historical representation features within the target cluster, and use it as the mean difference. Calculate the ratio of the minimum feature difference to the mean difference, and use it as the first ratio; normalize the absolute difference between the positive integer 1 and the first ratio to obtain the feature matching degree.

10. An emotion event recognition system based on multimodal physiological parameters, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 9.

Images