Respiratory training device based on multi-modal data fusion

Abstract

The invention discloses a respiratory training device based on multi-modal data fusion. The device comprises a multi-modal sensor, a data processing server and an interaction module, the multi-modal sensor is used for collecting multi-modal data of a user in real time; the data processing server extracts feature data of the user according to the multi-modal data; constructing a physiological feature sub-graph by taking each physiological feature as a node, constructing an emotional feature sub-graph by taking the emotional feature as a node, constructing a physiological genetic map of the user, and inputting the physiological genetic map into a physiological state analysis model to obtain a physiological state of the user; generating a respiratory training scheme according to the physiological state, wherein the respiratory training scheme comprises a multi-sensory training guide step, target key feature data and training intensity; or adjusting the current breathing training scheme based on the difference value between the extracted key feature data and the target key feature data and the training intensity; the interaction module is used for displaying the breathing training scheme. According to the invention, accurate and effective respiration monitoring can be provided for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training, and in particular to a breathing training device based on multimodal data fusion. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] In modern society, the incidence of various chronic respiratory conditions is increasing due to numerous factors, including environmental and lifestyle changes. Against this backdrop, respiratory rehabilitation and monitoring have become key tools for improving patients' quality of life and reducing healthcare costs. Their importance is becoming increasingly prominent, and they are receiving increasing attention from the medical community.

[0004] To meet the needs of respiratory rehabilitation and monitoring, a variety of devices for respiratory parameter monitoring and respiratory training have emerged on the market. These devices have provided some assistance to patients to a certain extent, but a deeper analysis reveals that they generally have some problems that need to be solved.

[0005] First, the problem of a single monitoring dimension is more prominent. The respiratory devices currently on the market usually limit their focus to basic parameters such as respiratory rate and depth. Moreover, in terms of parameter collection, they often rely on a single type of sensor, such as a flow sensor or a pressure sensor. This single sensor application mode means that the device can only obtain a certain type of information during the breathing process, and cannot fully and comprehensively reflect the relationship between breathing and other systems of the human body. In fact, the breathing process does not occur in isolation, it has a close synergistic relationship with the nervous system, muscular system, etc. For example, the nervous system is responsible for regulating the rhythm and depth of breathing, and the muscular system provides power support for respiratory movements. Without the coordinated analysis of these systems, it is difficult to accurately grasp the overall picture of the patient's respiratory function, and thus it is impossible to provide a comprehensive and effective basis for respiratory rehabilitation training.

[0006] Secondly, the lack of an intelligent feedback system is also a major drawback of existing respiratory equipment. During rehabilitation training, real-time guidance and adjustment suggestions are crucial to ensuring training effectiveness and advancing the recovery process. However, most existing equipment does not have this function. During training, patients can only rely on their own feelings and have no timely information on whether their training method is correct or the training intensity is appropriate. This lack of feedback in the training model can easily lead to patients using improper training methods, which in turn affects the training results. It may even cause additional damage to the body due to incorrect training methods, hindering the recovery process.

[0007] In summary, the current respiratory training devices and solutions on the market are significantly inadequate, and a multimodal data fusion respiratory training solution is urgently needed. This solution should be able to integrate data from multiple types of sensors to comprehensively monitor the respiratory process and related nervous and muscular systems. Through an intelligent feedback system, it should provide patients with precise and effective guidance during respiratory training. Only in this way can we truly meet patient needs, enhance the patient experience, and promote further development in the field of respiratory rehabilitation. Summary of the Invention

[0008] The embodiment of the present invention provides a breathing training device based on multimodal data fusion, which can fuse multimodal data to provide users with accurate and effective breathing monitoring and personalized guidance, and adjust and feedback breathing training plans in real time to improve user experience. The device includes: a multimodal sensor, a data processing server, and an interactive module;

[0009] The multimodal sensor is used to collect multimodal data of the user in real time and send it to a data processing server, wherein the multimodal data includes respiratory data, body temperature data, EEG data, EMG data and heart rate data;

[0010] The data processing server includes:

[0011] A feature extraction module, configured to extract feature data of the user based on the received multimodal data, wherein the feature data includes physiological features and emotional features;

[0012] The physiological gene map construction module is used to construct a physiological feature subgraph with each physiological feature as a node, and to construct an emotional feature subgraph with each emotional feature as a node. Based on the physiological feature subgraph and the emotional feature subgraph, the user's physiological gene map is constructed;

[0013] A physiological state analysis module is used to input the user's physiological gene map into the physiological state analysis model to obtain the user's physiological state;

[0014] A breathing training program generation module is configured to generate a breathing training program based on the user's physiological state, the breathing training program including multi-sensory training guidance steps, target key feature data, and training intensity; or to adjust the current breathing training program based on the difference between the key feature data in the extracted feature data and the target key feature data in the current breathing training program and the training intensity;

[0015] The interactive module is used to: display a breathing training program.

[0016] In an embodiment of the present invention, a multimodal sensor can obtain multimodal data of a user, wherein the multimodal data includes respiratory data, body temperature data, electroencephalogram data, electromyographic data, and heart rate data; the data processing server extracts characteristic data of the user based on the received multimodal data, wherein the characteristic data includes physiological characteristics and emotional characteristics; a physiological characteristic subgraph is constructed with each physiological characteristic as a node, and an emotional characteristic subgraph is constructed with the emotional characteristic as a node, and a physiological gene map of the user is constructed based on the physiological characteristic subgraph and the emotional characteristic subgraph; the physiological gene map of the user is input into a physiological state analysis model to obtain the physiological state of the user; a breathing training plan is generated based on the physiological state of the user, wherein the breathing training plan includes multi-sensory training guidance steps, target key feature data, and training intensity; or, based on the difference between the key feature data in the extracted characteristic data and the target key feature data in the current breathing training plan, and the training intensity, the current breathing training plan is adjusted; and the interactive module is used to: display the breathing training plan. Through the steps of the above device, multimodal sensors can acquire multimodal data from patients in real time. By fusing and analyzing multimodal data, not only can the accuracy of respiratory monitoring be improved, but the patient's lung function status and overall health status can also be more comprehensively assessed, thereby more accurately formulating respiratory training plans. By constructing a user's physiological genetic map, a personalized training plan that meets the patient's individual needs can be generated. The respiratory training plan can be adjusted based on real-time feedback during the respiratory training process, ensuring that every user receives targeted rehabilitation guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 Schematic diagram of the structure of a breathing training device based on multimodal data fusion in an embodiment of the present invention;

[0019] Figure 2 Flowchart of preprocessing multimodal data in an embodiment of the present invention;

[0020] Figure 3 This is a flow chart of eliminating motion artifacts in locally corrected EEG data according to an embodiment of the present invention;

[0021] Figure 4 This is a flow chart of the application of the adaptive wavelet denoising model in the implementation of the present invention;

[0022] Figure 5This is a flow chart of extracting user feature data in an embodiment of the present invention;

[0023] Figure 6 A flowchart of constructing a physiological gene map in an embodiment of the present invention;

[0024] Figure 7 is a training flow chart of a physiological state analysis model in an embodiment of the present invention;

[0025] Figure 8 Flowchart of adjusting the breathing training program in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0027] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0028] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0029] Figure 1 Schematic diagram of the structure of a breathing training device based on multimodal data fusion according to an embodiment of the present invention, comprising: a multimodal sensor 1, a data processing server 2 and an interaction module 3;

[0030] The multimodal sensor 1 is used to collect multimodal data of the user in real time and send it to a data processing server, wherein the multimodal data includes respiratory data, body temperature data, EEG data, EMG data and heart rate data;

[0031] The data processing server 2 includes:

[0032] A feature extraction module 21 is used to extract feature data of the user based on the received multimodal data, wherein the feature data includes physiological features and emotional features;

[0033] The physiological gene map construction module 22 is used to construct a physiological feature subgraph with each physiological feature as a node, and to construct an emotional feature subgraph with each emotional feature as a node, and to construct a user's physiological gene map based on the physiological feature subgraph and the emotional feature subgraph;

[0034] Physiological state analysis module 23, used to input the user's physiological gene map into the physiological state analysis model to obtain the user's physiological state;

[0035] A breathing training program generation module 24 is configured to generate a breathing training program based on the user's physiological state, the breathing training program including multi-sensory training guidance steps, target key feature data, and training intensity; or to adjust the current breathing training program based on the difference between the key feature data in the extracted feature data and the target key feature data in the current breathing training program, and the training intensity;

[0036] The interactive module 3 is used to display the breathing training program.

[0037] In an embodiment of the present invention, a multimodal sensor can acquire a patient's multimodal data in real time. By fusing and analyzing the multimodal data, not only can the accuracy of respiratory monitoring be improved, but the patient's lung function status and overall health status can also be more comprehensively assessed, thereby more accurately formulating a respiratory training program. By constructing a physiological genetic map of the user, a personalized training program that meets the individual needs of the patient can be generated. The respiratory training program can be adjusted based on real-time feedback during the respiratory training process, thereby ensuring that each user receives targeted rehabilitation guidance.

[0038] A detailed introduction is given below.

[0039] In one embodiment, the respiratory data includes respiratory rhythm data, respiratory pattern, and respiratory change data. The respiratory pattern includes different proportions of chest breathing and abdominal breathing. The respiratory change data includes the respective states of chest and abdominal breathing changes, the order of occurrence, the amplitude of change, respiratory coordination, oxygen uptake data, etc. The respiratory data can be collected by a respiratory monitoring unit, which includes an abdominal / chest respiratory sensor, a respiratory mask, etc. The abdominal / chest respiratory sensor obtains respiratory rate and respiratory intensity signals in real time by sensing the mechanical movement of the thorax or abdomen. The abdominal / chest respiratory sensor can accurately capture the dynamics of each breath of the patient, has high data acquisition sensitivity, and is suitable for long-term stable monitoring. The abdominal / chest respiratory sensor is in the form of a belt, composed of an elastic belt and a piezoelectric sensor, and is used to be tied to the chest or abdomen for detection. The respiratory mask integrates a temperature sensor, a pressure sensor, a respiratory flow rate sensor, an oxygen sensor, and a carbon dioxide sensor into the mask for monitoring the patient's breathing.

[0040] Oxygen data includes oxygen intake and carbon dioxide exhaled.

[0041] Body temperature data includes facial / torso temperature distribution data. Body temperature data can be collected by infrared thermal imaging units and is an auxiliary indicator of respiratory efficiency.

[0042] EEG data includes brain waves that reflect brain states such as relaxation, concentration, and stress. EEG data can be collected using a head posture sensor, which uses a single-channel dry electrode EEG to focus on monitoring theta / beta waves in the prefrontal lobe to obtain brain waves.

[0043] The electromyographic data is collected by the electromyographic acquisition unit. The purpose of collecting electromyographic data is to monitor key areas, such as muscle tension in the shoulders, back, and waist, and the symmetry of contraction function, in order to evaluate muscle tension and bilateral coordination.

[0044] Heart rate data, including heart rate variability, is acquired through a heart rate sensor. Heart rate sensors can be integrated with heart rate and blood oxygen monitoring devices, which record the patient's heart rate changes in real time. This data is used to analyze the impact of breathing training on the cardiovascular system. Heart rate data is an important basis for assessing breathing rhythm and depth adjustments, and can reflect the regulatory effects of breathing training on the autonomic nervous system. Blood oxygen levels can also be recorded to assess the patient's inhaled oxygen content and whether oxygen delivery efficiency has improved. The heart rate and blood oxygen monitoring sensor is a finger-clip type, designed to be clamped on the finger for measurement.

[0045] After obtaining the above data, it needs to be cleaned and preprocessed.

[0046] Figure 2 This is a flowchart of preprocessing multimodal data in an embodiment of the present invention. In one embodiment, after obtaining the multimodal data of the user, the process further includes:

[0047] Step 201: aligning timestamps of all multimodal data;

[0048] Obtaining timestamp information: For signals with different sampling rates and delays, the first step is to extract or append corresponding timestamp information from the multimodal data. Timestamps can represent time points or time intervals in multimodal data.

[0049] Unified timestamp format: Convert the timestamp information of different modal data into a unified time reference frame and format to ensure that they are temporally comparable.

[0050] Preliminary alignment based on timestamps: Based on the timestamp information, the multimodal data is preliminarily aligned on the time axis so that the modal data with the same or similar timestamps correspond in position.

[0051] Step 202: Perform sliding window cross alignment on the multimodal data after time stamp alignment;

[0052] Determine the window size and step size: Choose an appropriate sliding window size. The window size should be determined based on the characteristics of the multimodal data and the analysis requirements. Generally, it should contain sufficient characteristic information of the multimodal data. Also, determine the step size of the window as it slides across the signal. The step size determines the interval at which the window moves. A smaller step size can improve alignment accuracy, but it also increases the computational effort.

[0053] Calculate the cross-correlation function: For each sliding window position, calculate the cross-correlation function of the multimodal data within that window. The cross-correlation function measures the similarity between the two modal data at different delays. The optimal delay between the multimodal data is determined by finding the peak of the cross-correlation function.

[0054] Search for Maximum Correlation Peak: Searches for the largest peak in the calculated cross-correlation function values. The delay corresponding to this peak is the estimated relative delay between the two modal data in the current window.

[0055] Window sliding and iteration: Move the sliding window according to the set step size and repeat the previous two steps to analyze different parts of the multimodal data to obtain a series of delay estimates.

[0056] Delay estimation and alignment: Statistical analysis is performed on the multiple delay estimates, such as the average or median, as the final delay estimate. Based on this delay estimate, the multimodal data is shifted or adjusted accordingly to achieve temporal alignment of the different modal data.

[0057] Step 203: locally correcting the interference portion of the multimodal data after the sliding windows are cross-aligned using an adaptive filter based on a threshold change;

[0058] The interference part includes sudden electromyographic interference, EEG transient artifacts, respiratory disturbances, etc.

[0059] According to the characteristics of multimodal data and the type of interference, the appropriate adaptive filter type is selected. Common adaptive filters include least mean square (LMS) filters and recursive least squares (RLS) filters.

[0060] If the interference is mainly Gaussian white noise and the data changes relatively smoothly, the LMS filter may be a better choice because it is computationally simple and has good convergence performance. However, if the data has strong time-varying characteristics, the RLS filter may be more adaptable to rapid changes in the data, although its computational complexity is higher.

[0061] Initialize the parameters of the selected adaptive filter. For the LMS filter, for example, you need to set the step size parameter μ, which determines the speed at which the filter coefficients are updated. A large step size results in rapid filter convergence but can lead to instability. A small step size results in good filter stability but slow convergence. Experimentation is often required to determine an appropriate initial step size, such as a value between 0.01 and 0.1.

[0062] At the same time, the coefficient vector of the filter is initialized, which can generally be initialized to a zero vector or a random small vector.

[0063] The integrated multimodal data is divided into multiple data blocks, each containing a certain number of samples. The size of the data block is determined based on the characteristics of the data and computing resources, generally ranging from tens to hundreds of samples. A sliding window approach is used to move the window sequentially over the data blocks. The window size can be the same as or slightly smaller than the data block size. For example, setting the window size to 80% of the data block size ensures some overlap between windows during movement, thereby better capturing changes in local data features.

[0064] Within each window, the threshold is dynamically adjusted based on the comparison of the data characteristics of the current window with the initial threshold. For example, if a feature value within the window exceeds the initial threshold frequently, this indicates that strong interference may exist in that area, and the threshold range needs to be appropriately expanded. Conversely, if the feature values ​​are all within the initial threshold range and are relatively concentrated, it indicates that the data in that area is relatively stable, and the threshold range can be appropriately narrowed.

[0065] Some heuristic methods can be used to adjust the threshold, such as linearly adjusting the threshold based on the proportion of samples exceeding the threshold. Assuming the initial threshold is T0 and the proportion of samples exceeding the threshold is p, the adjusted threshold T = T0 × (1 + k × p), where k is the adjustment coefficient, which needs to be determined through experimentation and is generally between 0.1 and 1.

[0066] The adjusted threshold is used to identify data points that may be interfered with within the window. Data points that exceed the threshold range are determined to be interference points.

[0067] An adaptive filter is used to correct identified interference points. Based on the characteristics of the undisturbed data points within the window, the filter continuously adjusts the coefficients to fit and correct the interference points, bringing them closer to the characteristic distribution of normal data. For example, an LMS filter continuously updates the coefficients to minimize the mean square error between the filter output and the expected output (the undisturbed data characteristics), thereby correcting the interference points.

[0068] Step 204, freezing the locally corrected EEG data when the EMG signal bursts;

[0069] This is done to avoid misjudgments. A myoelectric burst is typically manifested by a sudden increase in signal amplitude or a change in frequency characteristics. By setting an amplitude threshold, a burst can be identified when the amplitude of the myoelectric data exceeds the threshold. Alternatively, using spectrum analysis, a significant increase in energy within a specific frequency range (such as the high-frequency band where myoelectric data is primarily distributed) can also be considered an EMG burst.

[0070] Once an EMG burst is detected, a control signal is automatically triggered, temporarily freezing the subsequent analysis process. This means that during this period of intense EMG noise, no further feature extraction, classification, or other analysis operations are performed on the EEG data. This is because the EEG data is likely to be severely interfered with by the EMG data, and continued analysis is likely to yield erroneous results.

[0071] When the EMG data burst ends, that is, when the EMG data returns to normal levels (the amplitude drops below the threshold or the high-frequency energy returns to normal), the system will unfreeze the EEG data analysis and continue the EEG data analysis process according to the normal process. When resuming analysis, additional preprocessing of the EEG data accumulated during the freezing period may be required to remove residual EMG interference traces and ensure the accuracy of subsequent analysis.

[0072] Through this EMG-EEG cross-validation mechanism, the interference of EMG data on EEG analysis can be effectively reduced, and the reliability of EEG data analysis results can be improved. It is especially suitable for application scenarios that require high accuracy of EEG data, such as clinical diagnosis and brain-computer interface control.

[0073] Step 205: When the electromyographic signal does not burst, the motion artifacts in the locally corrected EEG data are eliminated based on the locally corrected respiratory data. Figure 3 This is a flow chart of eliminating motion artifacts in locally corrected EEG data according to an embodiment of the present invention, including:

[0074] Step 301, obtaining a respiratory phase of the locally corrected respiratory data, wherein the respiratory phase is divided into an exhalation phase and an inhalation phase;

[0075] Specifically, the respiratory phases can be dynamically classified and refined, not simply divided into inhalation and exhalation. Instead, clustering algorithms in machine learning, such as K-means clustering or Gaussian mixture models (GMM), can be used to dynamically classify and refine the respiratory phases. Based on the characteristics of respiratory data, such as changes in respiratory frequency and amplitude fluctuations, the respiratory process can be divided into multiple, more detailed sub-phases. For example, the inhalation phase can be further subdivided into the initial rapid inhalation phase, the middle steady inhalation phase, and the final slow inhalation phase. Developing different threshold adjustment and artifact removal strategies for each sub-phase can more specifically address motion artifacts at different stages and improve the overall artifact removal effect.

[0076] The respiratory phase is modeled using the Hidden Markov Model (HMM) from time series analysis. The HMM considers the time series characteristics and state transition probabilities of the respiratory phase. By learning from a large amount of respiratory data, a model is established that accurately describes the dynamic changes in the respiratory phase. This model can be used to predict possible future respiratory phase states, adjust relevant parameters in advance, optimize motion artifact removal, and improve the system's adaptability and real-time performance.

[0077] Step 302, obtaining electromagnetic environment data of the user during breathing;

[0078] Environmental sensors such as electromagnetic interference sensors and body position sensors can be deployed. Electromagnetic interference sensors monitor the electromagnetic intensity in the environment in real time, while body position sensors continuously track changes in the patient's body position and obtain relevant electromagnetic environment data.

[0079] Step 303, calculating the spectrum characteristics and signal complexity of the locally corrected EEG data;

[0080] Calculate the spectral characteristics of EEG data and analyze the energy distribution in different frequency bands in detail. For example, convert EEG data from the time domain to the frequency domain through Fourier transform to obtain spectral characteristics.

[0081] To evaluate the signal complexity of EEG data, algorithms such as approximate entropy and sample entropy can be used for calculation. These indicators can reflect the irregularity and complexity of EEG signals.

[0082] Step 304: construct a multivariate function model, input the spectrum characteristics and signal complexity, respiratory phase and electromagnetic environment data into the multivariate function model to obtain a preliminary threshold, and the multivariate function model determines the preliminary threshold based on the input data, the first sensitivity and the second sensitivity.

[0083] The calculation steps of the multivariate function model are as follows:

[0084] Before inputting various types of data into the multivariate function model, the data needs to be standardized to eliminate the differences in dimensions and value ranges among different data types and ensure that the model is equally sensitive to all input data.

[0085] Normalization of EEG spectrum features: Assume that the energy distribution of EEG data power spectrum density in different frequency bands is E1, E2, ..., E n (n is the number of frequency segments). Calculate the mean and standard deviation Normalized spectrum characteristics

[0086] Signal complexity normalization: Assume that the signal complexity of EEG data obtained by algorithms such as approximate entropy or sample entropy is C. Assume that the mean signal complexity of a large number of similar EEG data is The standard deviation is σ C The normalized signal complexity is

[0087] Respiratory phase encoding: The respiratory phase is numerically encoded, with the inhalation phase being 1 and the exhalation phase being 0, recorded as P.

[0088] Standardization of electromagnetic environment data: If the electromagnetic intensity measured by the electromagnetic interference sensor is I. Standardized electromagnetic environment data

[0089] Set the form of the multivariate function model, for example, using the form of linear weighted combination:

[0090] T=w1E norm +w2C norm +w3P+w4I norm +b, where T is the initial threshold, w1, w2, w3, w4 are the weights corresponding to the input data, and b is the bias term.

[0091] Initialize the weights w1, w2, w3, w4 and the bias term b. Random initialization can be used, for example, to randomly select values ​​in the interval (-1, 1). At the same time, set the first sensitivity s1 and the second sensitivity s2.

[0092] The weights were preliminarily adjusted according to the respiratory phase to reflect the differences in the impact of motion artifacts on EEG data in different respiratory stages.

[0093] When the respiratory phase P=1 (inspiratory phase), the weights are adjusted as follows:

[0094] w′1=w1×s1

[0095] w′2=w2×s1

[0096] w′3=w3×s1

[0097] w′4=w4×s1

[0098] When the respiratory phase P=0 (expiratory phase), the weights are adjusted as follows:

[0099] w′1=w1×s2

[0100] w′2=w2×s2

[0101] w′3=w3×s2

[0102] w′4=w4×s2

[0103] The normalized EEG spectrum feature E norm , signal complexity C norm , respiratory phase P and electromagnetic environment data I norm , and the adjusted weights w′1, w′2, w′3, w′4 and the bias term b are substituted into the multivariate function model for calculation. That is, T=w′1E norm +w′2C norm+ w′3P+w′4I norm +b, and get the preliminary threshold T.

[0104] Step 305 , using an adaptive wavelet denoising model, based on the preliminary threshold, EEG data and respiratory phase, the locally corrected EEG data is subjected to motion artifact removal processing to obtain denoised EEG data;

[0105] Step 306, performing quality assessment on the denoised EEG data;

[0106] During quality assessment, key indicators such as the improvement in signal-to-noise ratio and signal distortion are calculated. For example, the improvement in signal-to-noise ratio can be calculated by comparing the difference between the two before and after denoising. Algorithms such as mean square error can be used to assess signal distortion.

[0107] Step 307: If the quality evaluation result does not meet expectations, the optimization algorithm is started to adjust the parameters of the multivariate function model, and then the initial threshold is updated until the quality evaluation result meets expectations.

[0108] If the improvement in the signal-to-noise ratio is lower than a set threshold or the signal distortion is too high, an optimization algorithm (such as a gradient descent algorithm) is started.

[0109] The optimization algorithm automatically adjusts the parameters of the multivariate function model based on the quality assessment results. For example, it increases the two sensitivities to more strictly screen and remove artifacts, or adjusts the weights of various factors in the multivariate function model to make the threshold adjustment more consistent with the characteristics of actual EEG data and motion artifacts.

[0110] Specifically, an optimization algorithm based on reinforcement learning can be adopted.

[0111] The optimization algorithm based on reinforcement learning is as follows:

[0112] Build an intelligent agent based on a deep Q-network. The agent takes the current state of breathing phase, EEG data characteristics (spectral characteristics, signal complexity, etc.) and environmental information as input.

[0113] The agent selects an appropriate threshold adjustment strategy based on the current state, such as adjusting sensitivity, changing the parameters of the threshold calculation function, etc. This selection process is based on the experience accumulated by the agent in the previous learning process, that is, the Q value table.

[0114] The quality assessment results of the denoised EEG data are fed back to the agent as a reward signal. Good denoising results are rewarded positively, while negative rewards are given when the results are negative. The agent updates its Q-value table based on the reward signal, continuously learning and optimizing its strategy to maximize long-term rewards, thereby finding the optimal threshold adjustment strategy in various complex situations.

[0115] Figure 4 This is a flowchart of the application of the adaptive wavelet denoising model in the implementation of the present invention. In one embodiment, the adaptive wavelet denoising model is used to perform motion artifact removal processing on the locally corrected EEG data based on the preliminary threshold, EEG data, and respiratory phase to obtain denoised EEG data, including:

[0116] Step 401 : Taking the denoising effect evaluation index as the objective function, and according to the respiratory phase, optimizing and searching the wavelet basis and the number of decomposition layers of the adaptive wavelet denoising model.

[0117] Specifically, denoising effect evaluation metrics, such as minimizing mean squared error and maximizing signal-to-noise ratio, can be optimized using genetic algorithms or particle swarm optimization algorithms. For example, in a genetic algorithm, the wavelet basis and the number of decomposition levels are encoded to form an initial population. Through continuous iterations such as crossover and mutation, the optimal individuals are selected based on the objective function value, ultimately generating a customized wavelet basis and number of decomposition levels.

[0118] Step 402 , performing wavelet transforms at different resolutions on the locally corrected EEG data in the time dimension and the space dimension, respectively, to obtain wavelet coefficients at different resolutions;

[0119] For example, low-resolution and high-resolution wavelet decomposition can be performed. The decomposition results at different resolutions contain information of different levels of detail in the EEG data. Performing the decomposition in both the temporal and spatial dimensions can eliminate artifacts in both dimensions.

[0120] Step 403 , selectively fusing wavelet coefficients at different resolutions in the time dimension and the space dimension according to the respiratory phase and EEG data characteristics, to obtain fusion coefficients in the time dimension and fusion coefficients in the space dimension;

[0121] During the inhalation phase, high-resolution decomposition of wavelet coefficients is more frequently used for denoising in areas rich in high-frequency motion artifacts. During the exhalation phase, low-resolution decomposition of wavelet coefficients is combined to preserve the key features of the EEG data. Alternatively, deep learning-assisted denoising can be used: convolutional neural networks (CNNs) within deep learning are used to learn and classify the features of wavelet coefficients at different resolutions. First, the EEG data is subjected to multi-resolution wavelet decomposition. The wavelet coefficients at different resolutions are then used as input to the CNN, and the CNN model is trained to identify and distinguish EEG signal and motion artifact features. During the denoising process, the wavelet coefficients are adaptively processed based on the CNN classification results, removing the wavelet coefficients corresponding to motion artifacts while retaining the wavelet coefficients of the EEG data.

[0122] Step 404: Fusing the fusion coefficients in the time dimension and the fusion coefficients in the space dimension to obtain comprehensive wavelet coefficients;

[0123] Step 405 , performing inverse wavelet transform on the integrated wavelet coefficients to recover the denoised EEG data.

[0124] The embodiments of the present invention can adopt a parallel computing framework based on a graphics processing unit (GPU) to distribute computing tasks such as wavelet decomposition, threshold processing, and inverse wavelet transform to multiple computing cores for parallel execution. For example, the wavelet decomposition tasks of different channels are distributed to different GPU cores for simultaneous execution, thereby speeding up the overall denoising process. Through reasonable task division and data parallelization strategies, computing efficiency is greatly improved to meet the requirements of EEG data processing application scenarios with high real-time requirements, such as clinical real-time monitoring and brain-computer interface real-time control.

[0125] In step 102, characteristic data of the user is extracted based on the multimodal data, where the characteristic data includes physiological characteristics and emotional characteristics;

[0126] In one embodiment, the physiological characteristics include at least one of respiratory efficiency, rhythm frequency, amplitude spectrum, chest-abdominal phase difference, airflow asymmetry, muscle discharge period, symmetric tension fluctuation and postural residual potential, nonlinear change of heart rate variability in each respiratory cycle, pressure accumulation index, theta / beta wave purity index and respiratory rhythm disorder;

[0127] The emotional characteristics include at least one of microcirculatory disorder characteristics, mental fluctuation state and cross-modal emotional perception index;

[0128] Figure 5This is a flow chart of extracting user feature data in an embodiment of the present invention. The feature extraction module is used to:

[0129] Step 501, calculating the fluctuation curves of the nasal and perioral temperatures based on the respiratory data and body temperature data, and calculating the expiratory and inspiratory temperature difference ratio based on the fluctuation curves as the respiratory efficiency, and extracting the rhythm frequency, amplitude spectrum, chest-abdomen phase difference, and airflow asymmetry from the respiratory data;

[0130] Step 502 , extracting muscle discharge cycles, symmetrical tension fluctuations, and postural residual potentials based on the electromyographic data, and identifying microcirculatory disturbance characteristics during muscle tension based on the body temperature data and electromyographic data;

[0131] Specifically, the characteristics of microcirculatory disorders during muscle tension can be identified through the muscle microcirculation assessment model;

[0132] The muscle microcirculation assessment model can be annotated through expert judgment, combination with other physiological indicators, or use of control experiments.

[0133] Step 503: Analyze the noise characteristics of the EEG data at each reference electrode under different head postures, determine the optimal reference electrode based on the noise characteristics, calculate the theta / beta wave purity index, transient waveform density, and edge frequency based on the EEG data at the optimal reference electrode, and analyze the user's mental fluctuation state in combination with the anxiety estimation model;

[0134] For each potential reference electrode location, noise characteristics are extracted based on the corresponding EEG data. For example, the noise level can be measured by calculating the signal's standard deviation and power spectral density over a period of time. A larger standard deviation indicates greater signal fluctuations and a relatively higher noise level. The power spectral density reflects the distribution of noise energy at different frequencies.

[0135] Head posture data is fused and analyzed with EEG data. Considering that changes in head posture may cause variations in noise levels at different locations, the noise characteristics of the EEG data at each reference electrode are analyzed based on the real-time head posture. For example, when the head tilts or rotates, electrodes at certain locations may be affected by body movement, changes in electrode-skin contact, and other factors, leading to increased noise levels.

[0136] Based on the noise level assessment results, the point with the lowest noise level among all possible reference electrode positions is selected as the optimal reference electrode at the current moment. This can be achieved by comparing noise characteristic indicators at each point, such as selecting the point with the smallest standard deviation or the lowest power spectral density within a specific frequency range.

[0137] To ensure the stability and accuracy of reference electrode selection, a decision-making mechanism can be set up. For example, the selection can be based not only on the current noise level but also on the average noise level over a period of time. Alternatively, a noise threshold can be set so that a point is selected as the optimal reference electrode only when the noise level at that point is significantly lower than that at other points and below the threshold.

[0138] In one embodiment, the θ / β wave purity index is calculated based on the EEG data at the optimal reference electrode, including:

[0139] Remove motion artifacts from EEG data at the optimal reference electrode;

[0140] The proportion of theta and beta wave components in the EEG data after removing motion artifacts in the entire signal was calculated as the theta / beta wave purity index.

[0141] EEG data contains a variety of frequency components, among which theta waves (frequency range is about 4-8Hz) and beta waves (frequency range is about 13-30Hz) are closely related to different cognitive and physiological states of the brain. However, when actually collecting EEG signals, they are often affected by interference factors such as motion artifacts, resulting in the collected signals not being pure EEG signals. The embodiment of the present invention processes the collected EEG signals by a harmonic analysis method, separates the interference components such as motion artifacts, and then calculates the proportion of the real theta wave and beta wave components in the entire signal after removing the motion artifacts. This proportion is the theta / beta wave purity index.

[0142] Transient waveform density calculation: This function analyzes transient waveforms in EEG signals, which may be associated with rapid information processing or specific cognitive processes. By setting a threshold, the number of transient waveforms in the EEG signal is detected and the density of these transient waveforms per unit time is calculated. Changes in transient waveform density may be related to mental fluctuations, such as changes in anxiety or concentration.

[0143] Edge frequency calculation: Determine the edge frequency of the EEG signal, which typically refers to the frequency corresponding to the power drop from low frequency to high frequency in the power spectrum density (PSD). Edge frequency can reflect the overall frequency characteristics of brain activity and is related to the stability of the mental state. Lower edge frequencies may indicate a more relaxed or inhibited brain state, while higher edge frequencies may be associated with mental states such as tension, anxiety, or excitement.

[0144] Establishing an Anxiety Estimation Model: Model Algorithm Selection: Based on the data characteristics and research objectives, select an appropriate machine learning or statistical model algorithm, such as multivariate linear regression, support vector machine, or artificial neural network. These models can establish the relationship between EEG features (theta / beta wave energy ratio, transient waveform density, and edge frequency) and anxiety levels. Model Training: Divide the preprocessed data into a training set and a test set, typically in a certain ratio (e.g., 70% training set, 30% test set). Use the training set data to train the selected model, adjusting the model parameters to ensure that the model can predict anxiety scores as accurately as possible. During training, methods such as cross-validation can be used to prevent overfitting and improve model generalization. Model Evaluation: Use the test set data to evaluate the trained model, calculating metrics such as prediction accuracy, mean squared error, and correlation coefficient. Based on the evaluation results, determine whether the model performance meets the requirements. If the model performance is poor, adjustments can be made, such as adjusting parameters, adding features, or changing the model algorithm, until satisfactory results are achieved.

[0145] Real-time EEG data acquisition and feature extraction: In actual applications, the user's EEG data is collected in real time, and according to the above-mentioned feature extraction method, features such as theta / beta wave energy ratio, transient waveform density and edge frequency are calculated.

[0146] Quantifying Mental Fluctuations: Real-time EEG features are input into a trained anxiety estimation model to obtain a predicted value for the user's current anxiety level. By observing changes in the predicted anxiety level, the user's mental fluctuations can be quantified. For example, a significant increase or decrease in the predicted anxiety level over a period of time indicates significant fluctuations in the user's mental state. Furthermore, the changing trends of features such as the theta / beta wave energy ratio, transient waveform density, and edge frequency can be combined to further analyze the specific causes and manifestations of mental fluctuations. For example, a sudden change in the theta / beta wave energy ratio may reflect a shift in the brain's neural oscillation patterns, an increase in transient waveform density may be associated with a sudden focus of attention or emotional fluctuations, and an increase in edge frequency may indicate a state of tension or excitement.

[0147] Step 504, calculating the nonlinear change of heart rate variability in each respiratory cycle based on the respiratory data and heart rate data, and calculating the pressure accumulation index based on the frequency domain characteristics of heart rate variability and respiratory rhythm disorder;

[0148] The start and end times of each respiratory cycle are determined based on the respiratory data, and the respiratory cycle is divided into several equally spaced time points. For example, each respiratory cycle can be divided into 10 or 20 time points to accurately analyze the changes in heart rate variability within the respiratory cycle.

[0149] At each time point within each respiratory cycle, corresponding heart rate variability is calculated, including standard deviation (SDNN), root mean square difference between adjacent RR intervals (RMSSD), low-frequency power (LF), high-frequency power (HF), etc. These parameters can reflect different characteristics of heart rate variability, including overall heart rate variability, short-term variability, and autonomic nervous system activity.

[0150] Nonlinear analysis methods, such as calculating the slope and curvature of heart rate variability within the respiratory cycle or employing chaos analysis, can be used to accurately describe the nonlinear variation of heart rate variability within each respiratory cycle. For example, a curve fitting can be used to analyze the variation of heart rate variability over the respiratory cycle and observe whether it exhibits periodicity, symmetry, or other nonlinear characteristics.

[0151] Frequency-domain analysis of heart rate variability typically uses methods such as fast Fourier transform (FFT) or wavelet transform to convert heart rate variability from the time domain to the frequency domain, obtaining the power spectral density function of heart rate variability. The power values ​​in the LF and HF bands are calculated, along with the LF / HF ratio. These frequency-domain features can reflect the balance between the sympathetic and parasympathetic nervous systems in the autonomic nervous system.

[0152] Respiratory rhythm dysregulation is assessed by comparing the actual respiratory rate with the normal respiratory rate range and the stability of the respiratory rate. For example, the coefficient of variation (CV) of the respiratory rate can be calculated, which is the ratio of the standard deviation of the respiratory rate to the mean. A larger CV value indicates a more unstable respiratory rhythm and a higher degree of dysregulation.

[0153] A stress accumulation index (SCI) is constructed based on the frequency-domain characteristics of heart rate variability and respiratory rhythm disturbances. One possible approach is to weight the LF / HF ratio and the respiratory rate coefficient of variation (CV) and calculate the SCI. For example, SCI = w1 × LF / HF + w2 × CV, where w1 and w2 are weight coefficients determined based on actual conditions to adjust the relative contributions of HRV frequency-domain characteristics and respiratory rhythm disturbances to the SCI.

[0154] Step 505: Obtain a time series of the user's emotional data at different times based on the electromyographic data and the heart rate data, calculate the mutual information and phase synchronization rate between the time series of the emotional data and the time series of the respiratory data, and calculate a cross-modal emotion perception index based on the mutual information and the phase synchronization rate;

[0155] Emotional data includes emotion categories and emotion intensity. Emotion categories include happiness, sadness, anger, etc.

[0156] Emotional data and respiratory data are treated as two time series. For emotional data, the emotional index values ​​at each time point are grouped into a series according to the time order of collection. For respiratory data, the respiratory feature values ​​(such as respiratory frequency) at each time point are grouped into a series according to the time order.

[0157] The mutual information algorithm from information theory is used to calculate the mutual information between the emotion time series and the respiration time series. Mutual information measures the degree of dependence between two random variables and is used here to reflect the degree of information sharing between emotion and respiration. Common methods for calculating mutual information include histogram-based methods and kernel density estimation-based methods. Taking the histogram-based method as an example, the emotion and respiration data are first divided into several bins. Then, at each time point, the number of times the emotion and respiration values ​​fall into the corresponding bins is counted to form a joint histogram. Based on the joint histogram, the joint probability distribution of emotion and respiration, as well as their respective marginal probability distributions, is calculated. Finally, the mutual information value is calculated according to the definition formula of mutual information.

[0158] Extracting respiratory and emotional phases: For respiratory data, instantaneous phase information is extracted using methods such as the Hilbert transform to obtain a respiratory phase time series. For emotional data, assuming that emotional changes also have a certain periodicity or rhythmicity, the emotional time series can be analyzed, for example using methods such as wavelet transform, to extract phase information related to emotional changes and obtain an emotional phase time series.

[0159] Calculating the Phase Synchronization Rate: Phase synchronization analysis methods, such as calculating the standard deviation of phase differences and the phase lock value, are used to measure the degree of synchronization between the respiratory and emotional phases. For example, the standard deviation of phase differences is used to calculate the difference between the respiratory and emotional phases at each time point, followed by the standard deviation of these phase differences. A smaller standard deviation indicates a higher degree of synchronization between the respiratory and emotional phases, indicating a stronger coupling between emotion and respiration.

[0160] The steps to calculate the cross-modal emotion perception index are as follows:

[0161] Normalization: The calculated time series mutual information and phase synchronization rate are normalized so that their values ​​range from 0 to 1. This makes it easier to merge the two indicators and eliminates the influence of different indicator dimensions.

[0162] Determine weights: Based on the research objectives and data characteristics, determine the weights of time series mutual information and phase synchronization rate in the cross-modal emotion perception index. This weighting can be determined through expert experience, data analysis (such as principal component analysis and factor analysis), or machine learning methods (such as regression analysis and decision trees). For example, if time series mutual information is found to contribute more to emotion perception, it can be given a higher weight.

[0163] Fusion Calculation: Based on the determined weights, the normalized time series mutual information and phase synchronization rate are weighted and summed to obtain the cross-modal emotion perception index. The calculation formula can be expressed as: Cross-modal emotion perception index = w1 × normalized mutual information + w2 × normalized phase synchronization rate, where w1 and w2 are the weights of the time series mutual information and phase synchronization rate, respectively, and w1 + w2 = 1.

[0164] In step 103, a physiological feature subgraph is constructed with each physiological feature as a node, and an emotional feature subgraph is constructed with the emotional feature as a node;

[0165] When constructing a physiological feature subgraph using each physiological feature as a node, in addition to determining edge connections and weights based on time series correlation and causality, we also incorporate knowledge of physiological mechanisms to define edges. For example, based on the physiological coupling mechanism of the cardiovascular and respiratory systems, we determine the weight and direction of the edge between heart rate and respiratory rate. Furthermore, considering the brain's control over muscle movement, we clarify the edge connections between movement-related potential features in EEG signals and EMG features to more accurately reflect the inherent connections between physiological features.

[0166] When constructing the emotion feature subgraph, dynamic edges are introduced to reflect the dynamic nature of emotions. For example, for nodes with the same emotion category at different time points, edge weights are determined based on the rate of change of emotion intensity over time to illustrate the development trend of emotions. For nodes with different emotion categories, when there is an emotion transition (such as from calm to angry), the edge connections and weights are determined based on the probability and speed of the transition in historical data.

[0167] In step 104, a physiological gene map of the user is constructed based on the physiological feature subgraph and the emotional feature subgraph;

[0168] Figure 6 This is a flow chart of constructing a physiological gene map in an embodiment of the present invention. In one embodiment, constructing a user's physiological gene map based on a physiological feature subgraph and an emotional feature subgraph includes:

[0169] Step 601: Dynamically learn the weights between physiological features of the physiological feature subgraph and dynamically learn the weights between emotional features of the emotional feature subgraph through the heterogeneous graph attention network model;

[0170] Taking the physiological feature subgraph as an example, a heterogeneous graph attention network model with a hierarchical attention mechanism is adopted. The bottom layer of the heterogeneous graph attention network model focuses on the correlation between physiological features in the same modality, and the high layer realizes the fusion of physiological features of different modalities; if some physiological features have continuous changes in time, a time series attention module is added to capture the dynamic evolution of the features.

[0171] Prioritize using the parameters of a heterogeneous graph attention network model pre-trained on a large-scale public physiological dataset to initialize the current heterogeneous graph attention network model. This allows the heterogeneous graph attention network model to have a certain understanding of physiological characteristics at the beginning of training, accelerating convergence. If there is no suitable pre-trained model, use a normal distribution combined with an orthogonal initialization method. For parameters near the input layer, a relatively large initialization range is set to accommodate the diversity of the raw data; for parameters near the output layer, a relatively small initialization range is set to stabilize the output results of the heterogeneous graph attention network model.

[0172] The physiological feature subgraph is input into the heterogeneous graph attention network model. The attention mechanism within the model calculates the attention weights between each node. Each node updates its feature representation based on the features of neighboring nodes and the attention weights. The results of multiple attention heads are fused to generate new node features. Through the backpropagation algorithm, the parameters of the heterogeneous graph attention network model are adjusted based on the error calculated by the loss function, enabling dynamic learning of the weights between physiological features. During training, TensorBoard is used to monitor loss values, accuracy, and other metrics in real time, plotting training and validation set curves to promptly identify overfitting or underfitting issues and adjust the model structure and training parameters accordingly.

[0173] The process of dynamically learning the weights between the emotion features of the emotion feature subgraph is similar to the above and will not be repeated here.

[0174] Step 602: using a deep multi-channel structure, aligning the physiological feature subgraph and the emotional feature subgraph in a unified embedding space to obtain aligned physiological feature subgraph and emotional feature subgraph;

[0175] Determine the number of layers in the deep multi-channel architecture and the specific composition of each layer. This typically includes multiple Transformer blocks, each composed of a multi-head attention mechanism and a feed-forward neural network. For example, construct a deep multi-channel architecture with five Transformer layers, with eight attention heads in each layer to fully capture the complex relationships between different features.

[0176] Separate channels are designed for physiological and emotional feature subgraphs. Each channel processes the feature data of the corresponding subgraph. Within the channel, features are gradually extracted and transformed through a series of linear transformations, activation functions, and normalization operations.

[0177] Design a cross-channel interaction module to achieve information fusion and alignment between physiological and emotional features. An attention-based approach can be used to calculate the correlation weights between physiological and emotional features to guide information interaction and fusion.

[0178] For the weight matrices and bias vectors in deep multi-channel structures, appropriate initialization methods should be used. Common initialization methods include random initialization, such as using a uniform or normal distribution to generate initial values. Furthermore, to accelerate model convergence and improve training stability, some pre-trained parameters can be incorporated. If a pre-trained model in a related field is available, some of its parameters can be migrated to the current model, particularly in the underlying feature extraction layer.

[0179] The constructed physiological and emotional feature subgraphs are fed into the corresponding channels of the deep multi-channel architecture. The nodes in each subgraph represent different features, and the edges represent the relationships between features. This information serves as the initial input to the model and is used for subsequent feature transformation and alignment.

[0180] Within each channel, the input features are transformed layer by layer through the Transformer block. The multi-head attention mechanism enables the model to simultaneously focus on information from different feature nodes. By calculating attention weights, it dynamically allocates attention to each node, thereby extracting more representative feature representations.

[0181] In the feedforward neural network layer, the features processed by the attention mechanism are further nonlinearly transformed to enhance the model's expressiveness. At the same time, through normalization, the features output by each layer are standardized to prevent gradient vanishing or exploding problems.

[0182] The cross-channel interaction module calculates the alignment between physiological and emotional features. Using an attention mechanism, a physiological-emotional feature alignment matrix is ​​generated, reflecting the similarity and correlation between different types of features. Based on this matrix, a weighted fusion of physiological and emotional features is performed to obtain a preliminary aligned feature representation.

[0183] Define a loss function to measure the degree of sub-graph alignment. You can use a loss function based on cosine similarity, Euclidean distance, or KL divergence to calculate the difference between the aligned physiological and emotional features, and optimize the model parameters by minimizing the loss function.

[0184] Using the backpropagation algorithm, the weight parameters in the deep multi-channel structure are updated according to the gradient calculated by the loss function. During the training process, suitable optimizers such as Adam and Adagrad are used to adjust the learning rate and other hyperparameters to accelerate model convergence and prevent overfitting.

[0185] After multiple rounds of training, the physiological feature subgraph and the emotional feature subgraph are well aligned in the unified embedding space, and the aligned physiological feature subgraph and emotional feature subgraph are obtained.

[0186] Step 603: constructing a physiological gene map of the user based on the aligned physiological feature subgraph and emotional feature subgraph;

[0187] The aligned physiological feature subgraph and emotional feature subgraph are merged. During the merging process, the original attribute information of each feature node is retained, while the edge weights between nodes are updated to reflect the new association relationship between the features after the alignment process.

[0188] For the physiological and emotional feature nodes with strong correlations found during the alignment process, additional connecting edges can be added or the weights of existing edges can be enhanced to highlight the coupling relationship between these key features.

[0189] Use an appropriate graph visualization tool, such as Graphviz or NetworkX, to convert the integrated subgraphs into a visual physiological and genetic map. Use different colors, shapes, and sizes to distinguish physiological and emotional feature nodes, as well as the importance of nodes (for example, based on the contribution of the node to the alignment process or the closeness of its connection with other nodes).

[0190] Add interactive features to the graph to facilitate user exploration and understanding of the information within it. For example, when a user hovers over a node, the corresponding feature name, data source, and related explanation are displayed; when a user clicks an edge, the association weight between features and the related physiological or emotional mechanism are displayed.

[0191] Based on the structure and characteristics of the physiological gene map, in-depth analysis and interpretation are carried out. For example, community discovery algorithms are used to identify closely related feature groups in the map, and the potential connection between these groups and specific physiological states or emotional responses is analyzed to provide a scientific basis for personalized health management and emotional regulation.

[0192] In step 105, the user's physiological gene map is input into the physiological state analysis model to obtain the user's physiological state;

[0193] The user's physiological state may include tension, fatigue, shallow relaxation, deep relaxation, and of course, other states, which can be further classified.

[0194] Figure 7 The following is a flow chart of training a physiological state analysis model according to an embodiment of the present invention. The training steps of the physiological state analysis model include:

[0195] Step 701: For each feature data in the physiological gene maps of multiple users, physiological status is labeled according to the weight corresponding to the feature data to form a training set;

[0196] Step 702: Build a physiological state analysis model based on reinforcement learning, determine the network structure of the physiological state analysis model, regard the physiological state analysis model as an intelligent agent, and initialize the model parameters;

[0197] The network structure of the physiological state analysis model, such as the number of layers and nodes of the neural network, regards the model as an intelligent agent, enabling it to receive physiological gene map data as input and output the predicted physiological state.

[0198] Randomly initialize parameters such as weights and biases in the model and set a reasonable initial value range to lay the foundation for model training.

[0199] Step 703 , determining whether the user is in a stress state is used as a high-level task. If the user is in a stress state, determining whether the user is nervous or tired is used as a low-level task. If the user is in a non-stress state, determining whether the user is in a shallow or deep relaxation is used as a low-level task.

[0200] Determine the classification criteria for stress and non-stress states (tension or fatigue belongs to the category of stress state, shallow relaxation and deep relaxation belong to the category of non-stress state), so that the model can make a preliminary state judgment on the input physiological gene map data.

[0201] Set clear goals and requirements for different underlying tasks so that the model can learn features in a targeted manner.

[0202] Step 704: Perform the following training steps for the high-level tasks and the low-level tasks respectively:

[0203] Step 7041: input the training set into the physiological state analysis model, so that the intelligent agent takes actions to predict the physiological state based on the current model parameters and training set, and outputs the predicted results of the user's physiological state.

[0204] Step 7042: Compare the predicted result with the user's actual physiological state, and calculate a reward signal based on the comparison result.

[0205] If the comparison result is accurate, a positive reward is given; if the comparison result is incorrect, a negative reward is given. The setting of the reward signal should take into account factors such as the accuracy of the prediction and the importance of different physiological states.

[0206] In step 7043, the agent uses a reinforcement learning algorithm to update the agent's prediction strategy based on the obtained reward signal.

[0207] Reinforcement learning algorithms (such as Q-learning, Policy Gradient, etc.) adjust the model parameters (weights and biases) so that when faced with similar physiological genetic map inputs in the future, they can take "actions" that are more likely to obtain high rewards, that is, improve the accuracy of predictions.

[0208] In the aforementioned training steps, the model first trains on high-level tasks, enabling it to distinguish between stressful and non-stressful states. Then, for data in stressful and non-stressful states, it trains on corresponding low-level tasks, gradually optimizing the model's ability to predict specific physiological states (tension, fatigue, shallow relaxation, and deep relaxation). During training, new data is repeatedly input, rewards are calculated, and strategies are updated, with multiple rounds of iterative training until the model's performance stabilizes or meets pre-set training targets (e.g., accuracy and recall meet certain thresholds).

[0209] In addition, the trained model is evaluated using a reserved test dataset, calculating performance metrics such as accuracy, recall, and F1 score to comprehensively measure the model's performance in predicting the user's physiological state. Based on the evaluation results, the model is analyzed for any problems or deficiencies, such as low accuracy in predicting certain physiological states. Targeted adjustments are made to the model's structure, parameters, or training strategy, such as increasing training data, adjusting the reward function, and optimizing the network structure. Training and evaluation are then repeated to continuously optimize the model's performance and improve the accuracy and stability of predictions.

[0210] In step 106, a breathing training program is generated according to the user's physiological state, wherein the breathing training program includes multi-sensory training guidance steps, target key feature data, and training intensity;

[0211] 1. Breathing training program generation for users in stressful situations

[0212] Determine your training goals:

[0213] The core goal is to quickly relieve users' tension, relax the body and mind, and reduce sympathetic nervous system excitability. Through specific breathing exercises, the body's stress response is regulated, returning the body from a tense fight-or-flight mode to a calm rest-and-digest mode.

[0214] Designing multi-sensory training guide steps:

[0215] Visual guidance: Show users pictures or videos of tranquil and soothing natural scenes, such as a peaceful forest or a calm lake. Guide users to focus on slowly moving elements in the image, such as swaying leaves or gently rippling water, while breathing. This allows visual perception to align with the rhythm of breathing and soothe nervous tension.

[0216] Auditory Guidance: Play soft, slow music, such as adagios from classical music, or natural sounds, such as a gurgling stream or gentle birdsong. During the user's breathing process, guide them to match their breathing rhythm to the beat of the music, feeling the rise of the music melody as they inhale and the fall of the melody as they exhale, using auditory stimulation to assist in breathing regulation.

[0217] Tactile Guidance: Provide soft objects, such as silk handkerchiefs or plush toys, and encourage users to gently touch them while breathing, feeling their texture. During inhalation, guide users to feel the gentle pressure changes between their hands and the object, and during exhalation, experience the relaxing touch, reinforcing the body's sense of relaxation through touch.

[0218] Target key characteristic data:

[0219] Respiratory rhythm and frequency: When you are nervous, your respiratory rhythm often becomes faster and more disordered. The training goal is to gradually adjust the respiratory rhythm and frequency from a faster state, such as 20-25 times per minute, to 12-16 times per minute, so that breathing becomes smooth and regular.

[0220] Theta / Beta Purity Index: During stressful situations, beta waves are active, while theta waves are relatively suppressed. Training aims to improve the theta / Beta Purity Index, shifting the brain from an overactive beta-dominated state to a relaxed state with a relatively high theta wave activity. For example, if the index is 0.3 before training, it is expected to increase to 0.5-0.6 after training.

[0221] Symmetrical tension fluctuations: Tension can cause abnormal symmetrical muscle tension fluctuations. By using electromyography to monitor specific muscle groups (such as bilateral biceps and trapezius muscles, etc.), the goal after training is to reduce the amplitude of symmetrical tension fluctuations to make it close to the fluctuation range in a normal calm state, thereby reducing the imbalance caused by muscle tension.

[0222] Respiratory rhythm disorder: When you are nervous, your respiratory rhythm disorder is higher. The goal of training is to reduce this value and make your breathing more regular. For example, if the disorder is 0.4 before training, try to reduce it to 0.1-0.2 after training.

[0223] Set the intensity and complexity of your training:

[0224] Training Intensity: Initially, set each training session to 5-10 minutes, 3-4 times per day. As the user adapts to the training, gradually increase the training session duration to 10-15 minutes, maintaining the same frequency. During training, the breathing rhythm should not be too fast or too deep, and should be comfortable to avoid physical strain.

[0225] Training Complexity: Use simple abdominal breathing as a foundation. Instruct the user to lie flat on their back in bed or sit comfortably in a chair, placing one hand on their chest and the other on their abdomen. Inhale slowly through their nose, allowing their abdomen to expand like a balloon, feeling it rise while keeping their chest as still as possible. Exhale slowly through their mouth, gradually contracting their abdomen. Once the user has mastered abdominal breathing, simple breath counting techniques can be introduced, such as counting to three while inhaling and five while exhaling, to increase the complexity and focus required of the training.

[0226] 2. Breathing training program generation for fatigued users

[0227] Determine your training goals:

[0228] It aims to enhance the user's energy and vitality, improve physical fatigue, promote oxygen supply, accelerate the excretion of metabolic waste from the body, and help the body restore energy balance.

[0229] Designing multi-sensory training guide steps:

[0230] Visual Guidance: Display images or videos of dynamic sporting scenes, such as athletes running outdoors or flowers blooming in the sun, to inspire positive emotions and a sense of vitality. Instruct users to imagine themselves energetically participating in these scenes while breathing, drawing energy from their breath.

[0231] Auditory Guidance: Play upbeat, energetic music, such as light pop or inspirational instrumental music. Encourage users to breathe in and out in time with the music, feeling the energy of the music as they inhale and exhale to expel fatigue.

[0232] The music is generated based on multimodal data, and the music parameters = {

[0233] Beat: 60+10×(respiratory frequency-base frequency),

[0234] Brightness: normalized value of theta wave power × 100,

[0235] Harmonic complexity: sliding variance of the myoelectric symmetry index,

[0236] Power: The percentile of the high-frequency power of the heart rate data.

[0237] }

[0238] Tactile Guidance: Prepare a hot towel at an appropriate temperature and have the user apply it to their forehead or neck while breathing. As they inhale, they feel the comfort and relaxation brought by the heat. As they exhale, they experience the relief of fatigue as the heat dissipates. Self-massage can also be used, gently massaging areas like the temples and neck while breathing to enhance the tactile relaxation effect.

[0239] Target key characteristic data:

[0240] Respiratory rhythm and frequency: When you are tired, your breathing may be shallow and slow. The training goal is to moderately increase the respiratory rhythm and frequency from 8-12 times per minute to 14-16 times per minute, increase oxygen intake, and promote metabolism.

[0241] Theta / beta wave purity index: When the brain is fatigued, it may be in a low-activity theta wave excessive state. Training is expected to reduce the relative proportion of theta waves and increase the activity of beta waves, so that the theta / beta wave purity index is adjusted from a higher 0.7-0.8 to 0.4-0.5, allowing the brain to regain clarity and vitality.

[0242] Symmetrical tension fluctuations: Due to uneven muscle strength caused by fatigue, symmetrical tension fluctuations are large. The goal after training is to reduce these fluctuations and enhance muscle coordination. For example, through training, the symmetrical tension fluctuations of the leg muscles should be stabilized under electromyographic monitoring, approaching the fluctuation level of normal activity.

[0243] Irregular respiratory rhythm: Fatigue may lead to irregular respiratory rhythm. The training goal is to reduce irregular respiratory rhythm from a higher level of 0.3-0.4 to 0.1-0.2, and restore normal respiratory rhythm.

[0244] Set the intensity and complexity of your training:

[0245] Training Intensity: Initially, each training session should last 8-10 minutes, 2-3 times per day. As users become accustomed to the training, the duration of each session can be increased to 10-15 minutes, while maintaining the same frequency. During training, the breathing intensity can be increased moderately, to the point where you feel a slight deepening of the breath, but not a rapid increase in breath.

[0246] Training Complexity: Begin by transitioning from natural breathing to slightly deeper chest breathing. Instruct the user to sit or stand upright. When inhaling, expand the chest upward and outward, feeling the air fill the chest. When exhaling, gradually contract the chest. After mastering chest breathing, introduce coordination between breathing and body movements, such as extending the arms upward while inhaling and slowly lowering them while exhaling. This increases the coordination and complexity of the training and further enhances physical vitality.

[0247] 3. Breathing training program generation for users in shallow relaxation

[0248] Determine your training goals:

[0249] The goal is to help users further deepen their relaxation, enter a deeper state of physical and mental relaxation, improve concentration and inner peace, and enhance the body's self-healing ability.

[0250] Designing multi-sensory training guide steps:

[0251] Visual Guidance: Presenting soft, hazy light and shadow images, such as sunlight filtering through a tulle or flickering candlelight. Guiding users to focus their gaze on the subtle changes in light and shadow while breathing, the soothing visual experience can further relax the body and mind.

[0252] Auditory Guidance: Plays soothing, guiding sounds, such as meditation music and ethereal chanting. During the user's breathing process, they are instructed to fully immerse themselves in the sound, breathing it in as if drawing it in, and exhaling to expel distracting thoughts along with the breath.

[0253] Tactile Guidance: As you inhale, feel the aroma diffuse through your nose and lungs, bringing a relaxing dual sense of touch and smell. As you exhale, relax all your muscles. Simultaneously, you can perform a gentle body scan, starting with your feet and moving through the body to feel the relaxation, enhancing your tactile perception.

[0254] Target key characteristic data:

[0255] Respiratory rhythm and frequency: Breathing is relatively stable during shallow relaxation. The training goal is to further reduce the respiratory rhythm and frequency from 15-18 times per minute to 10-12 times per minute, making breathing deeper and slower, and promoting deep relaxation.

[0256] Theta / beta wave purity index: Shallow relaxation already has a certain amount of theta wave activity. Training hopes to further enhance the dominance of theta waves and improve the theta / beta wave purity index from 0.5-0.6 to 0.7-0.8, thereby deepening the degree of brain relaxation.

[0257] Symmetrical tension fluctuations: Symmetrical muscle tension fluctuations should be minimal in a state of shallow relaxation. The training goal is to maintain and further reduce symmetrical tension fluctuations, ensuring that the muscles are relaxed and balanced. Through electromyographic monitoring, symmetrical tension fluctuations in each major muscle group are maintained at a low and stable level.

[0258] Respiratory rhythm disorder: The respiratory rhythm is more regular during shallow relaxation. The training goal is to maintain a low respiratory rhythm disorder, keeping it at 0.1-0.2, further improve the regularity of breathing, and help deep relaxation.

[0259] Set the intensity and complexity of your training:

[0260] Training intensity: Each training session should last 10-15 minutes, 1-2 times a day. The training intensity should be moderate, with a steady and slow breathing rhythm, focusing on the depth and quality of breathing.

[0261] Complexity: Combine deep breathing with meditation. Instruct users to inhale slowly, letting air fill the abdomen and chest, feeling the body expand. Then, exhale slowly, feeling the body contract, while visualizing a tranquil scene, such as a peaceful beach. Once you've mastered this combination of breathing and meditation, try introducing apnea—pausing for a few seconds after inhaling before slowly exhaling—to further increase the complexity and relaxation effect of the exercise.

[0262] 4. Generation of breathing training programs for users in deep relaxation

[0263] Determine your training goals:

[0264] Consolidate and maintain the user's deep state of relaxation, further enhance the harmony and balance of body and mind, strengthen the body's self-healing ability and psychological resilience, and explore deeper inner peace and self-awareness.

[0265] Designing multi-sensory training guide steps:

[0266] Visual guidance: Display abstract and artistic works, such as Monet's Impressionist paintings, to guide users to feel tranquility and harmony from the colors and lines of the paintings while breathing, integrating visual beauty into the breathing experience and deepening inner peace.

[0267] Auditory Guidance: Play extremely soft, almost imperceptible natural sounds, such as distant wind or faint insect chirping, or use white noise. As the user breathes, they are encouraged to focus on these subtle sounds, as if their breathing and the sounds become one, achieving a deeper level of immersion.

[0268] Tactile Guidance: Use a soft feather to gently touch the user's arms, cheeks, and other areas as they breathe. As they inhale, they feel the gentle touch of the feather, and as they exhale, they experience a relaxed state of gentle contact with the outside world. This can be combined with gentle body movements, such as when breathing, to enhance the integration of tactile and physical perception.

[0269] Target key characteristic data:

[0270] Respiratory rhythm and frequency: When you are in deep relaxation, your breathing frequency is slow and steady. The training goal is to maintain and fine-tune your breathing rhythm frequency at 8-10 times per minute, and maintain the best breathing rhythm when your body is in a state of deep relaxation.

[0271] Theta / Beta Wave Purity Index: Theta waves are highly active in a state of deep relaxation. The training goal is to maintain a high theta / beta wave purity index at 0.8-0.9 to ensure that the brain is continuously in a state of deep relaxation and high cognitive integration.

[0272] Symmetrical tension fluctuations: Symmetrical muscle tension fluctuations are minimal during deep relaxation. The training goal is to maintain extremely low symmetrical tension fluctuations, ensuring complete muscle relaxation and optimal balance. High-precision electromyography monitoring maintains symmetrical tension fluctuations in each muscle group at a stable near-zero state.

[0273] Respiratory rhythm disorder: When you are in deep relaxation, your respiratory rhythm is extremely regular. The training goal is to maintain respiratory rhythm disorder at an extremely low level, close to 0, to ensure a high degree of regularity in breathing and promote ultimate relaxation of the body and mind.

[0274] Set the intensity and complexity of your training:

[0275] Training intensity: Each training session can be extended to 15-20 minutes, 3-4 times a week. The training intensity is very gentle, and the breathing is almost natural and unconscious. The focus is on maintaining a deep relaxation of body and mind.

[0276] Training Complexity: Introducing advanced breathing techniques, such as the Complete Breathing Method (a comprehensive combination of abdominal, thoracic, and clavicular breathing). Instructing users to inhale, first expand the abdomen, then the chest, and finally feel the collarbone lift slightly; exhale, contracting in the reverse order. After mastering the Complete Breathing Method, incorporating deep meditation, engaging in self-reflection and inner exploration during the breathing process, can enhance the depth and complexity of the training, achieving ultimate relaxation and self-improvement.

[0277] In step 107, when the user performs breathing training according to the breathing training program, the user's multimodal data is monitored in real time, and the breathing training program is adjusted based on the difference between the key feature data of the multimodal data and the target key feature data in the training program, and the training intensity.

[0278] Training fatigue monitoring mechanism: Combines heart rate changes, EEG rhythms, and EMG activity frequency to determine whether the user is experiencing "training fatigue" and intelligently shortens or adjusts the training rhythm.

[0279] Dynamic fine-tuning strategy: During training, if the user shows rhythm drift or mood swings, the system automatically scales the training intensity and adjusts the feedback rhythm to achieve a "coach-like correction" effect;

[0280] Breathing and emotion synchronization guidance mechanism: When the EEG shows anxiety fluctuations, it automatically recommends slowing down the rhythm guidance training, emphasizing "regulating the mind first and then regulating the breathing".

[0281] Adaptive difficulty adjustment mechanism: Dynamic Difficulty Adjustment (DDA) algorithm based on reinforcement learning automatically adjusts training intensity and complexity according to the user's real-time performance.

[0282] Figure 8This is a flow chart of adjusting a breathing training program in an embodiment of the present invention. In one embodiment, when a user performs breathing training according to the breathing training program, the user's multimodal data is monitored in real time, and the breathing training program is adjusted based on the difference between key feature data of the multimodal data and target key feature data in the training program and the training intensity, including:

[0283] Step 801: Obtain control parameters, which include proportional coefficients, integral coefficients, and differential coefficients. The meanings of the control parameters are shown in Table 1.

[0284] Table 1

[0285]

[0286] In the embodiment of the present invention, the control parameters can be dynamically adjusted, and the adjustment method is shown in Table 2.

[0287] Table 2

[0288]

[0289] Step 802: Monitor the user's multimodal data in real time to obtain key feature data, and calculate the performance error based on the difference between the key feature data and the target key feature data in the current breathing training program. The key features include respiratory rhythm frequency, theta / beta wave purity index, symmetrical tension fluctuation, and respiratory rhythm dysregulation.

[0290] The performance error is calculated as follows:

[0291] e(t)=Σ(w_i×(T_i-S_i(t))) / Σw_i

[0292] Where: T_i is the key feature data of the i-th target;

[0293] S_i(t): the i-th key feature data obtained by monitoring;

[0294] w_i: weights in the state evaluation matrix.

[0295] Table 3 gives a description of the key feature data and weights.

[0296] Table 3

[0297] physiological signals Target indicators Normalization method Weight breathe Respiratory rhythm frequency 1-(actual respiratory rhythm frequency / reference respiratory rhythm frequency) 0.4 EEG Theta / Beta Wave Purity Index (Actual value – Baseline value) / (Target value – Baseline value) 0.3 electromyography Symmetrical tension fluctuations min(left / right, right / left) 0.2 Heart rate Respiratory rhythm disorder (Actual value -0.15) / 0.35 0.1

[0298] Step 803, calculating the training intensity at the next moment based on the performance error, control parameters and current training intensity;

[0299] The difficulty at the next moment is calculated using the following formula:

[0300] D_{t + 1}=D_t + K_p×e(t)+K_i×∫e(t)dt+K_d×de(t) / dt

[0301] Table 4 shows the meanings of the parameter symbols in the training intensity formula.

[0302] Table 4

[0303]

[0304] Step 804, determine the safety constraint conditions;

[0305] To ensure training safety, when adjusting the breathing training plan, the following must be observed:

[0306] If D_{t + 1}>D_max, then D_{t + 1}=min(D_t×1.2,D_max), and the single - time increase does not exceed 20%. D_max is the maximum training intensity;

[0307] If D_{t + 1}<D_min, then D_{t + 1}=max(D_t×0.8,D_min), and the single - time decrease does not exceed 20%. D_min is the minimum training intensity.

[0308] Step 805, adjust the current breathing training plan according to the training intensity at the next moment and the safety constraint conditions.

[0309] For example, when the user's performance suddenly improves:

[0310] ① The current error e(t) decreases → the derivative term de(t) / dt becomes negative;

[0311] ② Appropriately reduce the difficulty increase amplitude;

[0312] ③ If the improvement continues, the cumulative negative value of the integral term prompts the difficulty to decrease;

[0313] This design can achieve the following in breathing training:

[0314] For beginners: slow and progressive difficulty increase (dominated by K_i);

[0315] For proficient users: fast and responsive adjustment (dominated by K_p);

[0316] During the fluctuation period: trend - predictive adjustment (dominated by K_d);

[0317] The structure of the interaction module is as follows:

[0318] Multi - dimensional state panel: visually display dynamic curves such as breathing rhythm, EEG relaxation degree, muscle tension, heart rate variability, etc. and a comprehensive state radar chart;

[0319] Specifically, the interactive interface can provide audio and video guidance, including breathing prompts: using voice to prompt the patient to inhale and exhale in a rhythm. For example: inhale, 1, 2, 3, 4; exhale, 1, 2, 3, 4, 5, 6.

[0320] Rhythm animation: animation shows the rhythm and depth of breathing to help patients intuitively understand the training requirements.

[0321] Respiratory target chart: Displays the patient's real-time respiratory data compared with the target data to help patients understand their training progress.

[0322] Patients train according to audio and video instructions, and their performance is monitored in real time. For example:

[0323] If the patient fails to take a deep breath according to the prompt tone, the system will remind the patient to adjust the breathing rhythm through vibration or voice.

[0324] If the patient's breathing depth does not reach the target value, the system will prompt the patient to increase the breathing depth through animation.

[0325] Breathing training program recommendation panel: automatically generates recommended training content based on the current state (such as "deep breathing rhythm training" and "relaxation and focus exercise");

[0326] Individual growth curve tracking panel: records user training times, status change trajectories, and training response curves, and supports timeline playback and trend analysis.

[0327] Users can switch between multiple feedback fusion modes on the panel, which supports automatic switching / fusion of multiple feedback methods and dynamically adjusts the combination mode based on user preferences, feedback effects and fatigue.

[0328] In one embodiment, the interaction module is further configured to:

[0329] A natural language guide is generated according to the breathing training program, and the natural language guide is presented in the form of a virtual coach.

[0330] In one embodiment, the device further includes a communication module 4 for realizing data transmission between the data processing server and the multimodal sensor and the interaction module through a variety of wireless connection methods.

[0331] Wireless connection methods include Bluetooth / Wi-Fi / LoRa, etc.

[0332] The device also includes a battery and an external interface (such as a quick-release connector, a USB interface, etc.). The data of this device can be uploaded to the cloud server through the communication module and interact with the mobile terminal / PC application (APP).

[0333] This device features a rugged design. The interactive module interface is a built-in LCD touchscreen that supports resistive touchscreen technology, making it suitable for use in various lighting conditions and outdoor environments, even with gloves or wet fingers. It incorporates a dual-gas sampling system, allowing for either a mixed chamber or a single-blow method for gas exchange measurement. The oxygen sensor is user-replaceable, and the carbon dioxide sensor is durable and long-lasting. The device is lightweight, easy to carry and operate, and it supports replaceable batteries, making it suitable for extended use.

[0334] The device is designed with a quick-release connector to facilitate connection between the mask and sensor, enhancing the practicality and ease of use of the device.

[0335] The interface is intuitive and the operation process is simple. The calibration interface supports automatic sensor calibration; the measurement interface provides real-time data curves and parameter display; the status interface displays the latest status of each measurement in real time, updating once a second; and the settings interface supports parameter configuration and OTA upgrades. The touch screen also provides interfaces for data logging, training plan display, training plan generation, training process, and personal center for data interaction during use.

[0336] In summary, the device proposed in the embodiments of the present invention uses multimodal sensors to acquire multimodal data from patients in real time. By integrating and analyzing this multimodal data, the accuracy of respiratory monitoring can be improved, and a more comprehensive assessment of the patient's lung function and overall health status can be provided, enabling more precise respiratory training plans to be developed. By constructing a user's physiological and genetic profile, a personalized training plan tailored to the patient's individual needs can be generated. The training plan can be adjusted based on real-time feedback during respiratory training, ensuring that each user receives targeted rehabilitation guidance. The device features cloud data synchronization, uploading the patient's training data and monitoring results to a cloud server to create a long-term health profile for the patient. Through in-depth analysis and trend analysis of long-term data, changes in the patient's health and potential problems can be promptly identified, providing doctors with scientific evidence to support the development of subsequent treatment and rehabilitation plans, thereby facilitating full-cycle health management for patients. The device is lightweight, features quick-release connectors, and has a replaceable battery, making it easy for patients to use anytime, anywhere. The integrated LCD touchscreen interface is intuitive and easy to use, allowing for convenient monitoring and training at home, making it suitable for daily self-management and rehabilitation training, reducing dependence on hospitals and specialized equipment. The device boasts excellent scalability and compatibility, allowing it to connect to more sensor types as needed, supporting future functionality expansion. It also supports over-the-air (OTA) updates, facilitating software upgrades and algorithm optimization, providing users with long-term technical support.

[0337] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A breathing training device based on multimodal data fusion, characterized in that: include: Multimodal sensors, data processing servers and interaction modules; The multimodal sensor is used to collect multimodal data of the user in real time and send it to a data processing server, wherein the multimodal data includes respiratory data, body temperature data, EEG data, EMG data and heart rate data; The data processing server includes: A feature extraction module, configured to extract feature data of the user based on the received multimodal data, wherein the feature data includes physiological features and emotional features; The physiological gene map construction module is used to construct a physiological feature subgraph with each physiological feature as a node, and to construct an emotional feature subgraph with each emotional feature as a node. Based on the physiological feature subgraph and the emotional feature subgraph, the user's physiological gene map is constructed; A physiological state analysis module is used to input the user's physiological gene map into the physiological state analysis model to obtain the user's physiological state; A breathing training program generation module is configured to generate a breathing training program based on the user's physiological state, the breathing training program including multi-sensory training guidance steps, target key feature data, and training intensity; or to adjust the current breathing training program based on the difference between the key feature data in the extracted feature data and the target key feature data in the current breathing training program and the training intensity; The interactive module is used to: display a breathing training program.

2. The device according to claim 1, wherein The data processing server further includes a data preprocessing module, which is used to: After obtaining the user's multimodal data, all multimodal data are timestamp aligned; Perform sliding window cross alignment on the multimodal data after timestamp alignment; An adaptive filter based on threshold variation is used to locally correct the interference part of the multimodal data after the sliding window cross alignment; Freeze the locally corrected EEG data during EMG signal bursts; When the electromyographic signal is not bursting, the motion artifacts in the locally corrected electroencephalographic data are eliminated according to the locally corrected respiratory data.

3. The device according to claim 2, wherein The data preprocessing module performs the step of eliminating motion artifacts in the locally corrected EEG data based on the locally corrected respiratory data, including: Obtaining a respiratory phase of the locally corrected respiratory data, wherein the respiratory phase is divided into an exhalation phase and an inhalation phase; Obtain electromagnetic environment data of the user during breathing; Calculate the spectral characteristics and signal complexity of the locally corrected EEG data; Constructing a multivariate function model, inputting the spectrum characteristics and signal complexity, respiratory phase and electromagnetic environment data into the multivariate function model to obtain a preliminary threshold, wherein the multivariate function model determines the preliminary threshold based on the input data, the first sensitivity and the second sensitivity; Adaptive wavelet denoising model is used to remove motion artifacts from the locally corrected EEG data according to the preliminary threshold, EEG data and respiratory phase to obtain denoised EEG data. Perform quality assessment on denoised EEG data; If the quality assessment result does not meet expectations, the optimization algorithm is started to adjust the parameters of the multivariate function model, and then the initial threshold is updated until the quality assessment result meets expectations.

4. The device according to claim 3, characterized in that The data preprocessing module performs motion artifact elimination processing on the locally corrected EEG data using an adaptive wavelet denoising model according to a preliminary threshold, EEG data and respiratory phase to obtain denoised EEG data, including the following steps: Taking the denoising effect evaluation index as the objective function, and according to the respiratory phase, optimizing the search for the wavelet basis and the number of decomposition layers of the adaptive wavelet denoising model; Perform wavelet transform at different resolutions on the locally corrected EEG data in the time dimension and space dimension respectively to obtain wavelet coefficients at different resolutions; In the time dimension and space dimension, respectively, according to the respiratory phase and EEG data characteristics, the wavelet coefficients at different resolutions are selectively fused to obtain the fusion coefficients in the time dimension and the fusion coefficients in the space dimension; The fusion coefficients in the time dimension and the fusion coefficients in the space dimension are fused to obtain the comprehensive wavelet coefficients; The integrated wavelet coefficients are subjected to inverse wavelet transform to recover the denoised EEG data.

5. The device according to claim 1, wherein The physiological characteristics include at least one of respiratory efficiency, rhythm frequency, amplitude spectrum, chest-abdomen phase difference, airflow asymmetry, muscle discharge period, symmetrical tension fluctuation and postural residual potential, nonlinear change of heart rate variability in each respiratory cycle, pressure accumulation index, θ / β wave purity index and respiratory rhythm disorder; The emotional characteristics include at least one of microcirculatory disorder characteristics, mental fluctuation state and cross-modal emotional perception index; The feature extraction module is used to: Calculate the fluctuation curve of the nasal and perioral temperature based on the respiratory data and body temperature data, and calculate the expiratory and inspiratory temperature difference ratio based on the fluctuation curve as the respiratory efficiency, and extract the rhythm frequency, amplitude spectrum, chest-abdomen phase difference, and airflow asymmetry from the respiratory data; Extract muscle discharge cycles, symmetrical tension fluctuations, and postural residual potentials based on electromyographic data, and identify microcirculatory disturbances during muscle tension based on body temperature and electromyographic data. Analyze the noise characteristics of EEG data at each reference electrode under different head postures, determine the optimal reference electrode based on the noise characteristics, calculate the theta / beta wave purity index, transient waveform density, and edge frequency based on the EEG data at the optimal reference electrode, and analyze the user's mental fluctuation state in combination with the anxiety estimation model; Based on the respiratory data and heart rate data, the nonlinear changes of heart rate variability in each respiratory cycle are calculated. Based on the frequency domain characteristics of heart rate variability and respiratory rhythm disorder, the pressure accumulation index is calculated. Based on the electromyography data and heart rate data, the time series of the user's emotional data at different times is obtained, the mutual information and phase synchronization rate between the time series of the emotional data and the time series of the respiratory data are calculated, and the cross-modal emotion perception index is calculated based on the mutual information and phase synchronization rate.

6. The device according to claim 1, wherein The physiological gene map construction module is used to: Through the heterogeneous graph attention network model, the weights between physiological features in the physiological feature subgraph and the weights between emotional features in the emotional feature subgraph are dynamically learned; Using a deep multi-channel structure, the physiological feature subgraph and the emotional feature subgraph are aligned and represented in a unified embedding space to obtain the aligned physiological feature subgraph and emotional feature subgraph. Based on the aligned physiological feature subgraph and emotional feature subgraph, the user's physiological gene map is constructed.

7. The device according to claim 1, wherein The training steps of the physiological state analysis model include: For each feature data in the physiological gene maps of multiple users, the physiological state is labeled according to the weight corresponding to the feature data to form a training set; Build a physiological state analysis model based on reinforcement learning, determine the network structure of the physiological state analysis model, regard the physiological state analysis model as an intelligent agent, and initialize the model parameters; The high-level task is to determine whether the user is in a stressful state. If the user is in a stressful state, the low-level task is to determine whether the user is nervous or tired. If the user is in a non-stressful state, the low-level task is to determine whether the user is in a state of shallow relaxation or deep relaxation. The following training steps are performed for high-level tasks and low-level tasks respectively: Input the training set into the physiological state analysis model, so that the agent takes actions to predict the physiological state based on the current model parameters and training set, and outputs the predicted results of the user's physiological state; The predicted results are compared with the user's actual physiological state, and the reward signal is calculated based on the comparison results; The agent uses a reinforcement learning algorithm to update its prediction strategy based on the reward signal obtained.

8. The device according to claim 1, wherein The breathing training program generation module performs the steps of adjusting the breathing training program based on the difference between the key feature data and the target key feature data in the current breathing training program and the training intensity, including: Obtaining control parameters, wherein the control parameters include a proportional coefficient, an integral coefficient, and a differential coefficient; Real-time monitoring of the user's multimodal data to obtain key feature data, and calculation of performance error based on the difference between the key feature data and the target key feature data in the current breathing training program. The key features include respiratory rhythm frequency, theta / beta wave purity index, symmetrical tension fluctuation, and respiratory rhythm dysregulation. Calculating the training intensity at the next moment based on the performance error, the control parameter, and the current training intensity; Determine safety constraints; Adjust the current breathing training plan based on the training intensity and safety constraints at the next moment.

9. The device according to claim 1, wherein The interaction module is further configured to: A natural language guide is generated according to the breathing training program, and the natural language guide is presented in the form of a virtual coach.

10. The device according to claim 1, wherein It also includes a communication module for realizing data transmission between the data processing server and the multimodal sensor and the interaction module through a variety of wireless connection methods.

Images