Health monitoring method and system based on time sequence Transform and data fusion

Abstract

The invention is suitable for the technical field of health monitoring, and provides a health monitoring method and system based on time sequence Transform and data fusion, and the method comprises the following steps: collecting multi-modal biosensor data; carrying out asynchronous time processing and normalization processing on the acquired multi-mode biosensor data; extracting features from the processed multi-modal biosensor data by using TimesNet, forming an independent input channel by the data of each modal, performing feature extraction on the data of each modal, extracting time sequence features of different scales and abstract levels, and performing feature extraction on the extracted time sequence features of different scales and abstract levels; the extracted features are input into a time sequence Transform network for feature fusion, time sequence prediction and anomaly monitoring; carrying out online platform display on the obtained prediction data, and giving an alarm when abnormity exists in the prediction data; through the time sequence Transform model, the health state of the worker can be monitored in real time, potential abnormal behaviors can be found and diagnosed in time, and the safety management level is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of health monitoring, and in particular to a health monitoring method and system based on time series Transformer and data fusion. Background Art

[0002] Site health monitoring methods have received increasing attention in recent years. The goal of this technology is to improve worker safety and work efficiency by monitoring the health of workers on the site in real time and preventing potential health risks. The importance of site health monitoring is reflected in many aspects, mainly including ensuring worker safety, improving production efficiency, reducing economic losses and promoting sustainable development. Ensuring worker safety and health is the primary goal of site health monitoring. The site environment is complex and changeable, with many potential sources of danger, such as high-altitude work, heavy machinery operation, dust and harmful gases. By monitoring the health of workers in real time, it is possible to detect and prevent work-related accidents in a timely manner and ensure the safety of workers. At the same time, long-term physical labor and poor working environment may cause health problems for workers, such as heat stroke, excessive fatigue, heart disease, etc. Through the health monitoring system, these health risks can be warned in time, and necessary measures can be taken to intervene and treat them.

[0003] Traditional data analysis methods, such as simple statistical analysis and traditional machine learning algorithms (such as support vector machines, decision trees, random forests, etc.), have limited performance when processing complex time series data and large-scale multimodal data. These methods rely heavily on data volume and feature engineering, have poor model generalization capabilities, and have difficulty maintaining stable performance in different environments and worker groups. In addition, these methods are usually unable to fully explore the deep patterns and correlations in the data, and have low prediction accuracy and reliability.

[0004] With the development of artificial intelligence, deep learning methods can automatically process and analyze large amounts of complex data in construction site health monitoring based on data. Deep learning can discover patterns and trends in monitoring data and make predictions and early warnings through algorithms. Time series transformers excel in processing long sequence data and capturing complex time dependencies, and can more efficiently analyze and predict workers' health status. Traditional methods such as recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) often face problems of gradient vanishing and low computational efficiency when processing long sequence data.

[0005] Therefore, it is necessary to provide a health monitoring method and system based on time series Transformer and data fusion to solve the above problems. Summary of the invention

[0006] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a health monitoring method and system based on time series Transformer and data fusion to solve the problems existing in the above-mentioned background technology.

[0007] The present invention is implemented as follows: a health monitoring method based on time series Transformer and data fusion, the method comprising the following steps:

[0008] Collect multimodal biosensor data;

[0009] Asynchronous time processing and normalization of collected multimodal biosensor data;

[0010] TimesNet is used to extract features from the processed multimodal biosensor data. The data of each modality constitutes a separate input channel. Feature extraction is performed on the data of each modality to extract temporal features of different scales and abstraction levels.

[0011] The extracted features are input into the time series Transformer network for feature fusion, time series prediction, and anomaly detection;

[0012] The obtained prediction data is displayed on the online platform, and an alarm is issued when there are anomalies in the prediction data.

[0013] As a further solution of the present invention: the multimodal biosensor data includes heart rate, body temperature, skin moisture, blood oxygen saturation, blood pressure and blood sugar.

[0014] As a further solution of the present invention: the step of performing asynchronous time processing on the collected multimodal biosensor data specifically includes:

[0015] Clean the collected multimodal biosensor data to remove noise and outliers;

[0016] Asynchronous time processing is performed on multimodal biosensor data, and interpolation operations are performed on data with a time step greater than a step threshold to ensure time synchronization of data between different modalities.

[0017] As a further solution of the present invention: the step of extracting features from the processed multimodal biosensor data using TimesNet specifically includes:

[0018] Perform time series conversion to convert the data of each modality into time series data;

[0019] For each modality of time series data, the Fourier transform method in TimesNet is used to decompose the one-dimensional signal into multiple groups of signals with different frequencies;

[0020] Calculate the average amplitude of each group of signals and select the topk parts of the amplitude; use a two-dimensional convolutional layer to extract the two-dimensional features of each mode.

[0021] As a further solution of the present invention: the step of inputting the extracted features into the time series Transformer network for feature fusion, time series prediction and anomaly monitoring specifically includes:

[0022] Align features from different modalities into one feature space;

[0023] Input the aligned multimodal features into the multi-head attention layer of the Transformer network;

[0024] Establish associations between different modal features based on the multi-head attention mechanism and extract cross-modal correlation information;

[0025] Through stacking and fusion of temporal Transformer layers, features of different modalities are gradually extracted and combined;

[0026] Perform prediction and anomaly detection alerts on multimodal biosensor data in downstream tasks.

[0027] As a further solution of the present invention: the time series data of each modality is completed and reshaped to adapt to the input requirements of TimesNet.

[0028] Another object of the present invention is to provide a health monitoring system based on time series Transformer and data fusion, the system comprising:

[0029] A biological data acquisition module for acquiring multimodal biosensor data;

[0030] A biological data processing module, used for asynchronous time processing and normalization processing of the collected multimodal biosensor data;

[0031] The multimodal feature extraction module is used to extract features from the processed multimodal biosensor data using TimesNet. The data of each modality constitutes a separate input channel, and feature extraction is performed on the data of each modality to extract temporal features of different scales and abstraction levels;

[0032] The feature fusion prediction module is used to input the extracted features into the time series Transformer network for feature fusion, time series prediction, and anomaly detection;

[0033] The prediction data display module is used to display the obtained prediction data on an online platform and issue an alarm when there are anomalies in the prediction data.

[0034] As a further solution of the present invention: the biological data processing module includes:

[0035] A sensor data cleaning unit, used to clean the collected multimodal biosensor data to remove noise and outliers;

[0036] The asynchronous time processing unit is used to perform asynchronous time processing on multimodal biosensor data and perform interpolation operations on data with a time step greater than a step threshold to ensure time synchronization of data between different modalities.

[0037] As a further solution of the present invention: the multi-mode feature extraction module includes:

[0038] A time series conversion unit is used to perform time series conversion and convert the data of each modality into time series data;

[0039] The signal decomposition unit is used to decompose the one-dimensional signal into multiple groups of signals with different frequencies using the Fourier transform method in TimesNet for the time series data of each modality;

[0040] The modal feature extraction unit is used to calculate the average amplitude of each group of signals and select the topk parts of the amplitude; a two-dimensional convolutional layer is used to extract the two-dimensional features of each modality.

[0041] As a further solution of the present invention: the feature fusion prediction module includes:

[0042] A modal feature alignment unit, used to align features of different modalities into a feature space;

[0043] The modal feature input unit is used to input the aligned multimodal features into the multi-head attention layer of the Transformer network;

[0044] The correlation information extraction unit is used to establish correlations between different modal features based on the multi-head attention mechanism and extract cross-modal correlation information;

[0045] The stacking fusion unit is used to stack and fuse through the temporal Transformer layer to gradually extract and combine features of different modalities;

[0046] The prediction and alarm unit is used to perform prediction and anomaly detection alarms on multimodal biosensor data in downstream tasks.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention adopts the time series Transformer technology, allowing the deep neural network in the system to be trained on the basis of multimodal sensor data, thereby optimizing the model's time series feature extraction performance and improving prediction accuracy. By integrating data sources of multiple modalities, such as heart rate, blood oxygen, body temperature, ambient temperature and humidity, the present invention can capture the comprehensive characteristics of workers' health status from multiple angles and align the features of these different modalities in the feature space. Specifically, it adopts advanced self-attention mechanisms and asynchronous time processing strategies to strengthen the correlation between different sensor data and extract more accurate health status features. The trained time series Transformer model can monitor the health status of workers in real time, detect and diagnose potential abnormal behaviors in a timely manner, thereby significantly improving the safety management level of the construction site and the health protection effect of workers. Ultimately, the implementation of the present invention can provide an efficient and accurate solution for the construction site health monitoring system to ensure the health and safety of workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The schematic diagram of the health monitoring method based on time series Transformer and data fusion.

[0050] Figure 2 Flowchart of the health monitoring method based on time series Transformer and data fusion.

[0051] Figure 3 Flowchart of asynchronous time processing in the health monitoring method based on time series Transformer and data fusion.

[0052] Figure 4 Flowchart for feature extraction in health monitoring method based on temporal Transformer and data fusion.

[0053] Figure 5 Flowchart for feature fusion in health monitoring method based on temporal Transformer and data fusion.

[0054] Figure 6 Schematic diagram of the structure of the health monitoring system based on time series Transformer and data fusion. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0057] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a health monitoring method based on time series Transformer and data fusion, and the method includes the following steps:

[0058] S100, collecting multimodal biosensor data;

[0059] S200, performing asynchronous time processing and normalization processing on the collected multimodal biosensor data;

[0060] S300, extracts features from the processed multimodal biosensor data using TimesNet. The data of each modality constitutes a separate input channel, and features are extracted from the data of each modality to extract temporal features of different scales and abstraction levels;

[0061] S400, input the extracted features into the time series Transformer network for feature fusion, time series prediction and anomaly detection;

[0062] S500, display the obtained prediction data on an online platform, and issue an alarm when an abnormality exists in the prediction data.

[0063] In the embodiment of the present invention, multimodal biosensor data is first collected, and the multimodal biosensor data includes heart rate, body temperature, skin moisture, blood oxygen saturation, blood pressure and blood sugar. Because the judgment basis of each indicator is different, the time step of collecting data of different modes is inconsistent, that is, there is a time difference between data of different modes. In order to be able to perform multimodal prediction later, it is necessary to perform asynchronous time processing and normalization processing on the collected multimodal biosensor data. Then, TimesNet is used to extract features from the processed multimodal biosensor data, and the data of each mode constitutes a separate input channel, such as heart rate data, blood oxygen data, body temperature data, etc. Feature extraction is performed on the data of each mode to extract time series features of different scales and abstract levels; then the extracted features are input into the time series Transformer network for feature fusion, time series prediction and anomaly monitoring. The fused data obtained by the attention mechanism is stacked and fused through multiple Transformer layers to gradually extract and combine features of different modes. Finally, a comprehensive feature representation is generated for downstream tasks, namely prediction and anomaly detection. The fully connected layer processes the output of the Transformer encoder to generate the final prediction result. The model is trained using the MSE loss function and the Adam optimizer, and predictions are made after training. The multimodal data of the biosensor is obtained during runtime, and the data is imported into the shixu Transformer model to obtain the output value, and the output value is used to determine whether there is an abnormality. In this step, the trained model has been deployed on the construction site health monitoring platform, and the collected biosensor data is transmitted to the database via wireless transmission. Then it is input into the network, and the predicted data of a stage is displayed on the platform. Once an abnormality is detected during the prediction period, the system will issue an alarm in time.

[0064] like Figure 3 As shown, as a preferred embodiment of the present invention, the step of performing asynchronous time processing on the collected multimodal biosensor data specifically includes:

[0065] S201, cleaning the collected multimodal biosensor data to remove noise and outliers;

[0066] S202, performing asynchronous time processing on the multimodal biosensor data, and performing an interpolation operation on the data with a time step greater than a step threshold, so as to ensure time synchronization of data between different modalities.

[0067] In the embodiment of the present invention, the collected multimodal biosensor data also needs to be cleaned to remove noise and outliers; then the multimodal biosensor data will be asynchronously time processed, and the data with a time step greater than the step threshold will be interpolated to ensure the time synchronization of data between different modalities. In this way, the processed data will provide preparation for the multimodal data fusion in the subsequent steps.

[0068] like Figure 4 As shown, as a preferred embodiment of the present invention, the step of extracting features from the processed multimodal biosensor data using TimesNet specifically includes:

[0069] S301, performing time series conversion to convert the data of each modality into time series data;

[0070] S302, for each modality of time series data, using the Fourier transform method in TimesNet, decomposing the one-dimensional signal into multiple groups of signals with different frequencies;

[0071] S303, calculate the average amplitude of each group of signals, select the topk parts of the amplitude; use a two-dimensional convolutional layer to extract the two-dimensional features of each mode.

[0072] In the embodiment of the present invention, when performing single-modal feature extraction, time series conversion is required to convert the data of each modality into time series data; then, for the time series data of each modality, the Fourier transform method in TimesNet is used to decompose the one-dimensional signal into multiple groups of signals with different frequencies; then the amplitude average of each group of signals is calculated, and the important parts (such as the topk parts before the amplitude) are selected. If necessary, the signal is completed and reshaped to meet the input requirements of TimesNet; finally, a two-dimensional convolutional layer (such as InceptionBlock) is used to extract the two-dimensional features of each modality.

[0073] like Figure 5 As shown, as a preferred embodiment of the present invention, the step of inputting the extracted features into the time series Transformer network for feature fusion, time series prediction and anomaly monitoring specifically includes:

[0074] S401, aligning features of different modalities into a feature space;

[0075] S402, inputting the aligned multimodal features into the multi-head attention layer of the Transformer network;

[0076] S403, establishing associations between different modal features based on a multi-head attention mechanism, and extracting cross-modal correlation information;

[0077] S404, stacking and fusing through the temporal Transformer layer, gradually extracting and combining features of different modalities;

[0078] S405, performing prediction and anomaly detection alarm of multimodal biosensor data in downstream tasks.

[0079] In an embodiment of the present invention, the aligned multimodal features need to be input into the multi-head attention layer of the Transformer network, and associations are established between different modal features based on the multi-head attention mechanism to extract cross-modal correlation information; specifically, the multi-head attention mechanism focuses on different parts of the input features through multiple attention heads, and each attention head can learn the relationship between different modalities.

[0080] like Figure 6 As shown, an embodiment of the present invention further provides a health monitoring system based on time series Transformer and data fusion, the system comprising:

[0081] A biological data acquisition module 100, for acquiring multimodal biosensor data;

[0082] The biological data processing module 200 is used to perform asynchronous time processing and normalization processing on the collected multimodal biosensor data;

[0083] A multi-modal feature extraction module 300 is used to extract features from the processed multi-modal biosensor data using TimesNet, where each modality of data constitutes a separate input channel, and feature extraction is performed on each modality of data to extract temporal features of different scales and abstraction levels;

[0084] The feature fusion prediction module 400 is used to input the extracted features into the time series Transformer network for feature fusion, time series prediction and anomaly monitoring;

[0085] The prediction data display module 500 is used to display the obtained prediction data on an online platform and issue an alarm when there is an anomaly in the prediction data.

[0086] As a preferred embodiment of the present invention, the biological data processing module 200 includes:

[0087] A sensor data cleaning unit, used to clean the collected multimodal biosensor data to remove noise and outliers;

[0088] The asynchronous time processing unit is used to perform asynchronous time processing on multimodal biosensor data and perform interpolation operations on data with a time step greater than a step threshold to ensure time synchronization of data between different modalities.

[0089] As a preferred embodiment of the present invention, the multi-mode feature extraction module 300 includes:

[0090] A time series conversion unit is used to perform time series conversion and convert the data of each modality into time series data;

[0091] The signal decomposition unit is used to decompose the one-dimensional signal into multiple groups of signals with different frequencies using the Fourier transform method in TimesNet for the time series data of each modality;

[0092] The modal feature extraction unit is used to calculate the average amplitude of each group of signals and select the topk parts of the amplitude; a two-dimensional convolutional layer is used to extract the two-dimensional features of each modality.

[0093] As a preferred embodiment of the present invention, the feature fusion prediction module 400 includes:

[0094] A modal feature alignment unit, used to align features of different modalities into a feature space;

[0095] The modal feature input unit is used to input the aligned multimodal features into the multi-head attention layer of the Transformer network;

[0096] The correlation information extraction unit is used to establish correlations between different modal features based on the multi-head attention mechanism and extract cross-modal correlation information;

[0097] The stacking fusion unit is used to stack and fuse through the temporal Transformer layer to gradually extract and combine features of different modalities;

[0098] The prediction and alarm unit is used to perform prediction and anomaly detection alarms on multimodal biosensor data in downstream tasks.

[0099] The above only describes in detail the preferred embodiments of the present invention, which is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0100] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0101] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0102] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A health monitoring method based on time series Transformer and data fusion, characterized in that: The method comprises the following steps: Collect multimodal biosensor data; Asynchronous time processing and normalization of collected multimodal biosensor data; TimesNet is used to extract features from the processed multimodal biosensor data. The data of each modality constitutes a separate input channel. Feature extraction is performed on the data of each modality to extract temporal features of different scales and abstraction levels. The extracted features are input into the time series Transformer network for feature fusion, time series prediction, and anomaly detection; The obtained prediction data is displayed on the online platform, and an alarm is issued when there are anomalies in the prediction data.

2. The health monitoring method based on time series Transformer and data fusion according to claim 1 is characterized in that: The multimodal biosensor data includes heart rate, body temperature, skin moisture, blood oxygen saturation, blood pressure and blood glucose.

3. The health monitoring method based on time series Transformer and data fusion according to claim 1 is characterized in that: The step of performing asynchronous time processing on the collected multimodal biosensor data specifically includes: Clean the collected multimodal biosensor data to remove noise and outliers; Asynchronous time processing is performed on multimodal biosensor data, and interpolation operations are performed on data with a time step greater than a step threshold to ensure time synchronization of data between different modalities.

4. The health monitoring method based on time series Transformer and data fusion according to claim 1, characterized in that: The step of extracting features from the processed multimodal biosensor data using TimesNet specifically includes: Perform time series conversion to convert the data of each modality into time series data; For each modality of time series data, the Fourier transform method in TimesNet is used to decompose the one-dimensional signal into multiple groups of signals with different frequencies; Calculate the average amplitude of each group of signals and select the topk parts of the amplitude; use a two-dimensional convolutional layer to extract the two-dimensional features of each mode.

5. The health monitoring method based on time series Transformer and data fusion according to claim 1, characterized in that: The step of inputting the extracted features into the time series Transformer network for feature fusion, time series prediction and anomaly monitoring specifically includes: Align features from different modalities into one feature space; Input the aligned multimodal features into the multi-head attention layer of the Transformer network; Establish associations between different modal features based on the multi-head attention mechanism and extract cross-modal correlation information; Through stacking and fusion of temporal Transformer layers, features of different modalities are gradually extracted and combined; Perform prediction and anomaly detection alerts on multimodal biosensor data in downstream tasks.

6. The health monitoring method based on time series Transformer and data fusion according to claim 4 is characterized in that: The time series data of each modality is completed and reshaped to meet the input requirements of TimesNet.

7. A health monitoring system based on time series transformer and data fusion, characterized by: The system comprises: A biological data acquisition module for acquiring multimodal biosensor data; A biological data processing module, used for asynchronous time processing and normalization processing of the collected multimodal biosensor data; The multimodal feature extraction module is used to extract features from the processed multimodal biosensor data using TimesNet. The data of each modality constitutes a separate input channel, and feature extraction is performed on the data of each modality to extract temporal features of different scales and abstraction levels; The feature fusion prediction module is used to input the extracted features into the time series Transformer network for feature fusion, time series prediction, and anomaly detection; The prediction data display module is used to display the obtained prediction data on an online platform and issue an alarm when there are anomalies in the prediction data.

8. The health monitoring system based on time series transformer and data fusion according to claim 7, characterized in that: The biological data processing module comprises: A sensor data cleaning unit, used to clean the collected multimodal biosensor data to remove noise and outliers; The asynchronous time processing unit is used to perform asynchronous time processing on multimodal biosensor data and perform interpolation operations on data with a time step greater than a step threshold to ensure time synchronization of data between different modalities.

9. The health monitoring system based on time series transformer and data fusion according to claim 7, characterized in that: The multi-mode feature extraction module comprises: A time series conversion unit is used to perform time series conversion and convert the data of each modality into time series data; The signal decomposition unit is used to decompose the one-dimensional signal into multiple groups of signals with different frequencies using the Fourier transform method in TimesNet for the time series data of each modality; The modal feature extraction unit is used to calculate the average amplitude of each group of signals and select the topk parts of the amplitude; a two-dimensional convolutional layer is used to extract the two-dimensional features of each modality.

10. The health monitoring system based on time series transformer and data fusion according to claim 7, characterized in that: The feature fusion prediction module includes: A modal feature alignment unit, used to align features of different modalities into a feature space; The modal feature input unit is used to input the aligned multimodal features into the multi-head attention layer of the Transformer network; The correlation information extraction unit is used to establish correlations between different modal features based on the multi-head attention mechanism and extract cross-modal correlation information; The stacking fusion unit is used to stack and fuse through the temporal Transformer layer to gradually extract and combine features of different modalities; The prediction and alarm unit is used to perform prediction and anomaly detection alarms on multimodal biosensor data in downstream tasks.

Images