Multi-disease time series prediction and early warning method and system

Abstract

The present invention discloses a multi-disease time series prediction and early warning method, including: collecting data and using it as the input of a model; using the Transformer architecture to build a model and train it; inputting the real-time data of the disease into the built model, outputting the predicted value and performing an early warning operation; wherein the collected data includes data of multiple diseases, and in the process of building the model, a cross-attention mechanism is added to calculate the attention scores of multiple diseases. The cross-attention mechanism introduced in the present invention can use the query information of a disease time series to match the key and value of another disease time series, thereby efficiently capturing and characterizing the potential relationship and interaction between two different disease time series, and can effectively integrate the data of multiple diseases, which not only enhances the model's ability to understand complex data structures, but also realizes cross-disease joint prediction, thereby improving the accuracy of disease early warning.

Description

Technical Field

[0001] The present invention relates to a time series prediction and early warning method, and in particular to a multi-disease time series prediction and early warning method and system based on a Transformer architecture. Background Art

[0002] Infectious diseases are a type of infectious disease caused by specific pathogens. They are characterized by rapid onset and can cause extensive physiological system damage in a short period of time, including but not limited to central nervous system dysfunction, cardiovascular system damage, lung pathological changes and liver dysfunction. These complications bring profound physical and mental pain to children and seriously affect their quality of life. Therefore, accurate and real-time monitoring and early warning of infectious diseases are of great practical significance to hospitals and even public health and epidemic prevention departments.

[0003] At present, infectious disease prediction generally relies on time series analysis models based on autocorrelation, or regression models built by combining weather data, Baidu Index and other external features. Disease time series prediction refers to the use of historical disease data and time series analysis methods to predict the trend and scale of future diseases. As long as this method is based on the statistical laws of historical data and establishes a mathematical model to infer future disease conditions, it can predict the trend and scale of future diseases, allocate medical resources and formulate prevention and control measures in advance, and reduce the impact of diseases on society. For example, based on the prediction results, it is possible to increase the investment in medical resources in advance, strengthen vaccination and publicity and education, and effectively control the spread of the disease.

[0004] Chinese patent document CN116230250B discloses a disease prediction method and system based on big data, which expands the disease according to the name of the disease to obtain a set of related terms corresponding to the disease; uses Spark's operator to filter the health and medical big data stored in HDFS according to the first subset of the related term set to obtain the data to be processed; calculates the weight of each piece of the data to be processed according to the related term set; sets a mapping table in the memory of the Driver end, and the mapping table records the Block and weight to which each piece of the data to be processed belongs; step 3, at the Driver end, according to the Block and weight to which each piece of the data to be processed belongs in the mapping table, obtains the score of each Block, establishes the corresponding relationship between Block and Task, assigns Task to Executor, and obtains disease prediction results with different confidence levels. The present invention effectively improves the efficiency of disease prediction using Spark, and the accuracy is also greatly improved.

[0005] In the prior art, the accuracy of disease prediction can be improved by expanding the relevant terms corresponding to the disease name. However, the prior art can only predict a single disease, which has limitations, and the accuracy of disease warning needs to be improved.

[0006] In addition, Chinese patent document CN114783608A discloses a method for constructing a disease risk prediction model for chronic disease populations based on a graph autoencoder. A patient-disease bipartite graph is constructed based on the patient's hospitalization records and historical medical information, and then feature vectors are extracted for the patient and the disease respectively; finally, a disease risk prediction model based on a graph attention mechanism is constructed based on the graph autoencoder architecture to predict the future disease risk of chronic disease patients. In the decoder part of the disease risk prediction model, the present invention uses an attention mechanism and simultaneously considers the weight information of the edges, so that the topological information of the bipartite graph and the individual differences of the patients can be considered at the same time, and the complex influence relationship between diseases can be learned, thereby achieving the purpose of improving the prediction effect.

[0007] In the decoder part of the disease risk prediction model in this prior art, the attention mechanism is used and the weight information of the edge is considered at the same time, so that the topological information of the bipartite graph and the individual differences of the patients can be considered at the same time, and the complex influence relationship between diseases can be learned, thereby achieving the purpose of improving the prediction effect. However, the disease risk prediction model constructed in this prior art can only predict the risk of other diseases of the corresponding patient. In other words, the disease risk prediction in this prior art is a warning made for individual patients, and cannot be used as a warning for the risk of disease of the general public. Summary of the invention

[0008] In order to solve the technical problem that the prediction accuracy of the disease prediction method in the prior art needs to be improved, the present invention provides a multi-disease time series prediction and early warning method, comprising the following steps: collecting data and using it as the input of a model; using the Transformer architecture for modeling and training to complete the construction of the model; inputting the real-time data of the disease into the constructed model, outputting the predicted value and performing an early warning operation; wherein: the collected data includes data of multiple diseases, and in the process of building the model, a cross-attention mechanism is added to calculate the attention scores of multiple diseases.

[0009] Since there are interactions and potential impacts between multiple diseases, such as influenza can aggravate the symptoms of whooping cough, using multiple interrelated disease data for prediction can help improve the accuracy of disease warnings. However, in the prior art, commonly used time series analysis methods, such as ARIMA, or machine learning / deep learning-based models, such as XGBoost, LSTM, etc., cannot fuse data from multiple diseases for early warning, that is, they cannot fully utilize the relevant information between diseases. Therefore, there are limitations, and the accuracy of disease early warning cannot be effectively improved. Moreover, since multi-disease data also requires an increase in the complexity of the data structure, the time series analysis methods commonly used in the prior art cannot effectively utilize multi-disease data.

[0010] In this solution, in the process of building the model using the Transformer architecture, the cross-attention mechanism introduced can use the query information of one disease time series to match the key and value of another disease time series, thereby efficiently capturing and characterizing the potential relationship and interaction between two different disease time series, and effectively integrating data from multiple diseases. It not only enhances the model's ability to understand complex data structures, but also realizes cross-disease joint prediction, thereby improving the accuracy of disease warning.

[0011] Preferably, for the cross-attention mechanism, a sampling strategy is introduced to optimize the efficiency of computing resource utilization. The sampling strategy is to only calculate the dot product operation between the keys set for a specific query during the calculation process:

[0012]

[0013]

[0014] Where K selected It means selecting the key with the highest attention weight, and ProbSparseAttention(Q,K,V) means the result of weighted summation of attention weights after sampling.

[0015] Considering that the use of multi-disease data will greatly increase the computational complexity and computing power requirements of the model, this solution introduces a sampling strategy that can optimize the utilization efficiency of computing resources. The method of only calculating the dot product operations between keys set for specific queries can significantly reduce the time complexity, thereby alleviating the high computational complexity and huge computing power requirements of the model brought by multi-disease data, and achieving the effect of optimizing the utilization efficiency of computing resources.

[0016] Preferably, before extracting local features, the time series data of each disease is segmented to obtain time series data blocks, and local features are extracted through a convolutional neural network or a multi-head self-attention mechanism. In this solution, the operation of segmenting the time series data of each disease can enhance the extraction of local features and can explore the correlation and importance between features. On the one hand, it makes the model more focused on capturing subtle changes and anomalies, and can also reduce computational complexity and memory usage, improve parallel processing efficiency, and achieve more accurate and stable predictions in time series prediction tasks.

[0017] Preferably, when constructing the model, external factors of the time series are also introduced at the encoding position for processing. In this solution, after the external factors are introduced for processing, compared with the traditional prediction method, the influence of external factors on the prediction effect is more comprehensively considered, thereby improving the accuracy of the prediction.

[0018] Preferably, when introducing external factors of the time series, the external factors are embedded and weighted summed, and then processed by a multi-head attention mechanism. The multi-head attention mechanism used is:

[0019]

[0020] head i =Attention(QW i Q ,KW i K ,VW i V )

[0021] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O ;

[0022] Where Q, K, and V represent Query, Key, and Value, respectively. k represents the dimension of K, and Attention represents the weighted sum of attention weights. This solution can smoothly introduce and process external factors of the time series, put external factors into the model as covariates, and more comprehensively consider the impact of external factors on the prediction effect, thereby improving the accuracy of the prediction.

[0023] Preferably, the encoder layer is retained in the overall architecture of the constructed Transformer model. This solution can greatly improve the training speed of the model.

[0024] Preferably, during the training process of the model, the average quantile loss function is used as the loss function.

[0025]

[0026] Where y represents the true value, L ql (y,y ql ) represents the quantile loss function of the smaller quantile, L qu (y,y qu ) represents the quantile loss function of the higher quantile, and the constructed model outputs the prediction interval. This scheme averages the losses generated by the two quantile critical values ​​to more comprehensively evaluate the prediction performance of the model.

[0027] Preferably, after the data is collected, the data is first denoised using a denoising method based on Fourier transform and wavelet transform:

[0028]

[0029] W[j,k]=∑ n f[n]ψ j,k [n];

[0030] Where F[k] represents the discrete Fourier transform, N is the length of the time series, i is the imaginary unit, k represents the frequency index, and f[n] represents the nth sample in the time domain. W[j,k] represents the discrete wavelet transform, j and k are the scale and position indexes respectively, ψ j,k [n] is the wavelet basis function.

[0031] Since the collected data is full of a lot of noise, and the existence of this noise greatly increases the difficulty of data analysis and model building, and also increases the difficulty of extracting effective information from the data and reduces the accuracy of prediction, it is necessary to denoise the collected data. The traditional moving average denoising method will cause the lag of the time series. Therefore, in this scheme, a denoising method based on Fourier transform and wavelet transform is adopted, which combines the advantages of Fourier transform in global noise processing and the expertise of wavelet transform in local noise processing. Through the organic combination of the two, the noise of time series data is effectively removed, thereby improving the accuracy of prediction.

[0032] Preferably, the collected data is normalized using the RevIN normalization method. Since the time series prediction model is highly sensitive to changes in the distribution of data input, the advantage of using the RevIN normalization method in this solution is that it can solve the time series distribution offset of data between different diseases during normalization, and retain important time series features in the data to avoid information loss. This method performs refined normalization operations at the input layer of the model for each independent instance in the data set, so that the model can more accurately capture and predict local dynamic changes in the sequence. In addition, the RevIN normalization method also receives consistently distributed inputs based on the mean and variance of each instance, further enhancing the stability and predictive power of the model.

[0033] In a second aspect, the present invention also provides a multi-disease time series prediction and early warning system that uses the multi-disease time series prediction and early warning method of any of the above-mentioned schemes for early warning.

[0034] The present invention has the following beneficial effects:

[0035] 1. The multi-disease time series prediction and early warning method based on the Transformer architecture in the present invention can predict and warn the number of cases of multiple diseases, which solves the limitation of only predicting a single disease in the prior art, and can improve the accuracy of model prediction by introducing external factors that affect the time series.

[0036] 2. In the present invention, in the process of building the model, the accuracy of model prediction can be further improved through methods such as RevIN normalization technology, multi-head attention mechanism, and cross attention mechanism.

[0037] 3. When faced with large-scale, complex multi-disease data, the present invention can significantly improve the speed and efficiency of model training and reasoning. DETAILED DESCRIPTION

[0038] The following is further described in detail through specific implementation methods:

[0039] 1. Definition

[0040] Computing power requirement: refers to the ability of computer equipment or data centers to process information. It is the ability of computer hardware and software to work together to perform a certain computing task.

[0041] 2. The embodiment is basically as follows: A multi-disease time series prediction and early warning method comprises the following steps:

[0042] 1. Data collection.

[0043] Collect case data of various diseases, including historical case numbers, meteorological data, etc., where the historical case data includes various case data. In this embodiment, the disease refers to childhood infectious diseases, and the pathogens of infectious diseases include 12 common childhood infectious diseases such as whooping cough, hand, foot and mouth disease, and influenza virus. In this embodiment, typical influenza and whooping cough case number data are taken as examples.

[0044] 2. Data preprocessing and normalization.

[0045] The collected data is preprocessed, mainly by filling missing values ​​and removing outliers to clean the data. For time points with missing data, that is, there are no observations at that time point, linear interpolation, spline interpolation or other appropriate interpolation methods are used to fill the missing values. After filling the missing, the ADF test (Augmented Dickey-Fuller test) is used to verify the stability of the time series data. If the test p value is not less than the significance level, the data is considered to be unstable. The present invention supports converting daily data into weekly, monthly or cumulative data to obtain more stable data. An outlier refers to a value in a set of data where some data points deviate significantly from the normal range compared to the average or distribution of the overall data.

[0046] Disease time series data is full of noise. In order to reduce the impact of noise on subsequent modeling, a denoising method based on Fourier transform and wavelet transform is used:

[0047]

[0048] W[j,k]=∑ n f[n]ψ j,k [n];

[0049] Where F[k] represents the discrete Fourier transform, N is the length of the time series, i is the imaginary unit, k represents the frequency index, and f[n] represents the nth sample in the time domain. W[j,k] represents the discrete wavelet transform, j and k are the scale and position indexes respectively, ψ j,k [n] is the wavelet basis function. This method combines the advantages of Fourier transform in global noise processing and the strengths of wavelet transform in local noise processing. Through the organic combination of the two, it realizes the effective removal of time series data noise.

[0050] Then the preprocessed data is normalized: the collected time series data of each disease is normalized to solve the time series distribution offset of data between different diseases. In this embodiment, the RevIN normalization method is adopted.

[0051]

[0052] Among them, xi represents the data of each input step, t represents the time, k represents the number of variables of the data, represents the normalized result, T x represents the length of the input sequence, γ k ,β k is a learnable parameter, ε is usually set to a small number (such as 10 -5 ) to prevent the variance from being zero.

[0053] 3. Patch operation and feature extraction of data.

[0054] First, the processed data is segmented into time series using the Patch operation: the time series data of each disease is segmented according to a fixed length L (such as 7 days, 14 days, etc.), and each segmented time series data block (Patch) contains L days of data. The specific operation is: suppose there is a daily case data sequence of a disease: [Day1, Day2, ..., DayT], and the Patch length is set to L days (such as 7 days). The data is divided into multiple subsequences every L days, that is, each Patch contains L days of data: [Day1, ..., DayL], [DayL+1, ..., Day2L], and so on. If T is not an integer multiple of L, the last Patch contains the remaining days.

[0055] Then extract local features for each patch: local features are extracted for each patch through a convolutional neural network (CNN) or a multi-head self-attention mechanism (Multi-Head Self-Attention).

[0056] Finally, the feature vectors extracted from each patch are concatenated into a large feature vector as the input of the Transformer model.

[0057] The concatenation of feature vectors refers to concatenating multiple features to form a new feature vector. The concatenation method can be a simple connection or a complex operation such as merging or superposition. Through feature concatenation, the information of different features can be integrated to provide more comprehensive and richer input data to the model.

[0058] 4. Transformer modeling.

[0059] The Transformer architecture is used to model the time series of multiple diseases. The input data is the feature vectors extracted from each patch, which are spliced ​​into a large feature vector. During the modeling process, the Transformer uses parallel computing and self-attention mechanisms.

[0060] When performing parallel computing on multi-disease data, a multi-GPU parallel computing architecture is used. The timestamp information, weather information, and holiday information of the sequence are embedded at the encoding position and weighted summed, and then processed through the multi-head attention mechanism. The principle of the multi-head attention mechanism is as follows:

[0061]

[0062] head i =Attention(QW i Q ,KW i K ,VW i V );

[0063] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O ;

[0064] Where Q, K, and V represent Query, Key, and Value, respectively. k represents the dimension of K, and Attention represents the weighted sum of attention weights. When calculating the attention scores of multiple disease data, a cross-attention mechanism is added to use the query information of the time series of one disease to match the key and value of the time series of another disease, thereby obtaining the attention scores of each disease.

[0065] In the above attention mechanism calculation process, a sampling strategy is introduced. In this embodiment, the introduced sampling strategy only calculates the dot product operations between the keys that are most important for a specific query:

[0066]

[0067] Where K selected It means selecting the key with the highest attention weight, and ProbSparseAttention(Q,K,V) means the result of weighted summation of attention weights after sampling.

[0068] At the same time, the overall architecture layout of Transformer is simplified, retaining only the encoder layer, while abandoning the traditional decoder layer and using a fully connected layer instead.

[0069] 5. Model training.

[0070] During the training phase of the model, the mean squared error (MSE) is used as the loss function to measure the difference between the model's predicted value and the actual value, and the model is adjusted based on the difference to minimize the difference and optimize the model's performance.

[0071] 6. Calculate the predicted value of the model.

[0072] The trained Transformer model is used to predict the number of cases in the future time period, and the model outputs the predicted value.

[0073] 7. Model warning.

[0074] When issuing warnings, there are warnings based on the model prediction interval and warnings based on preset thresholds. Among them, warnings based on preset thresholds refer to setting the predicted value output by the model as the warning threshold. When the actual number of cases exceeds the set warning threshold, a warning will be issued.

[0075] For warnings based on the expected interval of the model, it is necessary to first use the average quantile loss function as the loss function during the model training process, and then obtain the prediction interval through the trained model. Then, when the actual number of cases exceeds the upper limit of the prediction interval, the warning is automatically triggered. Quantiles are to divide the data into several equal parts after arranging them in order of size, and each part contains the same proportion of data.

[0076] 8. Real-time dynamic warning of the model.

[0077] The model outputs real-time data and continuously updates the model. The model outputs corresponding prediction values ​​based on the input real-time data to achieve dynamic early warning.

[0078] In the above process, in order to effectively reduce storage overhead and significantly improve data reading efficiency, data compression algorithms and optimized data storage strategies are used. These algorithms and strategies not only greatly reduce storage space usage by reducing data redundancy and optimizing storage layout, but also significantly speed up data reading, thus providing solid support for efficient management of large-scale data.

[0079] This embodiment also discloses a multi-disease time series prediction and early warning system that uses the multi-disease time series prediction and early warning method to provide early warning.

[0080] The specific implementation process is as follows: First, case data of various disease data are collected, and the number of cases of various infectious diseases per day is extracted from the hospital's underlying database. In this embodiment, taking influenza and whooping cough cases as examples, external factors such as historical cases, meteorological data, weather information, and holiday information are also included, and then missing value filling, outlier removal and other processing are performed. For missing time points, linear interpolation, spline interpolation or other appropriate interpolation methods can be used to fill the missing values ​​to ensure that the subsequent modeling will not be affected when the time points of different time series are not aligned. Then the ADF test (Augmented Dickey-Fuller test) is used to verify the stability of the time series data. If the test p value is not less than the significance level, the data is considered to be unstable. In this embodiment, it is possible to support the conversion of daily data into weekly, monthly or cumulative data to obtain more stable data.

[0081] Then, the collected data was denoised using a denoising method based on Fourier transform and wavelet transform. The experiment adopted a two-way parallel research design: first, two differentiated preprocessing schemes were implemented on the original data set, and then the processed data were input into two different model architectures for training and evaluation. In order to eliminate potential confounding variables, the variables were strictly controlled during the experiment to ensure that the hyperparameter settings, optimizer configurations, and other training parameters of the isomorphic model remained completely consistent. Then, the MSE (mean square error) and MAE (mean square error) were statistically analyzed, and the statistical results are shown in Table 1 below.

[0082] Table 1

[0083]

[0084] The experimental results show that in the comparison of verification errors of model training, the denoising effect of this method is significantly improved compared with the traditional moving average method, and the problem of time lag is alleviated.

[0085] Then, the RevIN normalization method is applied to the time series data of each disease to solve the time series distribution offset of data between different diseases. In this embodiment, typical diseases of "influenza" and "pertussis" are used as examples. That is, during data collection, the number of cases of influenza and pertussis is collected. As shown in Table 2.

[0086] Table 2

[0087] influenza 21 1187 926 265 153 46 36 pertussis 7 6 8 7 15 15 6

[0088] Then, the input data of each disease is normalized using the RevIN normalization method. In Table 2, the mean of influenza is 376.28, the variance is 228799.24, the mean of pertussis is 9.14, the variance is 16.47, and the initial γ k,β k are 1 and 0 respectively. The normalized data are shown in Table 3.

[0089] Table 3

[0090] influenza -0.742763 1.69489 1.14924 -0.2327 -0.4668 -0.6905 -0.7114 pertussis -0.527916 -0.7743 -0.2816 -0.5279 1.44297 1.44297 -0.7743

[0091] Then, the fixed length L of the time series data for each disease is set to 7 days, and the data is divided into multiple subsequences every 7 days. Each patch includes 7 days of data: [Day 1, ..., Day L], [Day L+1, ..., Day 2L], and so on. If T is not an integer multiple of L, the last patch contains the remaining days.

[0092] The step size of Patch is set to 2, and the data in Table 2 is divided into [[21,7], [1178,6]], [[926,8], [265,7]], [[153,14], [46,15]], and mapped to a high-dimensional space through the convolution kernel nn.Conv2D(), so that the shape of the data matrix changes from (batch_size, input_size, 2,2) to (batch_size, output_size, 2,2). Then the feature vectors extracted from each Patch are concatenated into a large feature vector and used as the input of the Transformer model. The Transformer architecture is used to model the time series of multiple diseases based on the input data.

[0093] The Transformer architecture is used for time series modeling of multiple diseases. The timestamp information, weather information, and holiday information of the sequence are embedded at the encoding position and weighted summed, and then processed through the multi-head attention mechanism.

[0094] According to the influenza and whooping cough data mentioned above, the size of the data after preprocessing and RevIN normalization is (batch_size=1, length=7, dim=2), Query, Key and Value represent a matrix, the matrix size is (batch_size=1, length=7, d_model=512), after multi-head processing, the data is divided into head1,..., head8, the data size is (batch_size=1, num_head=8, length=7, d_k=64). The attention formula Attention(Q,K,V) is used to calculate the attention of the data after multi-head processing, and finally the result is spliced, the result size is (batch_size=1, length=7, d_model=512).

[0095] When calculating attention, in order to effectively integrate data from different diseases, a cross-attention mechanism is used to efficiently capture and characterize the potential relationship and interaction between two different infectious disease time series by using the query information of one sequence to match the key and value of another sequence. Assuming Q1, K1, V1 are the query, key and value of influenza, and Q2, K2, V2 are the query, key and value of pertussis, then the attention score of influenza is The attention score for pertussis is

[0096] In the above process, since multiple disease case data are introduced for calculation, the calculation of the model is complex and the computing power is huge. In this embodiment, a sampling strategy is introduced for the attention mechanism so that only the dot product operations between the most important keys for a specific query are calculated.

[0097]

[0098] Where K selected Indicates the selection of the key with the highest attention weight, and ProbSparseAttention(Q,K,V) indicates the result of weighted summation of the attention weights after sampling. This significantly reduces the time complexity, thereby optimizing the utilization efficiency of computing resources. Compared with the calculation without introducing the sampling strategy, the results are shown in Table 4.

[0099] Table 4

[0100] SelfAttention ProbSparseAttention Time Complexity <![CDATA[O(N 2 )]]> O(N*logN)

[0101] In Table 4, N is the length of the input sequence. It can be seen that after the sampling strategy is introduced, the time complexity is significantly reduced, thereby optimizing the utilization efficiency of computing resources.

[0102] In addition, the overall architectural layout of Transformer has been simplified, retaining only the encoder layer and abandoning the traditional decoder layer. Instead, a fully connected layer is used as a replacement. This not only greatly improves the training speed of the model, but also effectively reduces the demand for computing power, demonstrating in-depth exploration and significant results in model optimization.

[0103] The model is then trained, and the number of cases in the future time period is predicted using the trained Transformer model, and the model outputs the predicted value.

[0104] In this embodiment, different types of early warning can be performed according to the different loss functions used in the model training process. In the model training stage, when the mean square error (MSE) is used as the loss function, the model outputs a predicted value, such as 95%, and serves as a preset early warning threshold. When the actual number of cases exceeds the set early warning threshold, an early warning is triggered to prompt public health agencies to pay attention to abnormal fluctuations.

[0105] During the model training phase, when the loss function uses the average quantile loss function, the model output is the prediction interval. The average quantile loss function used is as follows:

[0106]

[0107] Where y represents the true value, L ql (y,y ql ) represents the quantile loss function of the smaller quantile, L qu (y,y qu ) represents the quantile loss function of the higher quantile. The lower quantile represents the proportion of smaller values, while the higher quantile represents the proportion of larger values. The model outputs a prediction interval, such as the 95% quantile interval. When the actual number of cases exceeds the upper limit of the prediction interval, an early warning is automatically triggered. During the experiment, the principle of controlled experiment was strictly followed, and all isomorphic models were treated with completely consistent hyperparameter configurations, optimizer settings, and other training parameters. The experimental results selected mean square error (MSE) and mean absolute error (MAE) as evaluation indicators, and the statistical results are shown in Table 5.

[0108] Table 5

[0109]

[0110] Note: An isomorphic model means that the structures of different models remain unchanged and two models are made structurally equivalent through mapping relationships.

[0111] Our model: refers to the model built in this application based on the Transforme architecture, which is different from the existing Transforme model.

[0112] Dlinear: A simple model designed for time series tasks.

[0113] LSTM: A special type of recurrent neural network;

[0114] ARIMA: A classic time series forecasting model, the full name is "Autoregressive Difference Moving Average Model";

[0115] Transformer: An attention-based sequence model.

[0116] It can be concluded from Table 5 that in the experiment of the isomorphic model, the mean square error (MSE) and mean absolute error (MAE) of the model of the present invention are the smallest, so the prediction result can be more accurate.

[0117] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. The multi-disease time series prediction and early warning method includes the following steps: Collect data and use it as input for the model; Modeling and training are performed to complete the construction of the model; real-time data of the disease is input into the constructed model, the predicted value is output and an early warning operation is performed; the characteristics are: the collected data includes data on multiple diseases, and in the process of building the model using the Transformer architecture, a cross-attention mechanism is added to calculate the attention scores of multiple diseases.

2. The multi-disease time series prediction and early warning method according to claim 1, characterized in that: For the cross-attention mechanism, a sampling strategy is introduced to optimize the efficiency of computing resource utilization. The sampling strategy is to only calculate the dot product operation between the keys set for a specific query during the calculation process: Where K selected It means selecting the key with the highest attention weight, and ProbSparseAttention(Q,K,V) means the result of weighted summation of attention weights after sampling.

3. The multi-disease time series prediction and early warning method according to claim 2 is characterized by: Before extracting local features, the time series data of each disease is segmented to obtain time series data blocks, and local features are extracted through convolutional neural networks or multi-head self-attention mechanisms.

4. The multi-disease time series prediction and early warning method according to any one of claims 1 to 3, characterized in that: When constructing the model, external factors of the time series are also introduced at the encoding position for processing.

5. The multi-disease time series prediction and early warning method according to claim 4, characterized in that: When introducing external factors of the time series, the external factors are embedded and weighted summed, and then processed by the multi-head attention mechanism. The multi-head attention mechanism used is: MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O ; Where Q, K, and V represent Query, Key, and Value, respectively. k represents the dimension of K, and Attention represents the weighted summation of attention weights.

6. The multi-disease time series prediction and early warning method according to claim 5, characterized in that: In the overall architecture of the constructed Transformer model, the encoder layer is retained.

7. The multi-disease time series prediction and early warning method according to claim 6, characterized in that: In the process of model training, the average quantile loss function is used as the loss function. Where y represents the true value, L ql (y,y ql ) represents the quantile loss function of the smaller quantile, L qu (y,y qu ) represents the quantile loss function of the higher quantile, and the constructed model outputs the prediction interval.

8. The multi-disease time series prediction and early warning method according to claim 7, characterized in that: After the data is collected, the data is first denoised using a denoising method based on Fourier transform and wavelet transform: W[j,k]=∑ n f[n]ψ j,k [n]: Where F[k] represents the discrete Fourier transform, N is the length of the time series, i is the imaginary unit, k represents the frequency index, and f[n] represents the nth sample in the time domain. W[j,k] represents the discrete wavelet transform, j and k are the scale and position indexes respectively, ψ j,k [n] is the wavelet basis function.

9. The multi-disease time series prediction and early warning method according to claim 8, characterized in that: The collected data were normalized using the RevIN normalization method.

10. A multi-disease time series prediction and early warning system that uses the multi-disease time series prediction and early warning method as described in any one of claims 1 to 9 to provide early warning.

Images