Time sequence prediction method and device, electronic equipment and storage medium

Abstract

The invention provides a time sequence prediction method and apparatus, an electronic device, a storage medium and a product. The method comprises the steps of obtaining historical event text data and historical time sequence data of a to-be-predicted event; performing event semantic coding on the historical event text data to obtain event semantic features, and performing time sequence feature extraction on the historical time sequence data to obtain time sequence features; performing cross-modal feature fusion on event semantic features and the time sequence features to obtain fusion features; and performing prediction based on the fusion features to obtain a prediction result. By adopting the technical scheme of the invention, the event text information is introduced into the time sequence prediction, and the event text information is fused with the time sequence data, so that when the event influencing the prediction content occurs, the event text information is combined to make a corresponding change in the prediction process, and the accuracy of the time sequence prediction is further improved.

Description

Technical Field

[0001] This application relates to the field of timing prediction technology, and in particular to a timing prediction method, apparatus, electronic device and storage medium. Background Technology

[0002] Time series forecasting, as an important branch of data analysis and machine learning, aims to predict future values ​​using historical time series data and is widely used in fields such as finance, meteorology, energy, transportation, and healthcare.

[0003] Existing time series forecasting typically relies on structured numerical sequences for prediction. However, forecasting based solely on time series data cannot make corresponding changes during the forecasting process when events that affect the forecast occur, thus affecting the accuracy of time series forecasting. Summary of the Invention

[0004] Based on the above requirements, this application proposes a timing prediction method, apparatus, electronic device, and storage medium, which can improve the accuracy of timing prediction.

[0005] To achieve the above objectives, this application proposes the following technical solution: According to a first aspect of the embodiments of this application, a time series prediction method is provided, comprising: Obtain historical event text data and historical time series data of the event to be predicted; The historical event text data is semantically encoded to obtain event semantic features, and the historical time series data is extracted to obtain time series features. The event semantic features and the temporal features are fused across modalities to obtain fused features; Based on the fusion features, a prediction result is obtained.

[0006] Optionally, the historical event text data is subjected to event semantic encoding to obtain event semantic features, including: The historical event text data is normalized into standard historical event text according to a preset standard format; The standard text of the historical events is segmented into words to obtain a segmentation sequence; The event semantic features are obtained by using a pre-trained large language model to encode the word segmentation sequence.

[0007] Optionally, time-series features are extracted from the historical time-series data to obtain time-series features, including: The historical time series data is divided to obtain a sequence of segments consisting of at least one segment; Based on the self-attention mechanism, the segment sequence is encoded to obtain temporal features.

[0008] Optionally, after dividing the historical time-series data to obtain a sequence of segments consisting of at least one segment, the method further includes: Each segment in the segment sequence is mapped from a first dimension to a second dimension to obtain a segment sequence composed of segments in the second dimension; wherein, the segments in the second dimension contain more features than the segments in the first dimension. The encoding of the segment sequence based on the self-attention mechanism to obtain temporal features includes: Based on the self-attention mechanism, the sequence of segments composed of segments of the second dimension is encoded to obtain the temporal features.

[0009] Optionally, after dividing the historical time-series data to obtain a sequence of segments consisting of at least one segment, the method further includes: According to the order of each segment in the segment sequence, a position code is added to each segment in the segment sequence; wherein the position code represents the order of the segments in the segment sequence.

[0010] Optionally, the historical time-series data is divided to obtain a sequence of segments consisting of at least one segment, including: The historical time series data is divided into segments according to a pre-set segment size and overlap amount to obtain a segment sequence; wherein, the overlap amount is the amount of data overlapping between adjacent segments in the segment sequence.

[0011] Optionally, the event semantic features and the temporal features are fused across modalities to obtain fused features, including: The event semantic features and the temporal features are dimensionally aligned, and the dimensionally aligned event semantic features and temporal features are concatenated to obtain a fused sequence; The fusion sequence is encoded based on a self-attention mechanism to obtain fusion features.

[0012] Optionally, prediction is performed based on the fused features to obtain prediction results, including: The fused features are input into a pre-trained prediction head to obtain the prediction result output by the prediction head; wherein the prediction head includes at least one prediction head, and if it includes multiple prediction heads, the prediction scale of each prediction head is different.

[0013] According to a second aspect of the embodiments of this application, a timing prediction apparatus is provided, comprising: The data acquisition module is used to acquire historical event text data and historical time series data of the event to be predicted; The feature determination module is used to perform event semantic encoding on the historical event text data to obtain event semantic features, and to extract time-series features from the historical time-series data to obtain time-series features. The feature fusion module is used to perform cross-modal feature fusion of the event semantic features and the temporal features to obtain fused features; The prediction module is used to make predictions based on the fused features and obtain prediction results.

[0014] According to a third aspect of the embodiments of this application, an electronic device is provided, including: a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the above-described timing prediction method by running the program in the memory.

[0015] According to a fourth aspect of the embodiments of this application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the above-described timing prediction method.

[0016] The time-series prediction method proposed in this application acquires historical event text data and historical time-series data of the event to be predicted; performs event semantic encoding on the historical event text data to obtain event semantic features, and extracts time-series features from the historical time-series data to obtain time-series features; fuses the event semantic features and the time-series features across modalities to obtain fused features; and performs prediction based on the fused features to obtain the prediction result. By adopting the technical solution of this application, event text information is introduced into time-series prediction, and the event text information is fused with time-series data. This allows for adjustments to the prediction process when events affecting the predicted content occur, based on the event text information, thereby improving the accuracy of time-series prediction. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a time series prediction method provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the event semantic encoding processing flow provided in the embodiments of this application.

[0020] Figure 3This is a schematic diagram of a time-series feature extraction process provided in an embodiment of this application.

[0021] Figure 4 This is a schematic diagram of another time-series feature extraction process provided in an embodiment of this application.

[0022] Figure 5 This is a schematic diagram of another time-series feature extraction process provided in an embodiment of this application.

[0023] Figure 6 This is a schematic diagram of another time-series feature extraction process provided in an embodiment of this application.

[0024] Figure 7 This is a schematic diagram of the structure of a timing prediction device provided in an embodiment of this application.

[0025] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of this application are applicable to time series prediction in the field of artificial intelligence technology. Using the technical solutions of this application can improve the accuracy of time series prediction.

[0027] Time series forecasting, an important branch of data analysis and machine learning, aims to predict future values ​​using historical time series data and is widely used in fields such as finance, meteorology, energy, transportation, and healthcare. Existing time series forecasting typically utilizes time series forecasting models, which are machine learning or statistical models specifically designed to analyze time series data and predict future trends. Its core objective is to predict values ​​or states at a future point in time or over a specific period by identifying time-dependent patterns in historical data. Therefore, it can be seen that existing time series forecasting usually relies on structured numerical sequences for prediction. However, relying solely on time series data means that when events affecting the predicted content occur, corresponding changes cannot be made during the forecasting process, thus impacting the accuracy of the time series forecast.

[0028] Based on this, this application proposes a time series prediction method. This technical solution can introduce event text information into time series prediction, integrate event text information with time series data, and make corresponding changes during the prediction process when an event that affects the prediction content occurs, in combination with event text information, thereby solving the problem of low accuracy of time series prediction in the prior art.

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] Exemplary methods See Figure 1 As shown in the figure, this application proposes a time series prediction method. The method includes: S101. Obtain historical event text data and historical time series data of the event to be predicted.

[0031] In this embodiment, when predicting the future value of an event, the event is taken as the event to be predicted. Then, historical event text data and historical time-series data of the event to be predicted are acquired. The historical data extraction duration can be preset, and the current time is used as the final time for historical data extraction. The start time for historical data extraction is determined according to the historical data extraction duration. Finally, based on the start and end times, the historical event text data and historical time-series data between the start and end times are extracted. The historical event text includes text data of sudden events related to the event to be predicted that occurred between the start and end times, and the historical time-series data includes time-series data related to the event to be predicted between the start and end times.

[0032] In traffic volume prediction scenarios, the event to be predicted is traffic volume, while historical event text data consists of text data of traffic-related emergencies occurring between the start and end times. For example, a network failure will affect traffic volume, and the corresponding event text could be "Date: 2023-08-02 14:30. Event Type: Network Failure. Level: Level 1. Affected Scope: Core Network. Estimated Duration: 2 hours." Historical time series data represents the traffic volume for each time period between the start and end times. A time period can be 1 hour, 1 day, or 1 month, etc. The duration of a time period can be set according to the prediction scale. For example, if the prediction scale is 1 hour, then a time period can be set to 1 hour; if the prediction scale is 1 day, then a time period can be set to 1 day; if the prediction scale is 1 month, then a time period can be set to 1 month.

[0033] In the context of weather forecasting, the event to be predicted is weather, while historical event text data consists of text data of weather-related emergencies that occurred between the start and end times. Historical time-series data, on the other hand, consists of weather data between the start and end times.

[0034] In the field of stock price prediction, the event to be predicted is the stock price, while historical event text data consists of text data of sudden events related to the stock price that occurred between the start and end times. Historical time-series data consists of the stock price at each moment between the start and end times.

[0035] This application can also be applied to other fields that require time series prediction, which will not be listed in this embodiment.

[0036] S102. Perform event semantic encoding on historical event text data to obtain event semantic features, and extract time series features from historical time series data to obtain time series features.

[0037] After acquiring historical event text data, this embodiment needs to perform event semantic encoding on the historical event text data to obtain encoded event semantic features. This embodiment can employ a structured knowledge-based approach, using syntactic analysis tools (such as Stanford Parser) to extract the dependency structure of the event text, transforming this structural information into features to highlight the logical relationships of the events, thereby encoding the event semantic features. Alternatively, traditional machine learning methods can be used, such as combining the Bag-of-Words (BoW) model with statistical features to split the event text into individual words, using statistical word frequency, TF-IDF, and other information as features. A pre-trained language model (PLM) approach can also be used, employing general models such as BERT, RoBERTa, and GPT, fine-tuned on specific event datasets to achieve event semantic encoding using the fine-tuned model.

[0038] After acquiring historical time-series data, this embodiment requires extracting time-series features. This can be achieved using time-domain analysis methods to directly extract periodicity and trend features along the time dimension, or using frequency-domain analysis methods such as Fourier transform to convert the time-domain data to the frequency domain and extract periodic features, or using wavelet transform for multi-scale analysis to simultaneously capture both time-domain and frequency-domain features. Alternatively, deep learning methods, such as recurrent neural networks, convolutional neural networks, autoencoders, or Transformer networks, can be employed for time-series feature extraction.

[0039] S103. Perform cross-modal feature fusion of event semantic features and temporal features to obtain fused features.

[0040] In this embodiment, after extracting the event semantic features and temporal features, it is necessary to perform cross-modal fusion of the event semantic features and temporal features to obtain fused features. Specifically, cross-modal fusion requires first concatenating the event semantic features and temporal features, and then using an attention mechanism to perform contextual fusion on the concatenated features, thereby obtaining the fused features after cross-modal fusion.

[0041] Specifically, this step includes: First, the event semantic features and temporal features are dimensionally aligned, and then the dimensionally aligned event semantic features and temporal features are concatenated to obtain a fused sequence.

[0042] In order to concatenate the event semantic features and the temporal features in this embodiment, it is first necessary to align the event semantic features and the temporal features in terms of dimensions, and then concatenate the dimensionally aligned event semantic features and temporal features to obtain the fused sequence.

[0043] For example, the dimension is Temporal characteristics and dimension are event semantic features Projected onto the same dimension Where N represents the number of segment features contained in the time series feature, i.e., the sequence length of the time series feature. This represents the number of feature elements contained in the event semantic features, i.e., the sequence length of the event semantic features. Aligning the dimensions of the event semantic features and the temporal features involves projecting both dimensions onto the same dimension. The projection formula is: in, This represents the temporal features after dimension alignment. This represents the semantic features of an event after dimension alignment.

[0044] Event semantic features after dimension alignment and time series characteristics The sequences are spliced ​​together to obtain the fused sequence. , .

[0045] Second, based on the self-attention mechanism, the fusion sequence is encoded to obtain fusion features.

[0046] In this embodiment, the event semantic features and temporal features after dimension alignment are concatenated, and the concatenated fused sequence is input into the cross-attention layer. Based on the self-attention mechanism, the query vector Q, key vector K, and value vector V are calculated. Based on the query vector Q, key vector K, and value vector V, the attention output is determined. The attention output is then input into the residual connection and layer normalization, as well as the feedforward network, to obtain the final fused features.

[0047] Specifically, the fusion sequence The input is fed into a cross-attention layer, where the query vector Q, key vector K, and value vector V are computed: in, Attention weight calculation : in, This represents the dimension of the key vector.

[0048] Attention output is : Then, residual connections and layer normalization are performed on the attention output to obtain normalized data. : Then, the normalized data is input into the feedforward network to obtain the output data. : in, , , is the dimension of the hidden layer of the feedforward network.

[0049] Finally, the concatenated data of the output data and the normalized data is normalized to obtain the final output fusion feature. : This embodiment employs a self-attention mechanism to fuse temporal features and event semantic features across modalities. Before fusion, the temporal features and event semantic features are dimensionally aligned, enabling deep interaction and dynamic alignment between semantic information and temporal data. Compared to existing cross-modal fusion methods that use splicing or weighted splicing, the fused features in this embodiment have higher accuracy, resulting in higher prediction accuracy when using these fused features for prediction.

[0050] S104. Make predictions based on fusion features to obtain prediction results.

[0051] This embodiment fuses event semantic features and temporal features across modalities to obtain fused features. These fused features are then used for prediction to obtain prediction results. This allows for the integration of textual descriptions of sudden events into time-series data, enabling more accurate and interpretable predictions.

[0052] This embodiment performs prediction based on fused features, and a prediction head is pre-trained, which includes a fully connected layer. Training the prediction head first requires collecting sample event text data, sample time-series data, and future true results. Then, following the execution method described in the previous embodiment, the sample event text data is semantically encoded to obtain sample event semantic features. Time-series features are extracted from the sample time-series data to obtain sample time-series features. The sample event semantic features and sample time-series features are then fused across modalities to obtain sample fused features. These fused features are input into the prediction head to obtain sample prediction results. Finally, the loss function between the sample prediction results and the future true results is calculated. The parameters of the prediction head are adjusted to minimize this loss function, thus training the prediction head. Preferably, the loss function is calculated using the mean squared error method.

[0053] In this embodiment, the pre-trained prediction head includes at least one, and when multiple prediction heads are included, each prediction head has a different prediction scale. For example, when three prediction heads are set, the prediction scales of the three prediction heads are time-scale, daily-scale, and monthly-scale, respectively; that is, the three prediction heads are a time-scale prediction head, a daily-scale prediction head, and a monthly-scale prediction head. Features will be fused. Three independent prediction heads are fed in simultaneously: The time-scale prediction head outputs hourly forecasts for the next 24 hours. : in, , .

[0054] The daily-scale forecast head outputs the predicted values ​​for each of the next 7 days. : in, , .

[0055] The monthly forecast head outputs the forecast values ​​for each month of the next 12 months. : in, , .

[0056] The loss function for each predictor is preferably calculated using the mean squared error (MSE) method. The total loss is the weighted sum of the losses from all predictors, as shown in the following formula: in, The loss function represents the time-scale prediction head. This represents the prediction result for the i-th hour output by the size prediction head. This represents the actual result for the i-th hour.

[0057] in, The loss function representing the daily-scale prediction head. This represents the prediction result for the i-th day output by the daily size prediction head. This represents the actual result on day i.

[0058] in, The loss function representing the monthly-scale prediction head. This represents the prediction result for the i-th month output by the monthly size prediction header. This represents the actual result for the i-th month.

[0059] in, This represents the total loss function of the prediction head. The weights of the loss function for the time-scale prediction head. The weights of the loss function for the day-scale prediction head are... The weights of the loss function for the monthly-scale prediction head.

[0060] The unified multi-scale prediction architecture adopted in this embodiment overcomes the limitation of existing technologies that require separate modeling for different time granularities. A single model simultaneously outputs predictions at the hour, day, and month scales, which not only reduces model maintenance costs but also ensures the inherent consistency of multi-scale prediction results and avoids prediction conflicts that may arise from multiple independent models in traditional methods.

[0061] Furthermore, in this embodiment, when acquiring historical time-series data of the event to be predicted, the granularity of time-series data collection needs to be determined based on the scale of the prediction head. If only one prediction head is set, the scale of that prediction head is directly used as the granularity of time-series data collection. For example, in the field of traffic volume prediction, if only one time-scale prediction head is set, then the hourly traffic volume is collected; if only one daily-scale prediction head is set, then the daily traffic volume is collected; and if only one monthly-scale prediction head is set, then the monthly traffic volume is collected. If multiple prediction heads are set, then the scale of the prediction head with the smallest scale is directly used as the granularity of time-series data collection. For example, in the field of traffic volume prediction, if daily-scale and monthly-scale prediction heads are set, then the daily traffic volume is collected; if time-scale, daily-scale, and monthly-scale prediction heads are set, then the hourly traffic volume is collected.

[0062] As described above, the time-series prediction method proposed in this application acquires historical event text data and historical time-series data of the event to be predicted; performs event semantic encoding on the historical event text data to obtain event semantic features, and extracts time-series features from the historical time-series data to obtain time-series features; fuses the event semantic features and the time-series features across modalities to obtain fused features; and performs prediction based on the fused features to obtain the prediction result. By adopting the technical solution of this embodiment, event text information is introduced into time-series prediction, and the event text information is fused with time-series data. This allows for adjustments to the prediction process when events affecting the predicted content occur, based on the event text information, thereby improving the accuracy of time-series prediction.

[0063] As an optional implementation, see [link to implementation details]. Figure 2 As shown, another embodiment of this application discloses the method of performing event semantic encoding on historical event text data to obtain event semantic features, specifically including the following steps: S201. According to the preset standard format, the historical event text data is normalized into the historical event standard text.

[0064] This embodiment pre-sets a standard template for event description text, using it as a preset standard format. The acquired historical event text data of the event to be predicted is then normalized into this preset standard format to obtain the historical event standard text. For example, the historical event standard text could be: "Date: 2023-08-02 14:30. Event Type: Network Failure. Level: Level 1. Affected Scope: Core Network. Estimated Duration: 2 hours." S202. Perform word segmentation on the standard text of historical events to obtain a word segmentation sequence.

[0065] This embodiment requires segmenting the standard text of historical events into at least one word, and then assembling all the words into a segmentation sequence. This embodiment preferably employs tokenization, which breaks down the standard text of historical events into discrete units (i.e., tokens). Tokenization is a core preprocessing step in Natural Language Processing (NLP), and it is the process of dividing text (sentences, paragraphs, or documents) into smaller, meaningful units (such as words, sub-words, characters, or symbols). These units are called "tokens".

[0066] Suppose we input the standard text of historical events into tokenization, and after tokenization, we get a word segmentation sequence, i.e., a token sequence. ,in, The sequence length is given.

[0067] S203. Use a pre-trained large language model to encode the event semantics of the word segmentation sequence to obtain event semantic features.

[0068] In this embodiment, the word segmentation sequence is input into a pre-trained large language model, which is capable of encoding event semantics to obtain event semantic features. These event semantic features are the hidden state sequences output by the large language model. ,in, This represents the dimension of the last hidden layer in the large language model.

[0069] Large language models, through self-attention mechanisms and multi-layer neural networks, can capture long-distance dependencies in text. By training on trillions of text data, they learn statistical patterns and semantic relationships between words, phrases, and sentences. Large language models can infer semantics based on context. Therefore, this embodiment uses large language models for event semantic encoding, which can improve the semantic depth understanding of historical event texts and thus improve the final prediction accuracy.

[0070] As an optional implementation, see [link to implementation details]. Figure 3 As shown, another embodiment of this application discloses the extraction of time-series features from historical time-series data to obtain time-series features, specifically including the following steps: S301. Divide the historical time series data to obtain a segment sequence consisting of at least one segment.

[0071] In this embodiment, after obtaining the historical time-series data of the event to be predicted, the historical time-series data needs to be divided to obtain at least one segment. All the segments are then arranged sequentially to form a segment sequence. In this embodiment, when dividing the data into segments, it is preferable to divide the historical time-series data into multiple overlapping segments, thereby improving the correlation between the segments.

[0072] Specifically, in this embodiment, the segment size and overlap amount are preset. Historical time-series data are divided according to the preset segment size and overlap amount to obtain a segment sequence. The overlap amount is the amount of data overlapping between adjacent segments in the segment sequence. For example, in this embodiment, if the segment size is set to P and the step size is set to S, then the overlap amount is PS.

[0073] For example, for historical time series data of length L Let the fragment size be P, the step size be S, and the overlap be PS. Then the formula for calculating the number of fragments N is: When P=3, S=1, and the overlap is 2, the first segment The second segment The third segment ..., the (N-1)th segment .

[0074] The sequence of segments formed by all the segments is then: .

[0075] S302. Based on the self-attention mechanism, the segment sequence is encoded to obtain temporal features.

[0076] After determining the segment sequence corresponding to the historical time series data, this embodiment needs to encode the segment sequence based on a self-attention mechanism to achieve context fusion and obtain the time series features corresponding to the historical time series data. Preferably, this embodiment uses PatchTST as the backbone network for time series feature extraction and a Transformer encoder to encode the segment sequence. The Transformer encoder contains at least one Transformer layer, and each Transformer layer needs to encode the input data based on the self-attention mechanism. Encoding based on the self-attention mechanism first requires calculating the query vector Q, key vector K, and value vector V. Based on these vectors, the self-attention output is determined. This output is then fed into residual connections and normalization, as well as a feedforward network. Finally, the data output from the feedforward network is again subjected to residual connections and normalization to obtain the final time series features.

[0077] Specifically, for the first The Transformer layer, based on the self-attention mechanism, uses the following calculation formula for encoding: Calculate the query vector based on the self-attention mechanism. Key vector Sum value vector : in, D represents the dimension of each segment in the segment sequence. This represents the dimension of the key vector. Indicates the first The output data of the Transformer layer, when hour, It is a fragment sequence.

[0078] Attention weight Calculation: Self-attention output is : The self-attention output is subjected to residual connection and normalization to obtain normalized data. : Normalized data The input is fed into a feedforward network to obtain the output data. : in, , is the hidden layer dimension of the feedforward network.

[0079] Output data and normalized data Perform residual join and normalization again: The final output is the data output by the last Transformer layer. , where L represents the number of Transformer layers in the Transformer encoder.

[0080] As an optional implementation, see [link to implementation details]. Figure 4 As shown, another embodiment of this application discloses the extraction of time-series features from historical time-series data to obtain time-series features, specifically including the following steps: S401. Divide the historical time series data to obtain a segment sequence consisting of at least one segment.

[0081] S402. Map each segment in the segment sequence from the first dimension to the second dimension to obtain a segment sequence composed of segments in the second dimension.

[0082] In this embodiment, after dividing the historical time series data into a fragment sequence, in order to learn more features, each fragment in the fragment sequence is mapped to a higher dimension. That is, each fragment in the fragment sequence is mapped from the first dimension to the second dimension, thereby obtaining a fragment sequence composed of fragments in the second dimension. Here, the first dimension is the dimension of each fragment in the fragment sequence obtained after dividing the historical time series data, and the second dimension is the dimension to which each fragment needs to be mapped. Furthermore, the second dimension is greater than the first dimension, and the fragments in the second dimension contain more features than the fragments in the first dimension.

[0083] Specifically, the mapping formula for mapping each segment in the segment sequence from the first dimension to the second dimension is as follows: in, Map the (j+1)th segment in the segment sequence to the segment after the second dimension; Let j+1 be the (j+1)th segment in the segment sequence, and let j be the segment in the first dimension. This is the weight matrix. P is the first dimension, and D is the second dimension. The sequence of segments is composed of all mapped segments of the second dimension. .

[0084] S403. Based on the self-attention mechanism, the sequence of segments composed of segments in the second dimension is encoded to obtain temporal features.

[0085] Based on the self-attention mechanism, the sequence of segments composed of segments in the second dimension is encoded to obtain temporal features.

[0086] In this embodiment, the specific execution methods of steps S401 and S403 are the same as those of steps S301 and S302 in the above embodiment, and will not be described in detail in this embodiment.

[0087] As an optional implementation, see [link to implementation details]. Figure 5 As shown, another embodiment of this application discloses the extraction of time-series features from historical time-series data to obtain time-series features, specifically including the following steps: S501. Divide the historical time series data to obtain a segment sequence consisting of at least one segment.

[0088] S502. Add positional codes to each segment in the segment sequence according to the order of each segment in the segment sequence.

[0089] In this embodiment, after dividing historical time-series data into a segment sequence, it is necessary to add positional codes to each segment in the segment sequence according to their order within the sequence, in order to preserve the sequence information of each segment. Specifically, a positional coding matrix is ​​defined. Where N represents the number of segments in the segment sequence, and D represents the dimension of each segment. In the matrix corresponding to the segment sequence, each row corresponds to a segment, and a position encoding vector is set in each row to realize the addition of position encoding.

[0090] S503. Based on the self-attention mechanism, the segment sequence is encoded to obtain temporal features.

[0091] Based on the self-attention mechanism, the segment sequence after adding positional encoding is encoded to obtain temporal features.

[0092] In this embodiment, the specific execution methods of steps S501 and S503 are the same as those of steps S301 and S302 in the above embodiment, and will not be described in detail in this embodiment.

[0093] As an optional implementation, see [link to implementation details]. Figure 6 As shown, another embodiment of this application discloses the extraction of time-series features from historical time-series data to obtain time-series features, specifically including the following steps: S601. Divide the historical time series data to obtain a segment sequence consisting of at least one segment.

[0094] S602. Map each segment in the segment sequence from the first dimension to the second dimension to obtain a segment sequence composed of segments in the second dimension.

[0095] S603. Add positional codes to each segment in the segment sequence according to the order of each segment in the segment sequence.

[0096] Positional encoding is added to each segment in the segment sequence composed of segments in the second dimension, according to the order of each segment in the segment sequence.

[0097] S604. Based on the self-attention mechanism, the segment sequence is encoded to obtain temporal features.

[0098] Based on the self-attention mechanism, the segment sequence after adding positional encoding is encoded to obtain temporal features.

[0099] In this embodiment, the specific execution method of steps S601-S604 has been specifically described in the above embodiments, and will not be repeated in this embodiment.

[0100] Exemplary device Accordingly, embodiments of this application also provide a timing prediction device, see [link to relevant documentation]. Figure 7 As shown, the device includes: The data acquisition module 100 is used to acquire historical event text data and historical time series data of the event to be predicted; The feature determination module 110 is used to perform event semantic encoding on historical event text data to obtain event semantic features, and to extract time series features from historical time series data to obtain time series features. The feature fusion module 120 is used to perform cross-modal feature fusion of event semantic features and temporal features to obtain fused features; The prediction module 130 is used to make predictions based on fused features and obtain prediction results.

[0101] As can be seen from the above description, the time series prediction device proposed in this application introduces event text information into time series prediction and integrates event text information with time series data. This allows for corresponding changes to be made during the prediction process when an event that affects the prediction content occurs, in combination with the event text information, thereby improving the accuracy of time series prediction.

[0102] As an optional implementation, another embodiment of this application discloses that the feature determination module 110 includes: a normalization unit, a word segmentation unit, and a semantic encoding unit.

[0103] The normalization unit is used to normalize historical event text data into standard historical event text according to a preset standard format. The word segmentation unit is used to segment the standard text of historical events into words to obtain a word segmentation sequence; The semantic encoding unit is used to encode the event semantics of the word segmentation sequence using a pre-trained large language model to obtain event semantic features.

[0104] As an optional implementation, another embodiment of this application discloses that the feature determination module 110 further includes: a segmentation unit and a timing coding unit.

[0105] The segmentation unit is used to divide historical time-series data into segments, resulting in a sequence of segments consisting of at least one segment. The temporal coding unit is used to encode segment sequences based on a self-attention mechanism to obtain temporal features.

[0106] As an optional implementation, another embodiment of this application discloses that the feature determination module 110 further includes a dimension mapping unit.

[0107] The dimension mapping unit is used to map each segment in the segment sequence from the first dimension to the second dimension, resulting in a segment sequence composed of segments in the second dimension; wherein the segments in the second dimension contain more features than the segments in the first dimension.

[0108] The temporal coding unit is specifically used to encode a sequence of segments composed of segments in the second dimension based on a self-attention mechanism to obtain temporal features.

[0109] As an optional implementation, another embodiment of this application discloses that the feature determination module 110 further includes an encoding addition unit.

[0110] The encoding addition unit is used to add position codes to each segment in the segment sequence according to the order of each segment in the segment sequence; wherein, the position code represents the order of the segments in the segment sequence.

[0111] As an optional implementation, another embodiment of this application discloses that the segmentation unit is specifically used for: Historical time-series data is divided into segments according to a pre-set segment size and overlap amount to obtain a segment sequence; where the overlap amount is the amount of data overlapping between adjacent segments in the segment sequence.

[0112] As an optional implementation, another embodiment of this application discloses a feature fusion module 120, specifically used for: The event semantic features and temporal features are dimensionally aligned, and the dimensionally aligned event semantic features and temporal features are concatenated to obtain a fused sequence; Based on the self-attention mechanism, the fusion sequence is encoded to obtain fusion features.

[0113] As an optional implementation, another embodiment of this application discloses that the prediction module 130 is specifically used for: The fused features are input into a pre-trained prediction head to obtain the prediction result output by the prediction head; wherein the prediction head includes at least one prediction head, and if it includes multiple prediction heads, the prediction scale of each prediction head is different.

[0114] The timing prediction apparatus provided in this embodiment belongs to the same concept as the timing prediction method provided in the above embodiments of this application. It can execute the timing prediction method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the timing prediction method. Technical details not described in detail in this embodiment can be found in the specific processing content of the timing prediction method provided in the above embodiments of this application, and will not be repeated here.

[0115] Exemplary electronic devices Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 8 As shown, the device includes: Memory 200 and processor 210; The memory 200 is connected to the processor 210 and is used to store programs; The processor 210 is configured to implement the timing prediction method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0116] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0117] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them: A bus can include a pathway for transmitting information between various components of a computer system.

[0118] Processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0119] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0120] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0121] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0122] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0123] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0124] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of any timing prediction method provided in the above embodiments of this application.

[0125] Exemplary computer program products and storage media In addition to the methods and apparatus described above, embodiments of this application may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the timing prediction methods according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0126] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0127] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor in the timing prediction methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0128] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0129] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0130] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0131] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0132] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0133] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0134] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0135] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0137] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0138] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A time series prediction method, characterized in that, include: Obtain historical event text data and historical time series data of the event to be predicted; The historical event text data is semantically encoded to obtain event semantic features, and the historical time series data is extracted to obtain time series features. The event semantic features and the temporal features are fused across modalities to obtain fused features; Based on the fusion features, a prediction result is obtained.

2. The method according to claim 1, characterized in that, The historical event text data is semantically encoded to obtain event semantic features, including: The historical event text data is normalized into standard historical event text according to a preset standard format; The standard text of the historical events is segmented into words to obtain a segmentation sequence; The event semantic features are obtained by using a pre-trained large language model to encode the word segmentation sequence.

3. The method according to claim 1, characterized in that, The historical time-series data is subjected to time-series feature extraction to obtain time-series features, including: The historical time series data is divided to obtain a sequence of segments consisting of at least one segment; Based on the self-attention mechanism, the segment sequence is encoded to obtain temporal features.

4. The method according to claim 3, characterized in that, After dividing the historical time-series data to obtain a sequence of segments consisting of at least one segment, the process further includes: Each segment in the segment sequence is mapped from a first dimension to a second dimension to obtain a segment sequence composed of segments in the second dimension; wherein, the segments in the second dimension contain more features than the segments in the first dimension. The encoding of the segment sequence based on the self-attention mechanism to obtain temporal features includes: Based on the self-attention mechanism, the sequence of segments composed of segments of the second dimension is encoded to obtain the temporal features.

5. The method according to claim 3, characterized in that, After dividing the historical time-series data to obtain a sequence of segments consisting of at least one segment, the process further includes: According to the order of each segment in the segment sequence, a position code is added to each segment in the segment sequence; wherein the position code represents the order of the segments in the segment sequence.

6. The method according to claim 3, characterized in that, The historical time-series data is divided to obtain a sequence of segments consisting of at least one segment, including: The historical time series data is divided into segments according to a pre-set segment size and overlap amount to obtain a segment sequence; wherein, the overlap amount is the amount of data overlapping between adjacent segments in the segment sequence.

7. The method according to claim 1, characterized in that, The event semantic features and the temporal features are fused across modalities to obtain fused features, including: The event semantic features and the temporal features are dimensionally aligned, and the dimensionally aligned event semantic features and temporal features are concatenated to obtain a fused sequence; The fusion sequence is encoded based on a self-attention mechanism to obtain fusion features.

8. The method according to claim 1, characterized in that, Based on the fusion features, a prediction result is obtained, including: The fused features are input into a pre-trained prediction head to obtain the prediction result output by the prediction head; wherein the prediction head includes at least one prediction head, and if it includes multiple prediction heads, the prediction scale of each prediction head is different.

9. A time series prediction device, characterized in that, include: The data acquisition module is used to acquire historical event text data and historical time series data of the event to be predicted; The feature determination module is used to perform event semantic encoding on the historical event text data to obtain event semantic features, and to extract time-series features from the historical time-series data to obtain time-series features. The feature fusion module is used to perform cross-modal feature fusion of the event semantic features and the temporal features to obtain fused features; The prediction module is used to make predictions based on the fused features and obtain prediction results.

10. An electronic device, characterized in that, include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the timing prediction method as described in any one of claims 1 to 8 by running a program in the memory.

11. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the timing prediction method as described in any one of claims 1 to 8.

Images