Time series prediction method, electronic device, storage medium, and computer program product

Abstract

Provided in the present disclosure is a time series prediction method. The method comprises: acquiring a historical indicator time series of a prediction object, wherein the historical indicator time series comprises a plurality of historical indicators of the prediction object, and the historical indicators comprise indicator values and time information; on the basis of a first historical indicator group in the historical indicator time series, obtaining an inter-period token, wherein the first historical indicator group comprises a plurality of historical indicators within the same time range in a plurality of periods under a preset period length in the historical indicator time series, and the length of the time range is less than a preset period length; and inputting a target token into a transformer model, so as to obtain a predicted indicator time series, wherein the target token comprises the inter-period token. Further provided in the present disclosure are an electronic device, a computer-readable storage medium, and a computer program product.

Description

Timing prediction methods, electronic devices, storage media and computer program products

[0001] Cross-reference to related applications

[0002] This disclosure claims priority to Chinese Patent Application No. 202411206567.4, filed on August 30, 2024, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates to the field of computer technology, and in particular to timing prediction methods, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0004] The Transformer is a deep learning model primarily used for processing sequential data. Transformer-based models have been successfully applied in many fields, such as natural language processing and computer vision. However, when dealing with time series, especially those with periodic characteristics, the predictive performance of Transformer models remains inaccurate. Summary of the Invention

[0005] This disclosure provides time series forecasting methods, electronic devices, computer-readable storage media, and computer program products.

[0006] In a first aspect, embodiments of this disclosure provide a time series prediction method, comprising: obtaining historical indicator time series of a prediction object; the historical indicator time series includes multiple historical indicators of the prediction object, each historical indicator including indicator value and time information; obtaining a periodic token Token based on a first historical indicator group in the historical indicator time series; the first historical indicator group includes multiple historical indicators within the same time range in multiple periods under a preset period length, wherein the length of the time range is less than the preset period length; inputting a target Token into a Transformer model to obtain the prediction indicator time series; the target Token includes a periodic token.

[0007] Secondly, embodiments of this disclosure provide an electronic device, including a memory and a processor; the memory stores a computer program that can be executed by the processor, and the computer program is executed by the processor, causing the processor to implement the timing prediction method in embodiments of this disclosure.

[0008] Thirdly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor, causing the processor to implement the timing prediction method in embodiments of this disclosure.

[0009] Fourthly, embodiments of this disclosure provide a computer program product, including a computer program that is executed by a processor, causing the processor to implement the timing prediction method in embodiments of this disclosure. Attached Figure Description

[0010] In the accompanying drawings of the embodiments disclosed herein:

[0011] Figure 1 is a flowchart illustrating a time series prediction method provided in an embodiment of this disclosure;

[0012] Figure 2 is a schematic diagram of a time series prediction model architecture provided in an embodiment of this disclosure;

[0013] Figure 3 is a schematic diagram of a time series prediction model architecture provided in an embodiment of this disclosure;

[0014] Figure 4 is a flowchart illustrating the performance timing prediction of a wireless network according to an embodiment of this disclosure;

[0015] Figure 5 is a schematic diagram of the text token extraction process of a text encoding module according to an embodiment of this disclosure;

[0016] Figure 6 is a schematic diagram of the original Transformer encoding architecture of a deep learning backbone learning network module according to an embodiment of the present disclosure;

[0017] Figure 7 is a schematic diagram of a prediction decoding module using a fully connected layer according to an embodiment of the present disclosure;

[0018] Figure 8 is a schematic diagram of a prediction decoding module employing a residual connection layer according to an embodiment of this disclosure;

[0019] Figure 9 is a block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of this disclosure, the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0021] The present disclosure will be described more fully below with reference to the accompanying drawings; however, the embodiments shown may be embodied in different forms, and the present disclosure should not be construed as limited to the embodiments set forth below. These embodiments are provided so that this disclosure will be thorough and complete, and that those skilled in the art will fully understand the scope of the disclosure.

[0022] The accompanying drawings of the embodiments disclosed herein are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the detailed description of the embodiments to explain this disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the description of the detailed embodiments with reference to the accompanying drawings.

[0023] This disclosure may be described with reference to plan and / or cross-sectional views using the ideal schematic diagrams of this disclosure. Therefore, the example illustrations may be modified according to manufacturing techniques and / or tolerances.

[0024] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0025] The terminology used in this disclosure is for describing particular embodiments only and is not intended to limit the disclosure. The term "and / or" as used in this disclosure includes any and all combinations of one or more of the associated enumerated entries. The singular forms "a" and "the" as used in this disclosure also include the plural forms, unless the context clearly indicates otherwise. The terms "comprising," "made of," etc., as used in this disclosure specify the presence of a particular feature, integral, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof.

[0026] Unless otherwise specified, all terms used in this disclosure (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so specified in this disclosure.

[0027] This disclosure is not limited to the embodiments and implementations shown in the accompanying drawings, but includes modifications to the configuration based on the manufacturing process. Therefore, the areas illustrated in the drawings are schematic, and the shapes of the areas shown in the drawings illustrate the specific shapes of the areas of the element, but are not limiting.

[0028] In some related technologies, Nie, Nguyen et al. first proposed the PatchTST (patch time series transformer) algorithm based on Transformer in their March 2023 paper, "A Time Series is Worth 64 Words: Long-Term Forecasting with Transformers." The PatchTST algorithm is a novel transformer-based model that has received widespread attention and application in the field of time series analysis. However, the PatchTST algorithm mainly focuses on local temporal semantic information, neglecting the periodic information of the time series.

[0029] To address the aforementioned problems, this disclosure provides a time series prediction method, an electronic device, a computer-readable storage medium, and a computer program product. This method involves obtaining a periodic token from a first historical indicator group composed of multiple historical indicators within the same time range across multiple periods of a preset period length. A target token including the periodic token is then input into a Transformer model, thereby incorporating periodic information from the historical indicator time series into the input of the Transformer model, enhancing the effectiveness of time series prediction.

[0030] In a first aspect, embodiments of this disclosure provide a time series prediction method.

[0031] Figure 1 is a flowchart illustrating a time series prediction method provided in an embodiment of this disclosure.

[0032] As shown in Figure 1, the time series prediction method includes, but is not limited to, the following steps S101 to S103.

[0033] Step S101: Obtain the historical time series of indicators for the prediction object.

[0034] Historical indicator time series includes multiple historical indicators of the predicted object, which include indicator values ​​and time information.

[0035] The historical time series of the predicted object includes multiple indicator values ​​sorted by time. These indicator values ​​can be the results of observations of the predicted object at equal intervals according to a given sampling rate. The time information can be the timestamp of the indicator value collection time, or other forms or combinations thereof representing the time period, such as sampling rate, sampling time granularity (when the time information is sampling rate or sampling time granularity, the time information of each historical indicator is the same), etc., so that the periodic information of the historical indicator time series can be obtained based on the time information.

[0036] The historical time series of the predicted object is the time series of technical indicators with periodic change patterns. For example, it can be the time series of network element indicators of communication network elements. Communication network elements can be wireless network elements, core network elements, transmission network elements, etc. Network element indicators can be performance indicators, statistical indicators, etc., specifically indicators such as traffic and number of users in the wireless network coverage area (city / district / county).

[0037] Step S102: Obtain the weekly token Token based on the first historical indicator group in the historical indicator time series.

[0038] A token is the smallest unit used for processing Transformer models.

[0039] The first historical indicator group includes multiple historical indicators within the same time range across multiple periods of a preset period length, where the length of the time range is less than the preset period length.

[0040] Based on the cyclical characteristics of the predicted object, the preset cycle length can include, but is not limited to, minutes, hours, days, weeks, months, years, etc. According to the preset cycle length, the historical indicator time series is divided into different cycles. Based on historical indicators within the same time range of multiple different cycles, a first historical indicator group is determined. For example, if the preset cycle length is days, the first historical indicator group can include historical indicators within the time range of 0:00 to 6:00 within multiple historical days.

[0041] "The same time range within multiple cycles" refers to the time range in which the two periods are in the same relative position. For example, if the preset cycle length is days, the same time range can be from the 0th hour to the 6th hour of each cycle.

[0042] By using multiple historical indicators within the same time range in different periods, the periodic information of the historical indicator time series of the prediction object can be obtained, so as to obtain better prediction results for periodic time series.

[0043] The time range is shorter than the preset period length, thus allowing the periodic information of the historical indicators of the predicted object to be obtained with less information.

[0044] Step S103: Input the target Token into the Transformer model to obtain the time series of the prediction index.

[0045] The target tokens include tokens issued during the week.

[0046] The forecast indicator time series includes the forecast indicators of the forecast object, and the forecast indicator includes the forecast indicator value and forecast time information. In some implementations, the forecast indicator value corresponding to the forecast time information can be obtained based on the set forecast time information.

[0047] It should be noted that the embodiments disclosed herein provide a univariate time series forecasting method. For multivariate time series, the multivariate time series can be split into multiple univariate time series, and univariate time series forecasting can be performed separately. Finally, the forecasting results can be merged and output as multivariate time series forecasting results.

[0048] In some implementations, the first historical indicator group includes multiple historical indicators within the same time range across all periods of a preset period length in the historical indicator time series.

[0049] For example, if the preset period length is days, the first historical indicator group can include historical indicators within the time range from 0:00 to 6:00 for all historical days.

[0050] In some implementations, before step S103, which involves inputting the target Token into the Transformer model to obtain the predicted index time series, the method further includes: obtaining the Token within the period based on the second historical index group in the historical index time series; and / or, obtaining the global Token for the period based on the third historical index group in the historical index time series.

[0051] The second historical indicator group includes all historical indicators within a certain period of the historical indicator time series, or the second historical indicator group includes a certain historical indicator within a certain period of the historical indicator time series, which is within a certain period of the preset period length; the target token also includes tokens within the period.

[0052] In this embodiment of the disclosure, the intra-period token differs from the period token, with the intra-period token focusing more on extracting time-series information within the period. By utilizing a target token that includes at least the intra-period token and the period token, periodic time series can be predicted more effectively and comprehensively.

[0053] The third historical indicator group includes all historical indicators in the historical indicator time series, and the target token also includes the periodic global token.

[0054] In this embodiment of the disclosure, the periodic global token focuses on extracting all time-series information. By utilizing a target token that includes at least the periodic global token and the periodic interval token, periodic time series can be predicted more effectively and comprehensively.

[0055] In some implementations, step S103, that is, before inputting the target Token into the Transformer model to obtain the time series of the prediction index, further includes: obtaining a text Token based on the text of the relevant information of the prediction object; the target Token also includes the text Token.

[0056] In this embodiment of the disclosure, the text related to the predicted object is a textual description of the predicted object's feature information, such as textual descriptions of the predicted object's POI information, engineering parameter information, and performance index information. The text token includes a token obtained based on the text related to the predicted object. The addition of text tokens can improve the generalization of the time series prediction model.

[0057] In some implementations, obtaining a text token based on text containing relevant information about the predicted object includes: processing the text using a Large Language Model (LLM) to obtain the text token.

[0058] In this embodiment of the disclosure, LLM can guarantee the quality of text token generation and further improve the generalization of the model.

[0059] In some implementations, processing text according to a Large Language Model (LLM) to obtain a text token includes: processing text according to a preset prediction object description template and a Large Language Model (LLM) to obtain a text token.

[0060] In this embodiment of the disclosure, by using a preset prediction object description template in combination with LLM, the quality of text token generation can be further improved, thereby more effectively improving the generalization of the time series prediction model.

[0061] In some implementations, there are multiple preset cycle lengths.

[0062] In this embodiment, historical indicator groups of different lengths can be obtained according to each preset period length, and then different time series tokens can be extracted from them. By dividing the historical indicator time series according to multiple preset period lengths, multiple historical indicator groups corresponding to multiple preset period lengths can be obtained, thereby obtaining multiple time series tokens. By utilizing the periodic information contained in multiple time series tokens under different period lengths, the periodic information of periodic time series under different period lengths can be extracted more comprehensively, resulting in more accurate prediction results.

[0063] In some implementations, obtaining the week period token based on the first historical indicator group in the historical indicator time series in step S102 includes: encoding each historical indicator in the first historical indicator group to obtain an indicator vector; and obtaining the week period token based on the indicator vector in the first historical indicator group.

[0064] In this embodiment of the disclosure, historical indicators can be encoded as vectors, that is, each historical indicator is a d+1 dimensional vector (d represents the time information embedding encoding dimension, and 1 represents the indicator value embedding encoding dimension), so that the subsequent convolution of the two-dimensional dot matrix is ​​also performed on this d+1 dimensional vector.

[0065] In some implementations, obtaining the period token in step S102 based on the first historical indicator group in the historical indicator time series includes: converting the historical indicator time series into a two-dimensional dot matrix according to a preset period length; and convolving the two-dimensional dot matrix with a convolution kernel whose size is the range of the first historical indicator group to obtain the period token.

[0066] Each point in the two-dimensional dot matrix represents a historical indicator, and each row of the two-dimensional dot matrix represents a historical indicator within a preset period.

[0067] In this embodiment, historical indicator time series are converted into a two-dimensional dot matrix according to a preset period length, so as to facilitate convolution of the two-dimensional dot matrix using existing operators. Of course, by modifying the code of existing operators, historical indicator time series can also be convolved without converting the two-dimensional dot matrix to obtain the periodic token.

[0068] In some implementations, the historical indicator time series includes multiple first historical indicator groups with the same shape but different positions; step S102, obtaining the week period token based on the first historical indicator group in the historical indicator time series, includes: obtaining a week period token based on each first historical indicator group.

[0069] In this embodiment of the disclosure, when the shape of the historical indicator time series is larger than the shape of the historical indicator group, the historical indicator group at all positions in the historical indicator time series can be traversed and convolved according to the shape of the historical indicator group. Alternatively, one or more historical indicator groups at a certain position can be selected for convolution to obtain the time series token corresponding to the number of convolution times.

[0070] The following is an exemplary description of a time series prediction model architecture provided by an embodiment of this disclosure.

[0071] Figure 2 is a schematic diagram of a time series prediction model architecture provided in an embodiment of this disclosure. As shown in Figure 2, the model architecture comprises a time series encoding module (used to obtain a time series token based on historical indicator time series), a deep learning backbone learning network module (Transformer), and a prediction decoding module (used to obtain the predicted indicator time series). The input of the time series prediction model includes the historical indicator time series of the predicted object; the output of the time series prediction model includes the predicted indicator time series of the predicted object.

[0072] Figure 3 is a schematic diagram of a time series prediction model architecture provided in an embodiment of this disclosure. Furthermore, as shown in Figure 3, the time series prediction model also includes a prediction object text encoding module (Large Language Model, LLM); the model input also includes text of relevant information about the prediction object (used to encode a text token).

[0073] Figure 4 is a schematic flowchart of a wireless network performance timing prediction according to an embodiment of the present disclosure.

[0074] For example, as a specific implementation of this disclosure, as shown in FIG4, taking the performance time series prediction of a wireless network as an example, the input of the time series prediction model in this disclosure can be the historical performance index time series of the wireless network (the historical index time series of the prediction object), the wireless network operating parameter information, the POI information of the wireless network coverage area (text related to the prediction object), etc. The historical performance metrics of the wireless network include, but are not limited to, downlink traffic, uplink traffic, downlink PRB utilization, uplink PRB utilization, average number of RRC connected users, maximum number of RRC connected users, downlink video transmission rate, uplink video transmission rate, and total uplink and downlink traffic of the cell / area (including the values ​​and time information of multiple historical metrics); the wireless network engineering parameters include, but are not limited to, the network standard of the wireless cell, the latitude and longitude of the wireless cell, the province, city, district / county, town / street where the wireless cell is located, the coverage scenario of the wireless cell (such as urban areas, suburbs, rural areas, etc.), the frequency band of the wireless cell, the frequency point of the wireless cell, the bandwidth of the wireless cell, and the antenna type of the wireless cell; and the POI (Point of Interest) information of the wireless network coverage area, such as stations, airports, highways, subways, schools, residences, hospitals, scenic spots, hotels, business office areas, government and enterprise units, etc. The input to the timing encoding module includes the historical performance metrics timing of the wireless network; the input to the text encoding module includes the historical performance metrics timing of the wireless network, wireless network operating parameter information, and POI information of the wireless network coverage area.

[0075] The text encoding module is configured to convert the text of information related to the predicted object into a text token. The text encoding module inputs the text of information related to the predicted object into an LLM (Large Language Model) based on the predicted object description template, and obtains the text token.

[0076] Taking time series prediction of wireless network performance metrics as an example, as shown in Figure 4, the prediction object description template can extract the prediction object description according to the following template:

[0077] (1) Geographic location information of wireless network cell / area, such as xx country xx province xx city xx county xx town;

[0078] (2) Surrounding environment information of wireless network cell / area, such as there is a high-speed rail station within xxx meters;

[0079] (3) Predictive indicator information for wireless network cells / areas, such as downlink traffic;

[0080] (4) Description of historical data time periods for wireless network cells / areas, such as historical data time periods from xxxx year xx month xx day xx hour xx minute xx second to xxxx year xx month xx day xx hour xx minute xx second;

[0081] (5) Predicted time series information of wireless network cells / areas, such as the predicted data time period from xxxx year xx month xx day xx hour xx minute xx second to xxxx year xx month xx day xx hour xx minute xx second;

[0082] (6) Historical holiday information of wireless network cells / areas, such as historical data time periods including xx holidays (from xxxx year xx month xx day to xxxx year xx month xx day);

[0083] (7) Predicted holiday information for wireless network cells / areas, such as historical data time periods containing xx holidays (from xxxx year xx month xx day to xxxx year xx month xx day);

[0084] Figure 5 is a schematic diagram of the text token extraction process of a text encoding module according to an embodiment of this disclosure.

[0085] As shown in Figure 5, the text encoding module can input the description information of each line of the predicted object into the LLM model according to the description template of the predicted object. The LLM model outputs the encoded text sequence token based on the description information of each line. A text token for the predicted object is obtained from each text sequence token. For example, the maximum value, average value, or last token value of each text sequence token can be taken as the text token for the predicted object.

[0086] Those skilled in the art will understand that, in the embodiments disclosed herein, LLM includes, but is not limited to, large language models such as GPT2 (Generative Pretrained Transformer, GPT), GPT3, LLaMA, and LLaMA 2; at the same time, LLM includes, but is not limited to, the LLM model itself and improved models based on LLM, such as LLM models that have undergone compression algorithms such as pruning, quantization, and distillation; the number of layers of the LLM model used for text processing is also not limited, for example, all layers of the LLM model can be selected as the text processing model, or the first few layers of the LLM model can be selected as the text processing model.

[0087] The methods described in this disclosure optimize the resource requirements of large language models, ensuring efficient training and inference on standard hardware platforms such as CPUs (Central Processing Units) and reducing reliance on high-performance hardware (e.g., GPUs, Graphics Processing Units). This not only broadens the applicability of large language models but also makes it possible to deploy advanced technologies in resource-constrained environments.

[0088] Furthermore, before training the time series prediction model, the text of the relevant information of the prediction object can be generated into a text token using LLM, and the encoded token value can be stored on the local hard drive to reduce the computational resource overhead caused by frequent calls to LLM during training.

[0089] The timing coding module may include a timing information coding module, a time-domain two-dimensional conversion module, and a timing semantic extraction module.

[0090] Taking the time series prediction of wireless network performance indicators as an example, the historical time series of the predicted object can be shown in Table 1.

[0091] Table 1: Historical Indicator Time Series

[0092] As shown in Table 1, the same cell / area number indicates that the predicted object is the same cell / area; the timestamp (time information) indicates the time information of historical indicators; downlink traffic indicates the indicator value of the predicted object at the historical timestamp.

[0093] Depending on the prediction task, the time information encoding module can parse the timestamp into information such as year, month, day, week, hour, minute, second, and holidays, as shown in Table 2. Holidays can include New Year's Day, Spring Festival, Qingming Festival, International Labor Day, Dragon Boat Festival, Mid-Autumn Festival, and National Day.

[0094] Table 2: Components of Timestamp Information

[0095] The parsed timestamp information and indicator values ​​are embedded through the embedding layer to generate d-dimensional timestamp embedding encoding vectors and indicator value vectors, respectively, forming a d+1-dimensional indicator vector. For example, for each historical indicator, the time information encoding module encodes the year, month, day, weekday, hour, minute, second, and holidays into d-dimensional vectors, then calculates the average or maximum value of all d-dimensional vectors to generate a d-dimensional timestamp embedding encoding vector, and encodes the indicator value into a 1-dimensional indicator value vector, ultimately forming a d+1-dimensional indicator vector. d represents the time information embedding encoding dimension.

[0096] The time-domain two-dimensional conversion module converts the indicator vectors of historical indicator time series into a two-dimensional dot matrix based on at least one preset period length. Each point in the two-dimensional dot matrix represents the indicator vector of a historical indicator, and each row (or column) of the two-dimensional dot matrix represents the indicator vector of a historical indicator within a preset period length.

[0097] Assume the historical time series indicators are hourly granular data, with a total of 672 indicator values. If the preset period length is 24, then the one-dimensional time series vector R can be... 1×672 Convert to a two-dimensional point matrix vector R 24×28 If the preset period length is 168, then the one-dimensional time vector R can be... 1×672 Convert to a two-dimensional point matrix vector R 168×4 If the length of a one-dimensional time series vector is not divisible by the preset period length, the length of the one-dimensional time series vector can be padded at the beginning or end of the time series to make it divisible by the preset period length.

[0098] Since a set of time series often has multiple period lengths, for example, the period lengths of minute-level data are 60, 1440, and 10080, which correspond to hourly, daily, and weekly periods, respectively; the period lengths of 15-minute-level data may be 4, 96, and 672, which correspond to hourly, daily, and weekly periods, respectively; the period lengths of hourly-level data may be 24 and 168, which correspond to daily and weekly periods, respectively; and the period lengths of daily-level data may be 7 and 365, which correspond to weekly and yearly periods, etc.

[0099] Therefore, further, the time-domain two-dimensional transformation module can convert the indicator vector of historical indicator time series into two-dimensional dot matrices of different shapes according to various preset period lengths. As shown in Figure 4, period 1, period 2... period K represent K different preset period lengths.

[0100] The temporal semantic extraction module convolves a two-dimensional point matrix using convolution kernels of different shapes (for example, based on a first historical indicator group, i.e., multiple historical indicators within the same time range across multiple periods of the historical indicator time series, where the length of the time range is less than a preset period length), to obtain temporal tokens containing different temporal information. The temporal tokens include at least a period-interval token. Furthermore, the temporal tokens also include an intra-period token and / or a period-wide token.

[0101] For example, the temporal semantic extraction module convolves the two-dimensional point matrix with a periodic convolution kernel whose size is the range of the first historical indicator group to obtain the periodic token; it convolves the two-dimensional point matrix with an intra-period convolution kernel whose size is the range of the second historical indicator group to obtain the intra-period token; and it convolves the two-dimensional point matrix with a global convolution kernel whose size is the range of the third historical indicator group to obtain the periodic global token.

[0102] When the shape of the convolution kernel is smaller than the shape of the two-dimensional dot matrix of the historical index time series, the two-dimensional dot matrix of the historical index time series includes multiple convolution kernels of historical index groups with the same shape but different positions. The time series semantic extraction module obtains a period token based on the convolution kernel at each position. For example, it can perform traversal convolution on the two-dimensional dot matrix of the historical index time series according to the shape of the convolution kernel (the stride of the convolution is greater than or equal to 1), or it can select one or more convolution kernels at one or more positions to perform one or more convolutions on the two-dimensional dot matrix of the historical index time series, thereby obtaining a number of tokens corresponding to the number of convolutions. For period tokens, it can perform traversal convolution only in the period direction (the dot matrix in the period direction is within one period), and for tokens within a period, it can perform convolution only in the cross-period direction (the dot matrix in the cross-period direction is within different periods).

[0103] When the side length of the convolution kernel is greater than 1, a certain number of rows (or columns) can be added to each side of the two-dimensional dot matrix by padding, so that the number of output tokens in length (or width) is consistent with the length (or width) of the two-dimensional dot matrix of historical index time series.

[0104] Suppose a one-dimensional time series vector is transformed into a two-dimensional lattice vector. m k S represents the period length. k If the period represents the number of periods, then the kernel shape within a period is defined as Kernel = (m k ,w),1≤w k The two-dimensional point matrix is ​​filled and subjected to traversal convolution with a stride of 1 to generate S in the cross-period direction. k Tokens within a period; the convolution kernel shape during the period is defined as kernel = (h, S) k ),1≤h <m k The two-dimensional point matrix is ​​filled and subjected to traversal convolution with a stride of 1 (each convolution is performed on a first historical index group, with the time range being the same time range within multiple periods), generating m in the periodic direction. k Tokens during each week; the global convolution kernel shape is defined as kernel = (m k ,S k ​This generates a global token for one period. w can be either 1 or 3, where w=1 indicates convolution within non-overlapping periods and w=3 indicates convolution within overlapping periods; h can also be either 1 or 3, where h=1 indicates convolution within non-overlapping periods and h=3 indicates convolution within overlapping periods.

[0105] Assuming the historical time series indicators are hourly granular data with a total of 672 values, then the one-dimensional time series vector R can be divided into two parts according to preset period lengths of 24 and 168 respectively. 1×672 Transform into a two-dimensional lattice vector R 24×28 and R 168×4 For a two-dimensional lattice vector R with a preset period length of 24... 24×28 The periodic convolution kernel is Kernel = (24, w), 1 ≤ w < 28. It fills the 2D point matrix and performs traversal convolution with a stride of 1, generating 28 periodic tokens in the cross-period direction. The periodic convolution kernel is kernel = (h, 28), 1 ≤ h < 24. It fills the 2D point matrix and performs traversal convolution with a stride of 1, generating 24 periodic tokens in the period direction. The global convolution kernel is kernel = (24, 28), generating 1 periodic global token. The periodic tokens, periodic tokens, and periodic global tokens are combined to form 53 temporal tokens. For a 2D point matrix vector R with a period length of 168... 168×4 The periodic convolution kernel is Kernel=(168,w),1≤w<4, which fills the two-dimensional point matrix and performs traversal convolution with a stride of 1, generating 4 periodic tokens in the cross-period direction; the periodic convolution kernel is kernel=(h,4),1≤h<168, which fills the two-dimensional point matrix and performs traversal convolution with a stride of 1, generating 168 periodic tokens in the periodic direction; the global convolution kernel is kernel=(168,4), generating 1 periodic global token; the periodic tokens, periodic tokens and periodic global tokens are combined to form 173 temporal tokens.

[0106] As shown in Figures 2 and 3, the input of the deep learning backbone learning network module includes temporal tokens or a token sequence consisting of temporal tokens and text tokens, and its output includes a deep encoded token sequence.

[0107] Figure 6 is a schematic diagram of the original Transformer encoding architecture of a deep learning backbone learning network module according to an embodiment of this disclosure.

[0108] As shown in Figure 6, the deep learning backbone learning network module can use the encoding part of the original Transformer architecture. Alternatively, the deep learning backbone learning network module can use the encoding part of a variant Transformer architecture.

[0109] Figure 7 is a schematic diagram of a prediction decoding module using a fully connected layer according to an embodiment of the present disclosure.

[0110] Figure 8 is a schematic diagram of a prediction decoding module employing a residual connection layer according to an embodiment of this disclosure.

[0111] As shown in Figure 7, the prediction decoding module can use a fully connected layer, or, as shown in Figure 8, the prediction decoding module can also use a residual fully connected layer. In the selection and flattening functions, selection involves choosing some or one token from L tokens as input to the next layer, while flattening transforms a 2D vector into a 1D vector.

[0112] In some embodiments, model training may employ optimization algorithms, such as the Adam optimization algorithm or the AdamW optimization algorithm. Model training may use MSE (Mean Squared Error) or MAE (Mean Absolute Error) as the loss function. MSE is defined as follows: x T+i Represents the true value. This represents the predicted value; MAE is defined as follows: x T+i Represents the true value. This represents the predicted value. T represents the length of the historical indicator time series, and F represents the length of the predicted indicator time series.

[0113] Secondly, embodiments of this disclosure provide an electronic device.

[0114] Figure 9 is a block diagram of an electronic device provided in an embodiment of this disclosure.

[0115] As shown in FIG9, the electronic device includes: at least one processor 901 and at least one memory 902. The memory 902 stores one or more computer programs, which are executed by the at least one processor 901 to enable the at least one processor 901 to implement the timing prediction method described in the above embodiments.

[0116] The modules in the aforementioned electronic devices can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0117] Thirdly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program causes the processor to implement the timing prediction method described in the above embodiments. The computer-readable storage medium may be volatile or non-volatile.

[0118] Fourthly, embodiments of this disclosure also provide a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the timing prediction method described above.

[0119] A processor is a device with data processing capabilities, including but not limited to a central processing unit (CPU); a memory is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); an I / O interface (read / write interface) connects the processor and the memory, enabling information exchange between the memory and the processor, including but not limited to a data bus (Bus).

[0120] Those skilled in the art will understand that all or some of the steps, functional modules / units in the apparatus disclosed above can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0121] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0122] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be executed by several physical components working together.

[0123] Some or all of the physical components may be implemented as software executed by a processor (such as a central processing unit (CPU), digital signal processor, or microprocessor), or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technique for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH) or other disk storage; read-only optical disc (CD-ROM), digital versatile disc (DVD) or other optical disc storage; magnetic cartridges, magnetic tapes, disk storage or other magnetic storage; and any other media that can be used to store desired information and can be accessed by a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0124] This disclosure has disclosed exemplary embodiments and implementations, and although specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments and implementations may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments and implementations, unless otherwise expressly stated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A time series prediction method, comprising: Obtain the historical time series of indicators for the prediction object; the historical time series of indicators includes multiple historical indicators of the prediction object, and the historical indicators include indicator values ​​and time information; A week-long token is obtained based on the first historical indicator group in the historical indicator time series; the first historical indicator group includes multiple historical indicators within the same time range in multiple periods under a preset period length in the historical indicator time series, and the length of the time range is less than the preset period length. The target token is input into the Transformer model to obtain the time series of the predicted index; the target token includes the token of the week period.

2. The method according to claim 1, wherein, The first historical indicator group includes multiple historical indicators within the same time range across all periods under the preset period length in the historical indicator time series.

3. The method according to claim 1, further comprising: Before inputting the target Token into the Transformer model to obtain the predicted indicator time series, the periodic Token is obtained according to the second historical indicator group in the historical indicator time series; the second historical indicator group includes all historical indicators in a portion of the historical indicator time series within the preset period length, or the second historical indicator group includes a portion of the historical indicators in a period within the preset period length, and the target Token also includes the periodic Token. And / or, The periodic global token is obtained based on the third historical indicator group in the historical indicator time series; the third historical indicator group includes all historical indicators in the historical indicator time series, and the target token also includes the periodic global token.

4. The method according to claim 1, further comprising: Before inputting the target token into the Transformer model to obtain the time series of the prediction index, a text token is obtained based on the text of the relevant information of the prediction object; the target token also includes the text token.

5. The method according to claim 4, wherein, The process of obtaining the text token based on the text of relevant information about the predicted object includes: The text is processed using a Large Language Model (LLM) to obtain the text token.

6. The method according to claim 1, wherein, There are multiple preset cycle lengths.

7. The method according to claim 1, wherein, The step of obtaining the weekly token based on the first historical indicator group in the historical indicator time series includes: Encode each historical indicator in the first historical indicator group to obtain an indicator vector; The weekly token is obtained based on the indicator vector in the first historical indicator group.

8. The method according to claim 1, wherein, The step of obtaining the weekly token based on the first historical indicator group in the historical indicator time series includes: The historical indicator time series is converted into a two-dimensional dot matrix according to the preset period length; each dot of the two-dimensional dot matrix represents a historical indicator, and each row of the two-dimensional dot matrix represents a historical indicator within the preset period length. The week period token is obtained by convolving the two-dimensional dot matrix with a convolution kernel whose size is the range of the first historical index group.

9. The method according to claim 1, wherein, The historical indicator time series includes multiple first historical indicator groups with the same shape but different positions; obtaining the week-period token based on the first historical indicator groups in the historical indicator time series includes: A weekly token is obtained for each of the first historical indicator groups.

10. The method according to claim 1, wherein, The historical time series of the predicted object is the time series of the network element indicators of the communication network element.

11. An electronic device, comprising a memory and a processor; the memory storing a computer program executable by the processor, the computer program being executed by the processor to cause the processor to implement the timing prediction method according to any one of claims 1 to 10.

12. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor such that the processor implements the timing prediction method according to any one of claims 1 to 10.

13. A computer program product comprising a computer program that is executed by a processor, such that the processor implements the timing prediction method according to any one of claims 1 to 10.

Images