Flow utilization rate prediction method and device, electronic equipment, medium and product

Abstract

The invention provides a traffic utilization rate prediction method and device, electronic equipment, a medium and a product, and belongs to the technical field of data processing, and the method comprises the steps that to-be-predicted traffic utilization rate data is acquired, and the to-be-predicted traffic utilization rate data comprises historical time sequence data and text description information; performing cue word recognition based on the text description information to obtain text embedded information; performing time sequence decomposition based on the historical time sequence data to obtain time sequence embedded information; the text embedding information and the time sequence embedding information are combined and then input into a large language model, and a prediction vector output by the large language model is obtained; and determining a predicted traffic utilization rate of the to-be-predicted traffic utilization rate data based on the prediction vector. According to the method, the predicted traffic utilization rate of the to-be-predicted traffic utilization rate data can be rapidly and accurately determined, so that the prediction efficiency of the traffic utilization rate can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a flow utilization prediction method, device, electronic equipment, medium and product. Background Art

[0002] Broadband Remote Access Server (BRAS) data usually contains user connection information, traffic transmission status, and other related network activity information. In addition, there are traffic differences in the distribution of business types at different times of the day on weekdays and holidays. Designing a single small time series prediction model cannot accurately model such data with diverse data distribution. The existing technical solution is to design dedicated small time series prediction models for different time series prediction tasks. When faced with diverse data and tasks, its ability to adapt to different types of data and tasks is relatively weak. In addition, due to the small model's fewer parameters and structures and weak feature representation capabilities, it may not be able to capture the complex patterns and relationships in the data, resulting in inferior performance to the large model.

[0003] When using a large model for time series prediction, the existing technical solution is to design and pre-train a basic large time series model from scratch, and then fine-tune the model accordingly for various downstream tasks. However, the shortcomings of the above solution are: this method requires training and fine-tuning from scratch, which requires a large amount of time series data, and time series data is usually professional data that is difficult to obtain; at the same time, training and fine-tuning from scratch require a lot of computing resources, which is time-consuming and labor-intensive.

[0004] Therefore, when traffic utilization prediction is performed based on the existing technology, the above-mentioned problem of being unable to quickly and accurately complete the traffic utilization prediction also exists, resulting in low efficiency in the current traffic utilization prediction. Summary of the invention

[0005] The present application provides a flow utilization prediction method, device, electronic device, medium and product to solve the problem that flow utilization prediction cannot be completed quickly and accurately when flow utilization prediction is performed based on the existing technology, so as to improve the prediction efficiency of flow utilization.

[0006] The present application provides a traffic utilization prediction method, the method comprising: obtaining traffic utilization data to be predicted, the traffic utilization data to be predicted comprising historical time series data and text description information; performing prompt word recognition based on the text description information to obtain text embedding information; performing time series decomposition based on the historical time series data to obtain time series embedding information; merging the text embedding information and the time series embedding information and inputting them into a large language model to obtain a prediction vector output by the large language model; and determining the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

[0007] According to a traffic utilization prediction method provided by the present application, the time series decomposition is performed based on the historical time series data to obtain time series embedding information, including: performing time series decomposition on the historical time series data based on the Prophet model to obtain each item of time series data after decomposition; normalizing each item of time series data respectively to obtain multiple normalized data; respectively obtaining the embedding vector corresponding to each normalized data; and generating time series embedding information based on each embedding vector.

[0008] According to a traffic utilization prediction method provided by the present application, when respectively obtaining the embedding vector corresponding to each normalized data, the following steps are respectively performed for each normalized data: determining the target boundary range of the current normalized data from multiple boundary ranges; wherein each boundary range is formed by multiple quantized values; determining the index number of the target boundary range; obtaining the embedding vector corresponding to the index number in a preset time series vocabulary; wherein the preset time series vocabulary includes a one-to-one corresponding association relationship between the index number and the embedding vector.

[0009] According to a traffic utilization prediction method provided by the present application, the preset time series vocabulary is obtained by performing dimensionality reduction processing on a pre-trained vocabulary through a learnable linear layer.

[0010] According to a traffic utilization prediction method provided by the present application, the predicted traffic utilization of the traffic utilization data to be predicted is determined based on the prediction vector, including: mapping the prediction vector through a fully connected layer to obtain a mapping result; determining the probability of the mapping result being classified into each quantization value through a preset function; determining a target quantization value from each quantization value based on each probability; and determining the predicted traffic utilization of the traffic utilization data to be predicted based on the target quantization value.

[0011] According to a traffic utilization prediction method provided by the present application, the predicted traffic utilization of the traffic utilization data to be predicted is determined based on the target quantization value, including: determining a boundary range corresponding to the target quantization value; randomly selecting a target value from the boundary range corresponding to the target quantization value; inverse normalizing the target value to obtain a continuous value; and determining the predicted traffic utilization of the traffic utilization data to be predicted based on the continuous value.

[0012] According to a traffic utilization prediction method provided by the present application, the prompt word recognition is performed based on the text description information to obtain text embedding information, including: encoding the text description information after prompt word segmentation to obtain segmentation information; mapping the segmentation information to a feature vector of a fixed length to obtain text embedding information.

[0013] The present application also provides a traffic utilization prediction device, including: an acquisition module, used to obtain traffic utilization data to be predicted, the traffic utilization data to be predicted includes historical time series data and text description information; an identification module, used to perform prompt word recognition based on the text description information to obtain text embedding information; a decomposition module, used to perform time series decomposition based on the historical time series data to obtain time series embedding information; an input module, used to merge the text embedding information and the time series embedding information and input them into a large language model to obtain a prediction vector output by the large language model; a determination module, used to determine the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

[0014] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the traffic utilization prediction method described in any one of the above methods is implemented.

[0015] The present application also provides a medium, which is a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the traffic utilization prediction method described in any one of the above is implemented.

[0016] The present application also provides a product, which is a computer program product, including a computer program, and when the computer program is executed by a processor, it implements any of the traffic utilization prediction methods described above.

[0017] The traffic utilization prediction method, device, electronic device, medium and product provided by the present application, after obtaining the traffic utilization data to be predicted including historical time series data and text description information, obtain text embedding information by performing prompt word recognition based on the text description information, and obtain time series embedding information by performing time series decomposition based on the historical time series data, so as to adapt to the processing format of the large language model, and directly align the text embedding information and the time series embedding information to the large language model, which can improve the reasoning ability of the large model, and can use the large language model to learn more complex and abstract time series patterns and features to improve the expression ability of the model, and avoid the time and labor-intensive training of the large language model, so that the accurate prediction vector output by the large language model can be obtained, and then based on the prediction vector, the predicted traffic utilization of the traffic utilization data to be predicted can be quickly and accurately determined, thereby improving the prediction efficiency of the traffic utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flow chart of the flow utilization prediction method provided by this application.

[0020] Figure 2 It is a schematic diagram of the application scenario of the traffic utilization prediction method provided in this application.

[0021] Figure 3 It is a schematic diagram of the structure of the electronic device provided by this application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0024] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0025] Combine the following Figure 1-Figure 3 Describe the traffic utilization prediction method, device, electronic device, medium and product of the present application.

[0026] Figure 1 is a flow chart of the flow rate prediction method provided by this application, such as Figure 1 As shown, the method includes the following steps 100 to 500.

[0027] Step 100: Obtain traffic utilization data to be predicted, where the traffic utilization data to be predicted includes historical time series data and text description information.

[0028] Step 200: Recognize prompt words based on text description information to obtain text embedding information.

[0029] Step 300: Perform time series decomposition based on historical time series data to obtain time series embedding information.

[0030] Step 400: Combine the text embedding information and the time series embedding information and input them into a large language model to obtain a prediction vector output by the large language model.

[0031] Step 500: Determine the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

[0032] It should be noted that the execution entity of the traffic utilization prediction method provided in the embodiment of the present application can be a server, computer equipment, etc., such as a mobile phone, tablet computer, laptop computer, PDA, vehicle-mounted electronic equipment, wearable device, ultra-mobile personal computer (Ultra-mobile Personal Computer, UMPC), netbook or personal digital assistant (Personal Digital Assistant, PDA), etc.

[0033] A traffic utilization prediction device may be provided or connected to the server or computer device of the present application, thereby controlling the traffic utilization prediction device to execute the traffic utilization prediction method of the present application.

[0034] The traffic utilization prediction device in the present application may include an acquisition module, a pre-processing module, a large language model module and a post-processing module, wherein the pre-processing module may include a recognition module, a decomposition module and an input module, and the post-processing module may include a determination module. The recognition module in the present application may also be referred to as a text embedding module, and the decomposition module may also be referred to as a time series embedding module.

[0035] The present application can obtain the traffic utilization data to be predicted including historical time series data and text description information through an acquisition module.

[0036] It should be noted that the traffic utilization data to be predicted in the present application is data in the BRAS, and therefore can also be referred to as BRAS data.

[0037] The text description information may include certain prompt words. Prompt words play a key role in the large language model question answering system. They can help the large model understand the requirements and give relevant answers. In order to activate this capability of the large language model (LLM), a prompt describing the BRAS timing information is used as the natural language input for specifying the LLM task. The following is an example of text description information, in which the text description information includes: [Introduction]: BRAS data usually refers to user access data collected through BRAS devices, including inflow and outflow rates, inflow and outflow utilization, etc. This data is of great significance to operators and can be used for network operation and maintenance management, user behavior analysis, network performance optimization, etc. Time sampling is performed at a granularity of five minutes, and the total sampling time is seven days.

[0038] [Domain knowledge]: BRAS data has a certain periodicity, which is one day. BRAS data generally reaches two peaks at noon and early morning. BRAS data has different traffic patterns on weekdays and holidays. Chinese holidays are used.

[0039] [Instructions]: You are an advanced time series forecasting model. Your task is to combine the contextual text information, understand the characteristics of the current data, and make accurate long-term and short-term forecasts of the data. <x>time steps, predicting the future <y>time steps.

[0040] [Statistics]: The minimum value of the input timing is <min>, the maximum value is <max>, the mean is <mean>, the variance is <var>The timing cycle is <t>, the sorting entropy is <entropy>The overall trend is <trend>The timing starts at <time>.

[0041] In the "Introduction", you can specify information related to the dataset, such as describing its background and time sampling operations. For the same dataset, the content of the introduction can be reused.

[0042] In "domain knowledge", you can add expert prior knowledge of the time series being predicted, such as what obvious characteristics the time series has and what trends it has.

[0043] In "Instructions" programmatically set the forecast time horizon and the input size of the series.

[0044] In "Statistical Information", the basic statistical properties of the input sequence are automatically calculated. The time series is converted to the frequency domain through fast Fourier transform, and the frequency with the largest amplitude in the frequency domain is selected to calculate the time series period; you can also select the frequencies corresponding to the first N amplitudes for statistics.

[0045] The historical time series data may be the original BRAS time series data, and specifically may be the known time series data of the traffic utilization rate in the original BRAS data, for example, represented as x 1:S =[x1, x2, …, x S ].

[0046] Furthermore, the recognition module and the decomposition module in the pre-processing module can be used to extract BRAS timing information based on historical time series data and text description information respectively. Specifically, prompt word recognition can be performed based on the text description information and timing decomposition can be performed based on the historical time series data. The extracted BRAS timing information contains text information and data information, thereby obtaining corresponding text embedding information and timing embedding information.

[0047] It should be further explained that BRAS, as a large network device, processes the data traffic of a large number of users. The traffic data is usually large-scale and needs to be effectively processed. Since BRAS data covers various types of network activities and the data distribution is different in different time periods, the large model can be used to represent more general patterns and laws, which has stronger robustness when processing complex BRAS time series data.

[0048] In order to improve the adaptability of the large language model to time series prediction tasks, the time series information is described from two dimensions: text and time series data. Time series features are extracted by using prompt word prefixes and time series decomposition, and the two are aligned to the reasoning module of the large language model to improve the reasoning ability of the large model for time series data.

[0049] Therefore, the present application can align (specifically, concatenate) the text embedding information and the time series embedding information through the input module to convert them into a form that can be understood by the large language model, and then directly input the aligned embedding information into the existing large language model for reasoning to obtain the prediction vector output by the large language model. text and timing embedded information Z seq The process can be realized by the following formula: Z = Concat(Z text , Z seq ).

[0050] Specifically, the large language model module in this application directly uses GLMBlock in ChatGLM2-6B as the large language model, where 28 GLMBlocks are stacked in ChatGLM2-6B. The parameters of GLMBlock are not updated during the training process, but are only used to output results. Among them, ChatGLM2-6B is a Chinese dialogue generation model. This application does not specifically limit the architecture of GLMBlock, and various existing architectures can be used for implementation.

[0051] After the aligned embedding information is input into the large language model (specifically, into the GLMBlock of ChatGLM2-6B), the token vector Z´ output by the large language model as the prediction vector can be obtained.

[0052] For time series prediction based on a large language model, for prediction tasks with large data distribution differences such as BRAS, its parameters and representation can be adjusted more flexibly to adapt to new data features and patterns.

[0053] Furthermore, the present application may perform timing prediction according to the output of the large language model based on the determination module in the post-processing module.

[0054] Specifically, the prediction vector can be first mapped through a fully connected layer to obtain a mapping result, and then the probability of the mapping result being classified into each pre-divided quantization value can be determined through a preset function. Further, the target quantization value can be determined from each quantization value based on each probability, and finally the predicted traffic utilization of the traffic utilization data to be predicted is determined based on the target quantization value.

[0055] The traffic utilization prediction method provided in the embodiment of the present application, after obtaining the traffic utilization data to be predicted including historical time series data and text description information, obtains text embedding information by performing prompt word recognition based on the text description information, and obtains time series embedding information by performing time series decomposition based on the historical time series data, so as to adapt to the processing format of the large language model, and directly aligns the text embedding information and the time series embedding information to the large language model, which can improve the reasoning ability of the large model, and can use the large language model to learn more complex and abstract time series patterns and features to improve the expression ability of the model, and avoid the time and labor-intensive training of the large language model. Therefore, an accurate prediction vector output by the large language model can be obtained, and then based on the prediction vector, the predicted traffic utilization of the traffic utilization data to be predicted can be quickly and accurately determined, thereby improving the prediction efficiency of the traffic utilization.

[0056] Different from traditional technologies that design dedicated small time series prediction models for different time series prediction tasks, this application takes into account that when faced with diverse data and tasks, the adaptability of small models is relatively weak and due to fewer parameters and structures, it is impossible to have good prediction accuracy on a large amount of complex time series data. Therefore, a large model architecture is designed to learn more complex and abstract time series patterns and features, thereby improving the prediction accuracy and generalization of the model.

[0057] Different from the traditional technology of designing and pre-training a basic large time series model from scratch, and then fine-tuning the model accordingly for various downstream tasks. This application takes into account that this method requires training and fine-tuning from scratch, which is time-consuming and labor-intensive; a large amount of time series data is required for training, and time series data is usually professional data that is not easy to obtain. Therefore, a special encoder and decoder structure is designed for training, which does not require any processing of the LLM and can adapt to few-sample and zero-sample scenarios. At the same time, in order to improve the prediction accuracy, the time series decomposition method is used to make it easier for LLM to learn different patterns that may exist in the time series; the text prompt word method is used to make the LLM output word embedding that can better represent the description of the time series information. Align these two heterogeneous information in one dimension to enhance the semantic understanding ability of LLM. The final output is the predicted probability distribution, which provides uncertainty information, supports risk assessment and decision making.

[0058] In one embodiment, prompt word recognition is performed based on text description information to obtain text embedding information, including: segmenting the text description information into prompt words and then encoding them to obtain segmentation information; mapping the segmentation information to a feature vector of a fixed length to obtain text embedding information.

[0059] This application can input text description information into an existing large language model. Taking ChatGLM2-6B as an example, the input text description information is first segmented and encoded, and the model adopts the wordpiece segmentation method to obtain the segmentation information. Then the embedding layer of ChatGLM2-6B is extracted. The parameters of the embedding layer are trained, and no additional training is required in this technical solution. The embedding layer directly maps the segmentation information to a fixed-length feature vector to obtain the text embedding information Z text .

[0060] This application adopts the method of text prompt words to make the LLM output more representative of the word embedding of the description of the timing information, which helps to improve the prediction accuracy of the traffic utilization rate.

[0061] In one embodiment, time series decomposition is performed based on historical time series data to obtain time series embedding information, including: performing time series decomposition on the historical time series data based on a Prophet model to obtain each item of time series data after decomposition; normalizing each item of time series data to obtain multiple normalized data; respectively obtaining an embedding vector corresponding to each normalized data; and generating time series embedding information based on each embedding vector.

[0062] Furthermore, when respectively obtaining the embedding vector corresponding to each normalized data, for each normalized data, the following steps are respectively performed: determining the target boundary range of the current normalized data from multiple boundary ranges; wherein each boundary range is formed by multiple quantization values; determining the index number of the target boundary range; obtaining the embedding vector corresponding to the index number in a preset time series vocabulary; wherein the preset time series vocabulary includes a one-to-one corresponding relationship between the index number and the embedding vector.

[0063] Specifically, the time series embedding module is the key to extracting time series features in the large model. The input is the original BRAS time series data (i.e. the historical time series data mentioned above). In order to reduce the complexity of the data and make it easier to understand and analyze, time series decomposition is used to decompose the original BRAS time series data into multiple components such as trends, seasonality, and holidays.

[0064] (X g (t), X s (t), X h (t), X r (t), X ε (t)) = Prophet(X(t)); The Prophet model is used to decompose the original BRAS time series data into trend X g (t), seasonality X s (t), Holidays X h (t), regression term X r (t) and the noise term X ε (t). The Prophet model is a tool for time series forecasting, especially suitable for processing data with obvious seasonality and holiday effects.

[0065] The embedding layer of the large language model can realize text encoding, but the mapping relationship between the source vector and the target vector of the text is not necessarily adapted to the data distribution of the time series. If the word embedding is directly applied to the original BRAS time series, there will be large losses and errors. Therefore, this application performs a learnable linear mapping on the word embedding of ChatGLM2-6B and converts it to a new feature space suitable for time series data encoding. Specifically, there are: E´=W Emb E; Among them, E∈ℝ V×D The pre-trained vocabulary for ChatGLM2-6B is passed through a learnable linear layer W Emb ∈ℝ V´×V Reduce the original text vocabulary (i.e. the pre-trained vocabulary mentioned above) to the temporal vocabulary E´∈ℝ V´×D , and as a preset temporal vocabulary.

[0066] Furthermore, the large language model operates on tokens in a finite vocabulary, so continuous time series data needs to be mapped to discrete values, and then the corresponding token embedding is queried from the preset time series vocabulary E'. It should be noted that in the large language model, a token refers to the smallest unit that is segmented in the text. It can be a word, a subword, a character, or a part of these combinations. Each token is defined as an embedding vector in the preset time series vocabulary of this application.

[0067] This application uses a quantized approach to complete the continuous-discrete value conversion: First, the decomposed time series data are mapped to the range of [0, 1] suitable for quantization through normalization: ; Among them, X represents the time series value of the time series data, X min Indicates the minimum time series value in each time series data, X max Indicates the maximum time series value among each time series data.

[0068] In this way, a plurality of normalized data can be obtained.

[0069] In this application, V´ quantization values ​​b1<… V´ ​, define the boundary range formed by each quantized value and set an ID number for each boundary range, so according to each normalized data, the data falling within each boundary range is discretized into a corresponding ID number (hereinafter referred to as an index number in this application): ; Since the data distribution of the unknown test set may be very different from the training distribution, a uniform boundary range is chosen.

[0070] It should be noted that the present application can also associate each embedded vector with one of the above-mentioned index numbers in a preset time-series vocabulary, thereby obtaining a one-to-one corresponding relationship between the index number and the embedded vector.

[0071] Furthermore, the quantized value is used as the index id to query the corresponding token embedding vector e∈ℝ in the preset time series vocabulary E′ 1×D , thus obtaining the embedding vectors corresponding to each normalized data.

[0072] In this way, each component of the BRAS time series is transformed by this normalization-quantization-mapping method, and finally the embedding vectors of each component are aggregated through the linear layer to obtain the time series embedding information Z as the time series feature vector seq Among them, the time series embedding information Zseq can be expressed by the following formula: Zseq=FC(E g , E s , E h , E r , E ε ).

[0073] This application designs a codec structure suitable for time series representation and combines it with the existing large language model, which only requires training of a small number of parameters. It can use the large model to learn more complex and abstract time series patterns and features, improve the expressiveness of the model, and avoid the time-consuming and labor-intensive training of the large model. The encoding and decoding of the data uses normalization-quantization-mapping to convert the regression problem into a classification problem, output the predicted probability distribution, and better evaluate the possibility of future events.

[0074] In one embodiment, the predicted traffic utilization rate of the traffic utilization data to be predicted is determined based on the prediction vector, including: mapping the prediction vector through a fully connected layer to obtain a mapping result; determining the probability of the mapping result being classified into each quantization value through a preset function; determining a target quantization value from each quantization value based on each probability; and determining the predicted traffic utilization rate of the traffic utilization data to be predicted based on the target quantization value.

[0075] Furthermore, the predicted traffic utilization rate of the traffic utilization rate data to be predicted is determined based on the target quantization value, including: determining a boundary range corresponding to the target quantization value; randomly selecting a target value from the boundary range corresponding to the target quantization value; inverse normalizing the target value to obtain a continuous value; and determining the predicted traffic utilization rate of the traffic utilization rate data to be predicted based on the continuous value.

[0076] Specifically, the main function of the post-processing module of this application is to perform time series prediction based on the output of the large language model. First, the token vector Z' output by the large language model is mapped through a Dense layer (fully connected layer).

[0077] It should be noted that the output token vector Z' of a large language model is usually a high-dimensional vector that represents the model's understanding of the current context. This vector is linearly transformed through a fully connected layer (Dense layer). The Dense layer has a weight matrix (W) and a bias (b), which maps the high-dimensional vector to the dimensions of the vocabulary.

[0078] The probability of the mapped vector being classified into each quantized value is further calculated by the preset softmax function. In this application, the Dense layer uses cross entropy loss to optimize the distribution between the quantized real label and the predicted distribution, converting the regression problem into a multi-classification problem. The prediction goal is to predict the probability of the next token being each quantized value based on the previous token. The expression of cross entropy loss is as follows: ; Where S is the known time series length, H is the predicted time series length, and V´ is the number of quantized values.

[0079] It should be noted that when calculating the probability that the mapped vector is classified into each quantization value, it is actually calculating the probability that the mapped vector is classified into the embedding vector corresponding to each quantization value.

[0080] Further, the quantized value corresponding to the maximum probability is selected as the predicted discrete value of the current point and can be defined as the target quantized value. Further, the predicted discrete value can be mapped back to the actual value.

[0081] Specifically, a target value can be randomly selected from the range of the target quantized value, and the target value can be inversely normalized to obtain the predicted continuous value. Since the value is randomly selected, the obtained continuous value is a probabilistic prediction P S+1:H =[p S+1 , p S+2 ,…,p S+H ], the present application can output point prediction by taking average of multiple samples, and finally output the predicted traffic utilization of the traffic utilization data to be predicted (i.e., the future prediction time series of BRAS) x S+1:H =[x S+1 , x S+2 , …, x S+H ].

[0082] This application converts the regression problem into a classification problem and outputs a predicted probability distribution, which can better assess the likelihood of future events.

[0083] Figure 2 is a schematic diagram of an application scenario of the flow utilization prediction method provided by this application, such as Figure 2 As shown in FIG. 1 , in one embodiment, the processing of the traffic utilization prediction method may include three parts: pre-processing, large language model, and post-processing. In the pre-processing, the pre-trained word embedding may be obtained first, and the original text vocabulary may be reduced to a time series vocabulary through a learnable linear layer to obtain a time series word embedding. Then, the input time series data of length S may be decomposed into trend X g (t), seasonality X s (t), Holidays X h (t), regression term X r (t) and the noise term X ε (t), further, quantize each decomposed item, for example, design V´quantized values ​​id1<… <id V´ , and define the boundary range of the quantized value. According to the time series normalization value, the values ​​falling within each boundary range are discretized into corresponding ID numbers. The quantized value is used as the index id to query the corresponding token vector in the time series vocabulary (five token vectors are obtained), and the token vectors of each component are aggregated (FC) by the linear layer to obtain the vector of the time series feature token. At the same time, the input text information is segmented and word vector mapped to obtain the corresponding token vector. The two token vectors are concatenated and input into the large language model including 28 GLMBlocks for prediction. The output of the large language model is aggregated (FC) by the linear layer to obtain a token vector Z, and then the probability of each quantized value is calculated by softmax. After that, the quantized value corresponding to the maximum probability of softmax is selected as the predicted discrete value of the current point, and it is mapped back to the actual value by dequantization to obtain the predicted time series data of length H.

[0084] The flow utilization prediction device provided in the present application is described below. The flow utilization prediction device described below and the flow utilization prediction method described above can be referenced to each other.

[0085] Furthermore, the present application also provides a flow utilization prediction device.

[0086] The traffic utilization prediction device may also include: an acquisition module, used to obtain the traffic utilization data to be predicted, and the traffic utilization data to be predicted includes historical time series data and text description information; an identification module, used to perform prompt word recognition based on the text description information to obtain text embedding information; a decomposition module, used to perform time series decomposition based on the historical time series data to obtain time series embedding information; an input module, used to merge the text embedding information and the time series embedding information and input them into a large language model to obtain a prediction vector output by the large language model; a determination module, used to determine the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

[0087] In one embodiment, the recognition module is specifically used to: segment the text description information according to the prompt words and then encode them to obtain segmentation information; map the segmentation information to a feature vector of a fixed length to obtain text embedding information.

[0088] In one embodiment, the decomposition module is specifically used to: perform time series decomposition on the historical time series data based on the Prophet model to obtain the decomposed time series data; normalize each time series data item to obtain multiple normalized data; obtain the embedding vector corresponding to each normalized data; generate time series embedding information based on each embedding vector.

[0089] In one embodiment, the decomposition module includes an acquisition unit, which is used to perform the following steps for each normalized data when respectively acquiring the embedding vector corresponding to each normalized data: determining a target boundary range of the current normalized data from multiple boundary ranges; wherein each boundary range is formed by multiple quantization values; determining an index number of the target boundary range; acquiring the embedding vector corresponding to the index number in a preset time series vocabulary; wherein the preset time series vocabulary includes a one-to-one corresponding association relationship between the index number and the embedding vector.

[0090] In one embodiment, the determination module is specifically used to: map the prediction vector through a fully connected layer to obtain a mapping result; determine the probability of the mapping result being classified into each quantization value through a preset function; determine the target quantization value from each quantization value based on each probability; and determine the predicted traffic utilization of the traffic utilization data to be predicted based on the target quantization value.

[0091] In one embodiment, the determination module includes a determination unit, which is used to: determine the boundary range corresponding to the target quantization value; randomly select a target numerical value from the boundary range corresponding to the target quantization value; inverse normalize the target numerical value to obtain a continuous numerical value; and determine the predicted traffic utilization of the traffic utilization data to be predicted based on the continuous numerical value.

[0092] The traffic utilization prediction device provided in the embodiment of the present application, after obtaining the traffic utilization data to be predicted including historical time series data and text description information, obtains text embedding information by performing prompt word recognition based on the text description information, and obtains time series embedding information by performing time series decomposition based on the historical time series data, so as to adapt to the processing format of the large language model, and directly aligns the text embedding information and the time series embedding information to the large language model, which can improve the reasoning ability of the large model, and can use the large language model to learn more complex and abstract time series patterns and features to improve the expression ability of the model, and avoid the time and labor-intensive training of the large language model. Therefore, an accurate prediction vector output by the large language model can be obtained, and then based on the prediction vector, the predicted traffic utilization of the traffic utilization data to be predicted can be quickly and accurately determined, thereby improving the prediction efficiency of the traffic utilization.

[0093] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the traffic utilization prediction method, which includes: obtaining traffic utilization data to be predicted, the traffic utilization data to be predicted includes historical time series data and text description information; performing prompt word recognition based on the text description information to obtain text embedding information; performing time series decomposition based on the historical time series data to obtain time series embedding information; merging the text embedding information and the time series embedding information and inputting them into a large language model to obtain a prediction vector output by the large language model; and determining the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

[0094] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0095] On the other hand, the present application also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the traffic utilization prediction method provided by the above methods, and the method includes: obtaining traffic utilization data to be predicted, and the traffic utilization data to be predicted includes historical time series data and text description information; performing prompt word recognition based on the text description information to obtain text embedding information; performing time series decomposition based on the historical time series data to obtain time series embedding information; merging the text embedding information and the time series embedding information and inputting them into a large language model to obtain a prediction vector output by the large language model; and determining the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

[0096] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the traffic utilization prediction method provided by the above-mentioned methods, the method comprising: obtaining traffic utilization data to be predicted, the traffic utilization data to be predicted comprising historical time series data and text description information; performing prompt word recognition based on the text description information to obtain text embedding information; performing time series decomposition based on the historical time series data to obtain time series embedding information; merging the text embedding information and the time series embedding information and inputting them into a large language model to obtain a prediction vector output by the large language model; and determining the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

[0097] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.< / time> < / trend> < / entropy> < / t> < / var> < / mean> < / max> < / min> < / y> < / x>

Claims

1. A method for predicting flow utilization, characterized in that: The method comprises: Acquire traffic utilization data to be predicted, wherein the traffic utilization data to be predicted includes historical time series data and text description information; Perform prompt word recognition based on the text description information to obtain text embedding information; Performing time series decomposition based on the historical time series data to obtain time series embedding information; Combining the text embedding information and the time series embedding information and inputting them into a large language model to obtain a prediction vector output by the large language model; The predicted traffic utilization rate of the traffic utilization rate data to be predicted is determined based on the prediction vector.

2. The method for predicting flow rate utilization according to claim 1, characterized in that: The performing time series decomposition based on the historical time series data to obtain time series embedding information includes: Performing time series decomposition on the historical time series data based on the Prophet model to obtain the decomposed time series data; Each time series data is normalized separately to obtain multiple normalized data; Get the embedding vectors corresponding to each normalized data respectively; Temporal embedding information is generated based on each embedding vector.

3. The method for predicting flow rate utilization according to claim 2, characterized in that: When obtaining the embedding vectors corresponding to each normalized data, the following steps are performed for each normalized data: Determining a target boundary range of current normalized data from a plurality of boundary ranges; wherein each boundary range is formed by a plurality of quantized values; Determine the index number of the target boundary range; Obtain an embedding vector corresponding to the index number in a preset time-series vocabulary; wherein the preset time-series vocabulary includes a one-to-one corresponding association relationship between the index number and the embedding vector.

4. The method for predicting flow rate utilization according to claim 3, characterized in that: The preset time series vocabulary is obtained by performing dimensionality reduction processing on a pre-trained vocabulary through a learnable linear layer.

5. The method for predicting flow rate utilization according to claim 3, characterized in that: The determining the predicted traffic utilization rate of the traffic utilization rate data to be predicted based on the prediction vector includes: Mapping the prediction vector through a fully connected layer to obtain a mapping result; Determining the probability of the mapping results being classified into each quantized value respectively by a preset function; determining a target quantized value from each quantized value based on each probability; The predicted traffic utilization rate of the traffic utilization rate data to be predicted is determined based on the target quantized value.

6. The method for predicting flow rate utilization according to claim 5, characterized in that: The determining the predicted traffic utilization rate of the traffic utilization rate data to be predicted based on the target quantized value includes: Determine a boundary range corresponding to the target quantized value; Randomly select a target value from the boundary range corresponding to the target quantized value; Performing inverse normalization on the target value to obtain a continuous value; The predicted traffic utilization rate of the traffic utilization rate data to be predicted is determined based on the continuous numerical values.

7. The method for predicting flow utilization according to any one of claims 1 to 6, characterized in that: The step of performing prompt word recognition based on the text description information to obtain text embedding information includes: The text description information is segmented by prompt words and then encoded to obtain segmentation information; The word segmentation information is mapped to a feature vector of a fixed length to obtain text embedding information.

8. A flow utilization rate prediction device, characterized in that: include: An acquisition module, used to acquire traffic utilization data to be predicted, wherein the traffic utilization data to be predicted includes historical time series data and text description information; A recognition module, used for performing prompt word recognition based on the text description information to obtain text embedding information; A decomposition module, used for performing time series decomposition based on the historical time series data to obtain time series embedding information; An input module, used for combining the text embedding information and the time series embedding information and inputting them into a large language model to obtain a prediction vector output by the large language model; A determination module is used to determine the predicted traffic utilization of the traffic utilization data to be predicted based on the prediction vector.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the traffic utilization prediction method according to any one of claims 1 to 7 is implemented.

10. A medium, the medium being a non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by a processor, the traffic utilization prediction method according to any one of claims 1 to 7 is implemented.

11. A product, the product being a computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the traffic utilization prediction method according to any one of claims 1 to 7 is implemented.

Images