Large model fine-tuning method, time series prediction method, device and storage medium

Abstract

The present application discloses a large model fine-tuning method, a time series prediction method, an apparatus and a storage medium, the method comprising: reading a pre-trained model in a preset manner; determining the low-rank matrix requirements of the pre-trained model, determining the initial value distribution of the low-rank matrix requirements, and calculating the gamma distribution of the low-rank matrix weights based on the initial value distribution; combining the gamma distribution, the variance of the activation function input and output, and the third-order center distance to derive the weight distribution of the low-rank matrix weights; processing the weight distribution through gamma sampling to obtain the low-rank matrix weights; adjusting the parameters of the pre-trained model based on the low-rank matrix weights to obtain the adjusted pre-trained model; training the adjusted pre-trained model to obtain a target model, wherein the target model is used to predict events associated with the time series. Through this application, the problems of high hardware cost and energy consumption and low availability required by large models in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the fields of machine learning and deep learning, and specifically to a large model fine-tuning method, a time series prediction method, a device, and a storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI), large pre-trained models (such as BERT, GPT, and T5) have achieved remarkable results in fields such as natural language processing and computer vision. These large models, pre-trained on massive amounts of data, have acquired powerful semantic understanding capabilities and are adaptable to a variety of downstream tasks. However, as the scale of these models continues to expand, effectively migrating these large models to specific tasks and fine-tuning them has become a pressing issue.

[0003] Traditional fine-tuning methods optimize parameters by training the entire model or part of the model on a specific task dataset. Although this can improve task performance, it also has some problems: on the one hand, large models require large-scale computing resources and storage space, resulting in high hardware costs and energy consumption; on the other hand, large models usually contain billions of parameters. The fine-tuning process of large models is a complex and time-consuming process that requires a large amount of data and computing resources, making the availability of large models low. Summary of the Invention

[0004] The main purpose of this application is to provide a large model fine-tuning method, a time series prediction method, a device and a storage medium to solve the problems in related technologies of high hardware cost and energy consumption and low availability required by large models.

[0005] In order to achieve the above-mentioned purpose, according to the first aspect of the present application, a large model fine-tuning method is provided. The method includes: reading a pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform; determining the low-rank matrix requirements of the pre-trained model, determining the initial value distribution of the low-rank matrix requirements, and calculating the gamma distribution of the low-rank matrix weights based on the initial value distribution; combining the gamma distribution, the variance of the activation function input and output, and the third-order center distance to derive the weight distribution of the low-rank matrix weights; processing the weight distribution through gamma sampling to obtain the low-rank matrix weights; adjusting the parameters of the pre-trained model based on the low-rank matrix weights to obtain the adjusted pre-trained model; training the adjusted pre-trained model to obtain a target model, wherein the target model is used to predict events associated with time series.

[0006] Furthermore, the low-rank matrix requirements of the pre-trained model are determined, including: determining the dimension information of the pre-trained model and the proportion of fine-tuning parameters; determining the matrix rank based on the dimension information and the proportion of fine-tuning parameters, and determining the low-rank matrix requirements based on the matrix rank.

[0007] Furthermore, the initial value distribution of the low-rank matrix requirement is determined, including: determining the input data at a preset time in the loop structure of the pre-trained model; when the input data is a skewed distribution with a small skewness, determining the third-order center distance of the input data; determining the skewed distribution probability density function based on the third-order center distance, and using the skewed distribution probability density function as the initial value distribution of the low-rank matrix requirement.

[0008] Furthermore, the pre-trained model is adjusted based on the low-rank matrix weight to obtain the adjusted pre-trained model, including: obtaining the model weight of the pre-trained model; adding the model weight to the low-rank matrix weight to adjust the pre-trained model to obtain the adjusted pre-trained model.

[0009] Furthermore, the pre-trained model after parameter adjustment is trained to obtain a target model, including: calculating the average number of activated neurons of the pre-trained model after parameter adjustment at a preset time step; and combining the regularization term coefficient and the loss function to construct a Lagrangian function, and minimizing the Lagrangian function to obtain an optimized function; based on the optimized function, the pre-trained model after parameter adjustment is trained to obtain a target model, wherein the target model is used to predict events associated with time series.

[0010] Furthermore, the pre-trained model after parameter adjustment is trained based on the optimized function to obtain a target model, including: calculating the parameter weights of the pre-trained model after parameter adjustment by the optimized function; normalizing the gradient of the parameter weights according to the gradient normalization method to obtain the target model, wherein the target model is used to predict events associated with the time series.

[0011] According to the second aspect of the present application, a time series prediction method is also provided, including: obtaining historical time feature data from a target data set, wherein the historical time feature data at least includes: meteorological time feature data; inputting the historical time feature data into a target model to obtain data feature relationships between different time points, wherein the target model is obtained by the above-mentioned large model fine-tuning method; and predicting data features in the future time based on the data feature relationships.

[0012] In order to achieve the above-mentioned purpose, according to the third aspect of the present application, a large model fine-tuning device is provided. The device includes: a reading unit for reading a pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform; a determination unit for determining the low-rank matrix requirements of the pre-trained model, determining the initial value distribution of the low-rank matrix requirements, and calculating the gamma distribution of the low-rank matrix weights based on the initial value distribution; combining the gamma distribution, the variance of the activation function input and output, and the third-order center distance to derive the weight distribution of the low-rank matrix weights; processing the weight distribution through gamma sampling to obtain the low-rank matrix weights; a parameter adjustment unit for adjusting the parameters of the pre-trained model based on the low-rank matrix weights to obtain the pre-trained model after parameter adjustment; a training unit for training the pre-trained model after parameter adjustment to obtain a target model, wherein the target model is used to predict events associated with time series.

[0013] Furthermore, the determination unit includes: a first determination module, used to determine the dimension information of the pre-trained model and the proportion of fine-tuning parameters; a second determination module, used to determine the matrix rank based on the dimension information and the proportion of fine-tuning parameters, and determine the low-rank matrix requirement based on the matrix rank.

[0014] Furthermore, the determination unit includes: a third determination module, used to determine the input data at a preset moment in the loop structure of the pre-trained model; a fourth determination module, used to determine the third-order center distance of the input data when the input data is a skewed distribution with a small skewness; a fifth determination module, used to determine the skewed distribution probability density function based on the third-order center distance, and use the skewed distribution probability density function as the initial value distribution required for the low-rank matrix.

[0015] Furthermore, the parameter adjustment unit includes: an acquisition module for obtaining the model weights of the pre-trained model; and a superposition module for adding the model weights to the low-rank matrix weights to adjust the parameters of the pre-trained model to obtain the pre-trained model after parameter adjustment.

[0016] Furthermore, it includes: a calculation module, which is used to calculate the average number of activated neurons of the pre-trained model after parameter adjustment at a preset time step; and a construction module, which is used to combine the regularization term coefficient and the loss function to construct a Lagrangian function, and minimize the Lagrangian function to obtain an optimized function; a training module, which is used to train the pre-trained model after parameter adjustment based on the optimized function to obtain a target model, wherein the target model is used to predict events associated with time series.

[0017] Furthermore, the training module includes: a calculation submodule for calculating the parameter weights of the pre-trained model after parameter adjustment based on the optimized function; a processing submodule for normalizing the gradient of the parameter weights according to the gradient normalization method to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0018] According to a fourth aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned large model fine-tuning methods is implemented.

[0019] According to a fifth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a large model fine-tuning method according to any one of the above items is implemented.

[0020] Through this application, the following steps are adopted: reading the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform; determining the low-rank matrix requirements of the pre-trained model, determining the initial value distribution of the low-rank matrix requirements, and determining the low-rank matrix weights based on the initial value distribution; adjusting the parameters of the pre-trained model based on the low-rank matrix weights to obtain the pre-trained model after parameter adjustment; training the pre-trained model after parameter adjustment to obtain a target model, wherein the target model is used to predict events associated with a time series. In this application, the weight matrix is initialized by sampling with a gamma distribution, so that the model can converge faster, and the pre-trained model is adjusted based on the low-rank matrix weights, and the pre-trained model after parameter adjustment is trained; in this way, the fine-tuning process of the large model is quickly completed with fewer parameters, thereby reducing the complexity of the fine-tuning process of the large model and saving the time of the fine-tuning process; thereby reducing the required amount of data and computing resources, making the large model more available, and reducing hardware costs and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0022] Figure 1 This is a flow chart of a large model fine-tuning method provided in accordance with an embodiment of the present application;

[0023] Figure 2 Schematic diagram of activation probability and gradient explosion according to an embodiment of the present application;

[0024] Figure 3Schematic diagram of the changes before and after gradient normalization of the large model fine-tuning method provided in an embodiment of the present application;

[0025] Figure 4 Schematic diagram of a framework of a large model fine-tuning method provided in accordance with an embodiment of the present application;

[0026] Figure 5 is a flowchart of a time series prediction method provided according to an embodiment of the present application;

[0027] Figure 6 is a schematic diagram of a large model fine-tuning device provided according to an embodiment of the present application;

[0028] Figure 7 Schematic diagram of the network architecture of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] According to an embodiment of the present application, a large model fine-tuning method is provided.

[0033] Figure 1 This is a flow chart of a large model fine-tuning method according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0034] Step S101: Read the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform.

[0035] Exemplary methods for users to load large models include: 1) reading the model through the Hugging Face API; 2) downloading the pre-trained model locally and loading it directly without the API; and 3) using the API to access the large model using cloud computing services. This embodiment of the present invention reads the pre-trained model using the user-selected method and loads it onto the GPU (Graphics Processing Unit), enabling flexible selection based on actual needs and scenarios.

[0036] Step S102: determine the low-rank matrix requirements of the pre-trained model, determine the initial value distribution of the low-rank matrix requirements, and determine the low-rank matrix weights based on the initial value distribution.

[0037] Specifically, determining the low-rank matrix requirements of the pre-trained model can be achieved through the following steps: determining the dimension information of the pre-trained model and the proportion of fine-tuning parameters; determining the matrix rank based on the dimension information and the proportion of fine-tuning parameters, and determining the low-rank matrix requirements based on the matrix rank.

[0038] For example, if d is the dimension of the pre-trained model, γ is the rank of the low-rank matrix, and the proportion of fine-tuning parameters is ρ, the following relationship is satisfied:

[0039] ;

[0040] That is, the matrix rank of the embodiment of the present invention can be obtained according to the above formula, and the size requirements of the low-rank matrices A and B can be determined based on the matrix rank.

[0041] Among them, the initial value distribution required for the low-rank matrix can be obtained by the following steps: determining the input data at a preset time in the loop structure of the pre-trained model; when the input data is a skewed distribution with a small skewness, determining the third-order center distance of the input data; determining the skewed distribution probability density function based on the third-order center distance, and using the skewed distribution probability density function as the initial value distribution required for the low-rank matrix.

[0042] For example, in tasks that require strong temporal dependencies and sequence modeling, the large model is based on a hybrid architecture that combines the advantages of Transformer (a neural network model with a self-attention mechanism) and RNN (Recurrent Neural Network). The input data of a node at a certain moment in the loop structure It can be expressed by the following formula:

[0043]

[0044] Among them, RNN has input dimension, intermediate hidden layer dimension, and output dimension. i represents the input dimension, h represents the dimension of the intermediate hidden layer, I represents the number of neurons from the input layer to the hidden layer, H represents the number of neurons between the hidden layers, and t represents the current time. for ; , which is the output of the neuron. Indicates that each neuron has unit. is the input data at the current time t, It represents the weight between the input and the neuron when the input dimension is i and the dimension of the intermediate hidden layer is h. It represents the weight of the connection between neurons in the network when the input dimension is h and the dimension of the intermediate hidden layer is h. is the input size of the model, is the hidden layer size of the model.

[0045] Assume that the input data are all skewed distribution with small skewness. Consider the third-order central moment of the data, and discuss the possibility of weight distribution based on the consistency of input and output distribution of activation function. Assume that in the loop structure and Independent and identically distributed, and is not independent and identically distributed, so the third-order center distance in the above formula is It can be expressed as the following formula:

[0046]

[0047] in, is the input data at the current time t, It represents the weight between the input and the neuron when the input dimension is i and the dimension of the intermediate hidden layer is h. It represents the weight of the connection between neurons in the network when the input dimension is h and the dimension of the intermediate hidden layer is h. The function is the third-order center distance of the parameter, The neuron output at the previous moment, is the input size of the model, is the hidden layer size of the model, is the neuron input at time t, They represent the third-order origin moments of the corresponding parameters respectively.

[0048] In the above formula, the output of the neural network activation function is , the activation function input is , the activation function is the Relu function, and the skewed distribution probability density function can be obtained as shown in the formula:

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] in, )for The third-order origin moment of is the output at the previous moment, is the input at the current moment, for The position parameters, for The scale parameter, for The shape parameters, for The skewness coefficient, is the standard normal probability density function of the skewed distribution, is the cumulative distribution function, and erf is the error function.

[0055] The embodiment of the present invention can capture data asymmetry and enhance model flexibility by using the skewed distribution probability density function as the initial value distribution required by the low-rank matrix.

[0056] Among them, the low-rank matrix weights are determined according to the initial value distribution, which can be obtained through the following steps: calculating the gamma distribution of the low-rank matrix weights according to the initial value distribution; combining the gamma distribution, the variance of the activation function input and output, and the third-order center distance to derive the weight distribution of the low-rank matrix weights; processing the weight distribution through gamma sampling to obtain the low-rank matrix weights.

[0057] For example, since the output distribution after the activation function will have positive skewness, in order to balance this situation, the embodiment of the present invention adopts a probability density function of the weight distribution based on the gamma distribution with negative skewness. The gamma distribution is a continuous probability distribution density with two parameters (shape parameter α and inverse scale parameter β). The random variable X that satisfies the gamma distribution can be expressed as:

[0058] X~Γ(α,β)≡Gamma(α,β)

[0059] Where Γ(α) is the gamma function. The general formula for the variance and center distance of the gamma function is as follows:

[0060]

[0061] The derivation is based on the premise that the variance of the activation function input and output is consistent with the third-order center distance, and then the obtained function is left-shifted by the mean length so that the distribution mean is 0. The distribution of the weights is shown in Table 1:

[0062] Table 1: Gamma distribution of RNN model weights under different activation functions

[0063]

[0064] in, is the input size of the model, is the hidden layer size of the model, is the output size of the model.

[0065] The above weight distribution is sampled by Gamma to obtain low-rank weights A and B.

[0066] Step S103: Adjust the parameters of the pre-trained model based on the low-rank matrix weights to obtain the adjusted pre-trained model.

[0067] The weight information provided by the low-rank matrix is used to optimize the relevant parameters in the pre-trained model to better meet the needs or goals. After this parameter adjustment process, a pre-trained model with adjusted parameters can be obtained, which will have improved performance or accuracy.

[0068] Specifically, the pre-trained model is adjusted based on the low-rank matrix weight to obtain the adjusted pre-trained model, which can be obtained by the following steps: obtaining the model weight of the pre-trained model; adding the model weight to the low-rank matrix weight to adjust the pre-trained model to obtain the adjusted pre-trained model.

[0069] For example, the above weight distribution is sampled by Gamma to obtain low-rank weights A and B, and then compared with the pre-trained large model weights Add up to get the new pre-training model parameters, the model parameters of the pre-training model after specific adjustment The formula is as follows:

[0070]

[0071] The embodiment of the present invention adjusts the parameters by adding the model weights of the pre-trained model to the low-rank matrix weights, which can reduce the model complexity, accelerate the training process, reduce the computing cost, and maintain the model performance.

[0072] Step S104: train the pre-trained model after parameter adjustment to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0073] Specifically, the pre-trained model after parameter adjustment is trained to obtain a target model. This is achieved by: calculating the average number of activated neurons in the pre-trained model at a preset time step; combining the regularization term coefficient with the loss function to construct a Lagrangian function, and minimizing the Lagrangian function to obtain an optimized function; and then training the pre-trained model after parameter adjustment based on the optimized function to obtain a target model. The target model is used to predict events related to time series. Time series events include: power consumption forecasting, power transformer oil temperature forecasting, weather forecasting, and influenza incidence forecasting. In a specific example, taking power consumption forecasting as an example, the historical data types used in training the target model include: high effective load, high useless load, intermediate effective load, intermediate useless load, low effective load, low useless load, and oil temperature. Historical data refers to a period of time connected to the prediction date, and the time span can be set by the user. For example, if electricity consumption needs to be predicted for the next week, the historical data can include data from the previous week, month, or year. During the target model inference process, the inputs are based on the task. The known load and target oil temperature are fed into the model for training. Subsequently, new load inputs are used to predict the target oil temperature. When using the target model for electricity forecasting, the input data is first normalized or standardized to ensure consistent distributions of several feature values across different dimensions. Then, the data is proportionally divided into training and test sets. Each step results in weightings for different data points in the model (i.e., features found to have the greatest impact on the results are assigned greater weights during model training). Using the features and target values in the training set, the target model determines the correlation between the features and the target values, as well as the weights of different features on the target values. When a prediction is needed, the collected feature values are fed into the model, and the model outputs a predicted target value based on the correlations between the features and the target values. For electricity usage duration prediction, users can choose a timeframe based on their task requirements, such as one day, one week, or one month. In this way, by predicting electricity consumption through the target model, users can input the model according to the load and oil temperature of the transformer in the historical period to obtain a trained target model, and then input the load in real time to monitor the predicted value of the oil temperature. When the predicted value is higher than the warning value, an alarm will be triggered to prevent the transformer from burning due to high load operation.

[0074] For example, by observing the activation probability of the neural unit of the pre-trained model after parameter adjustment within 28 time steps, as shown in Figure 2As shown in Figures (a) and (b), we can see the chain reaction of gradient explosion. As the training process progresses, there is a one-to-one correspondence between the sudden increase and decrease in the probability of neuron activation and the change in the Loss function. Figure 2 In Figure (a), curve 201 represents the activation probability of the neuron at different iteration numbers when the time step is 4; curve 202 represents the activation probability of the neuron at different iteration numbers when the time step is 14; curve 203 represents the activation probability of the neuron at different iteration numbers when the time step is 27; curve 204 represents the activation probability of the neuron at different iteration numbers when the time step is 13; curve 205 represents the activation probability of the neuron at different iteration numbers when the time step is 23; curve 206 represents the activation probability of the neuron at different iteration numbers when the time step is 10; curve 207 represents the activation probability of the neuron at different iteration numbers when the time step is 6. Figure 2 The curves in Figure (b) represent the corresponding loss values at different iteration times. By incorporating the regularization term into the loss function, we can reconstruct the loss function as a Lagrangian dual problem. This constrains the number of activated neurons to prevent them from being over-activated or over-inhibited, thereby guiding the network to maintain a more balanced activation pattern. The specific steps are as follows:

[0075] Calculate the number of activated neurons under different time_step (time step), set is the number of activated neurons under the nth time_step, taking its mean As shown below:

[0076]

[0077] After that, the regularization term is combined with the loss function to construct the Lagrangian function , as follows:

[0078]

[0079] in, is the regularization coefficient, the regularization term The minimum value of 2 , Represents the average number of activated neurons in the i-th iteration.

[0080] When the constraints are met, the Lagrangian function, that is, the total loss function, is minimized to obtain the following , the formula is:

[0081]

[0082] Where min represents the minimum value of the total loss function, and max represents the maximum value of the total loss function. Specifically, training the pre-trained model after parameter adjustment based on the optimized function to obtain a target model includes: calculating the parameter weights of the pre-trained model after parameter adjustment based on the optimized function; normalizing the gradients of the parameter weights according to the gradient normalization method to obtain the target model, wherein the target model is used to predict events associated with the time series.

[0083] The specific implementation of the gradient normalization method (WGN) of the target model in the embodiment of the present invention is as follows:

[0084]

[0085] in, is the parameter weight of the pre-trained model after parameter adjustment, is the weight gradient, is the mean of the weight gradient, is the variance of the weight gradient, is the deviation correction value, is the scaling factor.

[0086] In the above formula, the gradients of each weight are normalized, and the mean and variance of the gradients during each training session are normalized. The contribution of this normalization is controlled by parameters, making WGN applicable to most datasets. During experiments, different values of η were set for different datasets. By adjusting the value of η, the range of loss can be controlled, effectively and stably accelerating the model's convergence. Experiments have shown that the most effective range of η is between 0.005 and 0.0005. The parameter δ is used to maintain consistency in the weight variance and is taken to be a minimum value close to 0.

[0087] The gradient update method of WGN combines the first-order moment estimation and second-order moment estimation of the gradient, so that the gradients of each dimension of a parameter can share each other's information, and adjust the direction and span of the gradient so that the mean of the gradient is basically close to 0 and the variance is close to 1. In the parameter space, the WGN method not only considers the information of a single point, but also counts the information of the surrounding points, searches in a more robust way, and increases the adaptability to complex structures in the parameter space. With the forward update and back propagation of the gradient flow, the WGN method will continue to normalize the gradient to ensure that the gradient changes smoothly throughout the training process, and the weights will also increase steadily. The specific changes before and after the gradient normalization can be referred to. Figure 3 As shown. Among them, Figure 3Figures (a) and (c) represent the distribution of gradients before normalization and the distribution of gradients after normalization, respectively; in Figure (a), the dotted line 31 represents the standard deviation, and the dotted line 32 represents the average value; in Figure (c), the dotted line 33 represents the standard deviation, and the dotted line 34 represents the average value. Figure 3 Figures (b) and (d) represent the heat maps of the weight gradient before and after normalization, respectively. The horizontal and vertical axes represent the number of occurrences. The colors of the vertical bars on the right represent the colors of different values, and each grid represents the number of times a certain value appears.

[0088] Optionally, Figure 4 Schematic diagram of the framework of the large model fine-tuning method according to an embodiment of the present invention. Figure 4 As shown, the large model reading module includes a reading mode selection submodule and a reading submodule.

[0089] Low-rank branch construction module: includes a fine-tuning parameter requirement submodule, an initialization weight receiving submodule and a low-rank weight construction module. Among them, the fine-tuning parameter requirement submodule: determines the number of fine-tuning parameters based on the input data and the task type performed by the model, and after determining the matrix size requirements of the low-rank matrices A and B, passes them to the initialization module.

[0090] Initialization module: includes a low-rank matrix demand acceptance submodule and a low-rank weight initialization submodule. After constructing the low-rank matrix weight initialization distribution above, the initialization scheme is returned to the low-rank construction submodule for low-rank matrix weight initialization.

[0091] Model training module: includes loss function creation submodule and gradient normalization submodule.

[0092] In summary, the large model fine-tuning method provided in the embodiment of the present application reads the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of the cloud computing service platform; determines the low-rank matrix requirements of the pre-trained model, determines the initial value distribution of the low-rank matrix requirements, and determines the low-rank matrix weights based on the initial value distribution; adjusts the parameters of the pre-trained model based on the low-rank matrix weights to obtain the pre-trained model after parameter adjustment; trains the pre-trained model after parameter adjustment to obtain a target model, wherein the target model is used to predict events associated with time series. Through this application, the problems of poor model convergence performance and poor training effect of the fine-tuning model in the related art are solved. By initializing the weight matrix with gamma distribution sampling, the model can converge faster, and the parameters of the pre-trained model are adjusted based on the low-rank matrix weights, and the pre-trained model after parameter adjustment is trained, thereby effectively improving the training effect of the fine-tuning model.

[0093] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0094] According to another embodiment of the present application, a time series prediction method is also provided. Figure 5 As shown, the method includes the following steps:

[0095] S501: Acquire historical time characteristic data from a target data set, wherein the historical time characteristic data at least includes: meteorological time characteristic data.

[0096] The target dataset can be a weather forecast dataset or a power consumption forecast dataset, and the historical time feature data in the dataset is obtained. The power consumption forecast dataset includes power data such as high payload, high dead load, intermediate payload, intermediate dead load, low payload, low dead load, oil temperature, and power transformer oil temperature forecast.

[0097] Take the meteorological time feature data in the weather forecast dataset as an example:

[0098] The following are the 21 meteorological time feature data in this dataset:

[0099] 1. Temperature

[0100] 2.DewPointTemperature

[0101] 3.Relative Humidity

[0102] 4. WindSpeed

[0103] 5. WindGust - Wind gust speed

[0104] 6. WindDirection

[0105] 7.Solar Radiation

[0106] 8.Visibility

[0107] 9. CloudCover - Cloud Cover

[0108] 10. Precipitation

[0109] 11. Pressure

[0110] 12.SeaLevelPressure - Sea level pressure

[0111] 13.Rainfall - Amount of precipitation (sometimes a separate feature)

[0112] 14. Snowfall

[0113] 15.SnowDepth

[0114] 16.Condition - weather conditions (such as sunny, cloudy, rainy, etc.)

[0115] 17.TimeofDay - time of day (e.g., morning, noon, evening)

[0116] 18.DayofWeek - Day of the week

[0117] 19.Month

[0118] 20. Season - Season (such as spring, summer, etc.)

[0119] 21.HourofDay – Hour of Day (often used for event analysis related to time series)

[0120] S502: Input the historical time feature data into the target model to obtain the data feature relationship between different time points, wherein the target model is obtained through the above steps S101 to S104.

[0121] For example, a meteorological data source captures time-series event data containing features such as temperature, humidity, wind speed, and precipitation. The extracted feature vectors are fed into the target model, which is then trained using historical meteorological data. During training, the model learns patterns, trends, and cyclical characteristics in the data, enabling it to predict meteorological data for future points in time. The predictions reveal relationships between data features at different time points, such as daily temperature trends and cyclical fluctuations in humidity.

[0122] S503: Predicting data features in the future based on the data feature relationship.

[0123] Data feature relationships refer to the inherent connections or patterns between data points discovered through early analysis of historical data. These relationships may reveal trends over time, the interactions between different variables, or cyclical patterns in the data. For example, based on data feature relationships, it is possible to predict daily temperatures for the next month or the temperature range for the same period next year.

[0124] In summary, the time series prediction method provided in the embodiments of the present application obtains historical time feature data from a target dataset, wherein the historical time feature data includes at least meteorological time feature data; inputs the historical time feature data into a target model to obtain data feature relationships between different time points, wherein the target model is obtained through the large model fine-tuning method such as steps S101 to S104; and predicts data features in the future based on the data feature relationships. That is, the method of predicting data features in the future based on known data feature relationships in the embodiments of the present invention brings significant benefits in many aspects, including improving prediction accuracy, enhancing decision support, and optimizing resource allocation.

[0125] The embodiment of the present application also provides a large model fine-tuning device. It should be noted that the large model fine-tuning device of the embodiment of the present application can be used to execute the large model fine-tuning method provided in the embodiment of the present application. The large model fine-tuning device provided in the embodiment of the present application is introduced below.

[0126] Figure 6 Schematic diagram of a large model fine-tuning device 600 according to an embodiment of the present application. Figure 6 As shown, the device includes: a reading unit 601, a determination unit 602, a parameter adjustment unit 603, and a training unit 604.

[0127] Specifically, the reading unit 601 is used to read the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform;

[0128] A determination unit 602 is used to determine the low-rank matrix requirements of the pre-trained model, determine the initial value distribution of the low-rank matrix requirements, and calculate the gamma distribution of the low-rank matrix weights based on the initial value distribution; combine the gamma distribution, the variance of the activation function input and output, and the third-order center distance to derive the weight distribution of the low-rank matrix weights; and process the weight distribution through gamma sampling to obtain the low-rank matrix weights;

[0129] A parameter adjustment unit 603 is used to adjust the parameters of the pre-trained model based on the low-rank matrix weight to obtain a pre-trained model after parameter adjustment;

[0130] The training unit 604 is used to train the pre-trained model after parameter adjustment to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0131] In summary, the large model fine-tuning device provided in the embodiment of the present application is used to read the pre-trained model in a preset manner through a reading unit 601, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform; a determination unit 602 is used to determine the low-rank matrix requirements of the pre-trained model, determine the initial value distribution of the low-rank matrix requirements, and calculate the gamma distribution of the low-rank matrix weights based on the initial value distribution; the gamma distribution, the variance of the activation function input and output, and the third-order center distance are combined to derive the weight distribution of the low-rank matrix weights; the weight distribution is processed by gamma sampling to obtain the low-rank matrix weights; a parameter adjustment unit 603 is used to adjust the parameters of the pre-trained model based on the low-rank matrix weights to obtain the pre-trained model after parameter adjustment; a training unit 604 is used to train the pre-trained model after parameter adjustment to obtain a target model, which solves the problems of poor model convergence performance and poor training effect of the fine-tuning model in related technologies. By initializing the weight matrix with gamma distribution sampling, the model can converge faster, and the pre-trained model is adjusted based on the low-rank matrix weights, and the pre-trained model after parameter adjustment is trained, thereby effectively improving the training effect of the fine-tuning model.

[0132] Optionally, in the large model fine-tuning device provided in the embodiment of the present application, the determination unit includes: a first determination module, used to determine the dimension information of the pre-trained model and the proportion of fine-tuning parameters; a second determination module, used to determine the matrix rank based on the dimension information and the proportion of fine-tuning parameters, and determine the low-rank matrix requirement based on the matrix rank.

[0133] Optionally, in the large model fine-tuning device provided in an embodiment of the present application, the determination unit includes: a third determination module, used to determine the input data at a preset time in the loop structure of the pre-trained model; a fourth determination module, used to determine the third-order center distance of the input data when the input data is a skewed distribution with a small skewness; a fifth determination module, used to determine the skewed distribution probability density function based on the third-order center distance, and use the skewed distribution probability density function as the initial value distribution required for the low-rank matrix.

[0134] Optionally, in the large model fine-tuning device provided in the embodiment of the present application, the parameter adjustment unit includes: an acquisition module for obtaining the model weight of the pre-trained model; and a superposition module for adding the model weight to the low-rank matrix weight to adjust the parameters of the pre-trained model to obtain the pre-trained model after parameter adjustment.

[0135] Optionally, the large model fine-tuning device provided in the embodiment of the present application includes: a calculation module for calculating the average number of activated neurons of the pre-trained model after parameter adjustment at a preset time step; and a construction module for combining the regularization term coefficient and the loss function to construct a Lagrangian function, and minimize the Lagrangian function to obtain an optimized function; a training module for training the pre-trained model after parameter adjustment based on the optimized function to obtain a target model, wherein the target model is used to predict events associated with time series.

[0136] Optionally, in the large model fine-tuning device provided in the embodiment of the present application, the training module includes: a calculation submodule, which is used to calculate the parameter weights of the pre-trained model after parameter adjustment based on the optimized function; and a processing submodule, which is used to normalize the gradient of the parameter weights according to the gradient normalization method to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0137] The large model fine-tuning device includes a processor and a memory. The above-mentioned reading unit 601, determination unit 602, parameter adjustment unit 603, training unit 604, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0138] The processor contains a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and large models can be fine-tuned by adjusting kernel parameters.

[0139] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0140] In an exemplary embodiment of the present application, a computer storage medium capable of implementing the above method is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification, such as the following steps can be executed: reading the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform; determining the low-rank matrix requirements of the pre-trained model, determining the initial value distribution of the low-rank matrix requirements, and calculating the gamma distribution of the low-rank matrix weights based on the initial value distribution; combining the gamma distribution, the variance of the activation function input and output, and the third-order center distance to derive the weight distribution of the low-rank matrix weights; processing the weight distribution through gamma sampling to obtain the low-rank matrix weights; adjusting the parameters of the pre-trained model based on the low-rank matrix weights to obtain the pre-trained model after parameter adjustment; training the pre-trained model after parameter adjustment to obtain a target model, wherein the target model is used to predict events associated with time series.

[0141] In an optional implementation: determining the dimension information of the pre-trained model and the proportion of fine-tuning parameters; determining the matrix rank based on the dimension information and the proportion of fine-tuning parameters, and determining the low-rank matrix requirement based on the matrix rank.

[0142] In an optional embodiment: determining the input data at a preset time in the loop structure of the pre-trained model; determining the third-order center distance of the input data when the input data is a skewed distribution with a small skewness; determining the skewed distribution probability density function based on the third-order center distance, and using the skewed distribution probability density function as the initial value distribution required for the low-rank matrix.

[0143] In an optional implementation: obtaining the model weight of the pre-trained model; adding the model weight to the low-rank matrix weight to adjust the parameters of the pre-trained model to obtain the adjusted pre-trained model.

[0144] In an optional embodiment: the average number of activated neurons of the pre-trained model after parameter adjustment at a preset time step is calculated; and the regularization term coefficient and the loss function are combined to construct a Lagrangian function, and the Lagrangian function is minimized to obtain an optimized function; the pre-trained model after parameter adjustment is trained based on the optimized function to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0145] In an optional embodiment: the parameter weights of the pre-trained model after parameter adjustment are calculated based on the optimized function; the gradient of the parameter weights is normalized according to the gradient normalization method to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0146] In an optional embodiment, the embodiments of the present application may further include a program product for implementing the above method, which may be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0148] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0149] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0150] The program code used to perform the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0151] In addition, in an exemplary embodiment of the present application, an electronic device capable of implementing the above method is also provided.

[0152] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0153] Refer to the following Figure 7 hereinafter, an electronic device 700 according to this embodiment of the present application is described. Figure 7 The electronic device 700 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0154] like Figure 7 As shown, electronic device 700 is implemented as a general-purpose computing device. Components of electronic device 700 may include, but are not limited to, the aforementioned at least one processing unit 710, the aforementioned at least one storage unit 720, a bus 730 connecting various system components (including storage unit 720 and processing unit 710), and a display unit 740.

[0155] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 710, so that the processing unit 710 executes the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present application. For example, the processing unit 710 can execute the following steps: read the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of the cloud computing service platform; determine the low-rank matrix requirements of the pre-trained model, determine the initial value distribution of the low-rank matrix requirements, and calculate the gamma distribution of the low-rank matrix weights based on the initial value distribution; combine the gamma distribution, the variance of the activation function input and output, and the third-order center distance to derive the weight distribution of the low-rank matrix weights; process the weight distribution through gamma sampling to obtain the low-rank matrix weights; adjust the parameters of the pre-trained model based on the low-rank matrix weights to obtain the pre-trained model after parameter adjustment; train the pre-trained model after parameter adjustment to obtain the target model, wherein the target model is used to predict events associated with time series.

[0156] In an optional implementation: determining the dimension information of the pre-trained model and the proportion of fine-tuning parameters; determining the matrix rank based on the dimension information and the proportion of fine-tuning parameters, and determining the low-rank matrix requirement based on the matrix rank.

[0157] In an optional embodiment: determining the input data at a preset time in the loop structure of the pre-trained model; determining the third-order center distance of the input data when the input data is a skewed distribution with a small skewness; determining the skewed distribution probability density function based on the third-order center distance, and using the skewed distribution probability density function as the initial value distribution required for the low-rank matrix.

[0158] In an optional implementation: obtaining the model weight of the pre-trained model; adding the model weight to the low-rank matrix weight to adjust the parameters of the pre-trained model to obtain the adjusted pre-trained model.

[0159] In an optional embodiment: the average number of activated neurons of the pre-trained model after parameter adjustment at a preset time step is calculated; and the regularization term coefficient and the loss function are combined to construct a Lagrangian function, and the Lagrangian function is minimized to obtain an optimized function; the pre-trained model after parameter adjustment is trained based on the optimized function to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0160] In an optional embodiment: the parameter weights of the pre-trained model after parameter adjustment are calculated based on the optimized function; the gradient of the parameter weights is normalized according to the gradient normalization method to obtain a target model, wherein the target model is used to predict events associated with the time series.

[0161] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 7201 and / or a cache memory unit 7202 , and may further include a read-only memory unit (ROM) 7203 .

[0162] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0163] Bus 730 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0164] The electronic device 700 can also communicate with one or more external devices 800 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication can occur via an input / output (I / O) interface 750. Furthermore, the electronic device 700 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 700, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0165] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0166] Furthermore, the above-mentioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It is readily understood that the processes illustrated in the above-mentioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0167] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the claims.

Claims

1. A large model fine-tuning method, characterized in that: include: Reading the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform; Determine the low-rank matrix requirement of the pre-trained model, determine the initial value distribution of the low-rank matrix requirement, and calculate the gamma distribution of the low-rank matrix weights based on the initial value distribution; wherein the low-rank matrix requirement is used to characterize the size of the low-rank matrix; the initial value distribution includes: a skewed distribution probability density function for capturing the asymmetry of the input data of the pre-trained model; the initial value distribution is a skewed distribution of the third-order center distance of the input data associated with the time series, and is used to capture the asymmetry of the input data; The gamma distribution, the variance of the activation function input and output, and the third-order center distance are combined to derive the weight distribution of the low-rank matrix weight; Processing the weight distribution by gamma sampling to obtain a low-rank matrix weight; Adjusting the parameters of the pre-trained model based on the low-rank matrix weights to obtain a pre-trained model after parameter adjustment; Calculating the average number of activated neurons in the pre-trained model after parameter adjustment at a preset time step; and Combining the regularization term coefficient and the loss function to construct a Lagrangian function, and minimizing the Lagrangian function to obtain an optimized function; Calculating the parameter weights of the optimized function on the pre-trained model after parameter adjustment; The gradient of the parameter weight is normalized according to a gradient normalization method to obtain a target model, wherein the gradient normalization method includes: ;in, is the parameter weight of the pre-trained model after parameter adjustment, is the weight gradient, is the mean of the weight gradient, is the variance of the weight gradient, is the deviation correction value, is a scaling factor; the target model is used to predict events associated with a time series.

2. The method according to claim 1, characterized in that Determining the low-rank matrix requirements of the pre-trained model includes: Determine the dimension information of the pre-trained model and the proportion of fine-tuning parameters; The matrix rank is determined according to the dimension information and the proportion of the fine-tuning parameters, and the low-rank matrix requirement is determined according to the matrix rank.

3. The method according to claim 1, characterized in that Determining the initial value distribution of the low-rank matrix requirement includes: Determining input data at a preset time in a loop structure of the pre-trained model; In a case where the input data is a skewed distribution with a small skewness, determining the third-order center distance of the input data; A skewed distribution probability density function is determined based on the third-order center distance, and the skewed distribution probability density function is used as the initial value distribution required by the low-rank matrix.

4. The method according to claim 1, wherein Adjusting the parameters of the pre-trained model based on the low-rank matrix weight to obtain the adjusted pre-trained model includes: Obtaining a model weight of the pre-trained model; The model weight is added to the low-rank matrix weight to adjust the parameters of the pre-trained model to obtain the adjusted pre-trained model.

5. A time series prediction method, characterized in that: include: Acquire historical time characteristic data from the target data set, wherein the historical time characteristic data at least includes: meteorological time characteristic data; Inputting the historical time feature data into a target model to obtain data feature relationships between different time points, wherein the target model is obtained by the large model fine-tuning method according to any one of claims 1 to 4; Based on the data feature relationship, data features in the future are predicted.

6. A large model fine-tuning device, characterized in that: include: A reading unit, configured to read the pre-trained model in a preset manner, wherein the preset manner includes at least one of the following: API reading, local reading, and API reading of a cloud computing service platform; A determination unit is used to determine the low-rank matrix requirement of the pre-training model, determine the initial value distribution of the low-rank matrix requirement, and calculate the gamma distribution of the low-rank matrix weights based on the initial value distribution; wherein the low-rank matrix requirement is used to characterize the size of the low-rank matrix; the initial value distribution includes: a skewed distribution probability density function for capturing the asymmetry of the input data of the pre-training model; the initial value distribution is a skewed distribution based on the third-order center distance of the input data associated with the time series, and is used to capture the asymmetry of the input data; the gamma distribution, the variance of the activation function input and output, and the third-order center distance are combined to derive the weight distribution of the low-rank matrix weights; the weight distribution is processed by gamma sampling to obtain the low-rank matrix weights; A parameter adjustment unit, configured to adjust the parameters of the pre-trained model based on the low-rank matrix weights to obtain a pre-trained model after parameter adjustment; A training unit is configured to calculate the average number of activated neurons of the pre-trained model after parameter adjustment at a preset time step; and to combine the regularization term coefficient and the loss function to construct a Lagrangian function, and to minimize the Lagrangian function to obtain an optimized function; calculate the parameter weights of the optimized function for the pre-trained model after parameter adjustment; and to normalize the gradients of the parameter weights according to a gradient normalization method to obtain a target model, wherein the gradient normalization method includes: ;in, is the parameter weight of the pre-trained model after parameter adjustment, is the weight gradient, is the mean of the weight gradient, is the variance of the weight gradient, is the deviation correction value, is a scaling factor; the target model is used to predict events associated with a time series.

7. A computer-readable storage medium, characterized in that The storage medium includes a stored program, wherein the program executes the large model fine-tuning method according to any one of claims 1 to 4 or the time series prediction method according to claim 5.

8. An electronic device, characterized in that: include: One or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the large model fine-tuning method described in any one of claims 1 to 4 or the time series forecasting method described in claim 5.

Images