Charging behavior prediction model construction and prediction method and system based on large model

Abstract

The invention provides a charging behavior prediction model construction and prediction method and system based on a large model, and the method comprises the steps: carrying out the time sequence arrangement of a plurality of groups of historical charging data, selecting the historical charging data of a corresponding group number according to the window width of an incremental window method and a time sequence, and constructing a plurality of samples, and obtaining a sample set; the window width is gradually increased as time goes on; based on the sample set, training a large language model by adopting a low-rank adaptive supervised fine tuning method to obtain a charging behavior prediction model, and performing prediction by adopting the charging behavior prediction model to obtain charging data of next charging of the to-be-predicted user; according to the method and system, a sample set is constructed through multiple groups of historical charging behavior data and an incremental window method, multi-source information is fused, and the application value and prediction accuracy of a prediction model are improved; and meanwhile, historical charging behavior data of the user is deeply understood by utilizing the generation capability of the large language model to carry out comprehensive prediction, richer and more accurate prediction results are provided, and the interpretability of the prediction results is improved.

Description

Technical Field

[0001] This invention belongs to the field of charging behavior prediction technology, specifically involving a charging behavior prediction model construction, prediction method and system based on a large model. Background Technology

[0002] Given the dwindling supply of fossil fuels and their negative impact on the human environment, coupled with the continuous advancements in new energy technologies, the electric vehicle market has experienced rapid growth in recent years. To meet the needs of electric vehicle users for convenient and efficient charging, while ensuring the safe and stable operation of the power distribution network, we urgently need to provide comprehensive technical support and improved infrastructure for electric vehicles. In-depth research into charging transaction data across various regions, analyzing user charging characteristics, and predicting charging behavior will help charging service providers offer more customized services, potentially reducing charging costs for users and operating costs for charging service providers.

[0003] Traditional machine learning and deep learning methods (such as random forests, neural networks, and LSTM) rely on user historical data and vehicle driving data to model and predict the charging behavior of electric vehicle users. In addition, some studies use large-scale datasets and methods such as regression analysis and time series analysis to build data-driven predictive models, which can also be used to predict electric vehicle charging.

[0004] Similar work has been done in predicting the charging behavior of electric vehicle users, including:

[0005] The Journal of Beijing University of Aeronautics and Astronautics (Yu Haiyang et al., Vol. 45, No. 9, 2019) proposes a method for analyzing the influencing factors of electric vehicle charging behavior based on the trip chain. By integrating data from the charging and discharging processes of electric vehicles through the trip chain, the method comprehensively considers various potential factors affecting electric vehicle charging behavior. The method also identifies the factors that significantly influence charging behavior through logistic regression analysis, and models and predicts whether charging will occur during the trip chain.

[0006] Patent CN117874447A proposes a method for predicting electric vehicle user charging behavior based on activity analysis. It constructs a spatiotemporal three-dimensional model of electric vehicles using activity analysis and describes user travel behavior in different times and scenarios using a probability density function. Based on user travel behavior and the charging characteristics of electric vehicles, an aggregation model of an electric vehicle cluster is established. User charging behavior is predicted by calculating the parameters of the equivalent energy storage devices in the cluster model.

[0007] The patent with publication number CN118100240A proposes an energy storage method and system for electric vehicle charging stations. First, it uses Internet of Things (IoT) technology to collect historical charging data, surrounding traffic flow, weather forecasts, and user behavior patterns of the charging station to generate a comprehensive dataset. Then, it uses a long short-term memory (LSTM) network algorithm to predict and analyze charging demand, thereby determining the charging and discharging strategy of the energy storage device and generating an energy management strategy. Finally, it uses an energy scheduling optimization algorithm to integrate solar and wind energy output data to generate an optimized energy fusion scheme, and adjusts the strategy in real time and performs intelligent fault diagnosis.

[0008] While various methods and technologies for predicting electric vehicle user charging behavior have emerged, shortcomings remain. Some technologies only predict the decision of whether to charge, exhibiting poor multi-source information fusion capabilities, which limits the application value of research results in power system planning and operation. Some data-dependent and complex algorithms, when performing time series forecasting, require significant computational resources, and their accuracy is affected by data quality, easily leading to prediction errors. Furthermore, they suffer from poor interpretability. Summary of the Invention

[0009] To overcome the shortcomings of the existing technology, this invention proposes a method for constructing a charging behavior prediction model based on a large model, comprising:

[0010] Obtain multiple sets of historical charging data from electric vehicle users;

[0011] The historical charging data from multiple sets are arranged in time sequence. Based on the incremental window method, a number of samples are constructed by selecting the corresponding number of historical charging data sets in time sequence according to the window width of the incremental window method, thus obtaining a sample set. The window width gradually increases over time.

[0012] Based on the sample set, a large language model is trained using a low-rank adaptive supervised fine-tuning method to obtain a well-constructed charging behavior prediction model.

[0013] Preferably, the step of arranging multiple sets of historical charging data in time sequence, based on the incremental window method, involves selecting historical charging data of a corresponding number of groups according to the window width of the incremental window method to construct several samples, resulting in a sample set, including:

[0014] The multiple sets of historical charging data are arranged in chronological order from earliest to latest, and the arranged sets of historical charging data are numbered.

[0015] In the multiple sets of historical charging data after arrangement, according to the preset initial window width of the incremental window, starting from the first set of historical charging data, a number of sets of historical charging data corresponding to the initial window width are selected as the input data of the first sample, and the next set of historical charging data outside the current incremental window is used as the output data of the first sample.

[0016] Then, the window width of the incrementing window is increased according to the set increment. Based on the current window width and position of the incrementing window, several sets of historical charging data with corresponding quantity and position are selected as the input data of the second sample. The next set of historical charging data outside the current incrementing window is used as the output data of the second sample.

[0017] Then, the incrementing window is updated again according to the set increment until the output data of the sample is the last set of historical charging data in the sorted set of historical charging data, so as to obtain multiple samples and construct the sample set.

[0018] Preferably, the step of training a large language model based on the sample set using a low-rank adaptive supervised fine-tuning method to obtain a constructed charging behavior prediction model includes:

[0019] The prompt words for the large language model are set based on the historical charging data in the sample set;

[0020] The sample set is divided into a training set and a test set according to a set ratio;

[0021] Based on the training set, using the input data of the samples in the training set as input and the corresponding output data as output, the low-rank matrix in the low-rank adaptive supervised fine-tuning method is trained. The low-rank matrix is ​​then introduced into the large language model, and the parameters of the large language model are supervisedly fine-tuned to minimize the loss function of the large language model. Gradient descent is used to optimize and update the low-rank matrix. The large language model during training is evaluated using the test set until a preset number of iterations is reached, resulting in the constructed charging behavior prediction model.

[0022] Preferably, the step of introducing the low-rank matrix into the large language model and performing supervised fine-tuning of the parameters of the large language model includes:

[0023] The low-rank matrix is ​​incorporated into the query matrix and value matrix of the large language model, and the query matrix and value matrix of the large language model are then subjected to supervised fine-tuning updates.

[0024] Preferably, obtaining multiple sets of historical charging data from electric vehicle users includes:

[0025] The system obtains the charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to the multiple charging behaviors of the electric vehicle users.

[0026] The charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to a single charging behavior are collected as a set of historical charging data.

[0027] Based on the same inventive concept, this invention also provides a charging behavior prediction model construction system based on a large model, comprising:

[0028] The historical data acquisition module is used to acquire multiple sets of historical charging data from electric vehicle users;

[0029] The sample construction module is used to arrange multiple sets of historical charging data in time sequence. Based on the incremental window method, it selects the corresponding number of historical charging data in time sequence according to the window width of the incremental window method to construct several samples and obtain a sample set; wherein, the window width gradually increases with time.

[0030] The model training module is used to train a large language model based on the sample set using a low-rank adaptive supervised fine-tuning method, thereby obtaining a well-constructed charging behavior prediction model.

[0031] Preferably, the sample construction module is specifically used for:

[0032] The multiple sets of historical charging data are arranged in chronological order from earliest to latest, and the arranged sets of historical charging data are numbered.

[0033] In the multiple sets of historical charging data after arrangement, according to the preset initial window width of the incremental window, starting from the first set of historical charging data, a number of sets of historical charging data corresponding to the initial window width are selected as the input data of the first sample, and the next set of historical charging data outside the current incremental window is used as the output data of the first sample.

[0034] Then, the window width of the incrementing window is increased according to the set increment. Based on the current window width and position of the incrementing window, several sets of historical charging data with corresponding quantity and position are selected as the input data of the second sample. The next set of historical charging data outside the current incrementing window is used as the output data of the second sample.

[0035] Then, the incrementing window is updated again according to the set increment until the output data of the sample is the last set of historical charging data in the sorted set of historical charging data, so as to obtain multiple samples and construct the sample set.

[0036] Preferably, the model training module is specifically used for:

[0037] The prompt words for the large language model are set based on the historical charging data in the sample set;

[0038] The sample set is divided into a training set and a test set according to a set ratio;

[0039] Based on the training set, using the input data of the samples in the training set as input and the corresponding output data as output, the low-rank matrix in the low-rank adaptive supervised fine-tuning method is trained. The low-rank matrix is ​​then introduced into the large language model, and the parameters of the large language model are supervisedly fine-tuned to minimize the loss function of the large language model. Gradient descent is used to optimize and update the low-rank matrix. The large language model during training is evaluated using the test set until a preset number of iterations is reached, resulting in the constructed charging behavior prediction model.

[0040] Preferably, the model training module is further specifically used for:

[0041] The low-rank matrix is ​​incorporated into the query matrix and value matrix of the large language model, and the query matrix and value matrix of the large language model are then subjected to supervised fine-tuning updates.

[0042] Preferably, obtaining multiple sets of historical charging data from electric vehicle users includes:

[0043] The system obtains the charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to the multiple charging behaviors of the electric vehicle users.

[0044] The charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to a single charging behavior are collected as a set of historical charging data.

[0045] Based on the same inventive concept, this invention also provides a charging behavior prediction method based on a large model, comprising:

[0046] Obtain multiple sets of historical charging data for the user to be predicted;

[0047] Using multiple sets of historical charging data of the user to be predicted as input, a pre-built charging behavior prediction model is used to predict the charging data of the user to be predicted for the next charging.

[0048] The charging behavior prediction model is constructed based on the large model-based charging behavior prediction model construction method described above.

[0049] Based on the same inventive concept, this invention also provides a charging behavior prediction system based on a large model, comprising:

[0050] The data acquisition module is used to acquire multiple sets of historical charging data for the user to be predicted;

[0051] The prediction module is used to take multiple sets of historical charging data of the user to be predicted as input, and use a pre-built charging behavior prediction model to predict the charging data of the user to be predicted for the next charging.

[0052] The charging behavior prediction model is constructed based on the large model-based charging behavior prediction model construction method described above.

[0053] Based on the same inventive concept, the present invention also provides a computer device, comprising: one or more processors;

[0054] Memory, used to store one or more programs;

[0055] When the one or more programs are executed by the one or more processors, a charging behavior prediction model construction method based on a large model as described above, or a charging behavior prediction method based on a large model as described above, is implemented.

[0056] Based on the same inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the charging behavior prediction model construction method based on a large model as described above, or the charging behavior prediction method based on a large model as described above.

[0057] Compared with the closest existing technology, the present invention has the following beneficial effects:

[0058] This invention provides a method and system for constructing a charging behavior prediction model based on a large model. The method includes acquiring multiple sets of historical charging data from electric vehicle users; arranging the historical charging data in a time sequence; constructing several samples by selecting a corresponding number of historical charging data sets in a time sequence based on an increasing window method, according to the window width of the increasing window method; wherein the window width gradually increases over time; and training a large language model using a low-rank adaptive supervised fine-tuning method based on the sample set to obtain the constructed charging behavior prediction model. This method and system, by acquiring multiple sets of historical charging behavior data and constructing a sample set using the increasing window method, enables the prediction model to integrate multi-source information, improving the application value and prediction accuracy of the prediction model. Simultaneously, it utilizes the generative capabilities of the large language model to predict the charging behavior of electric vehicle users, performing comprehensive predictions through a deep understanding of users' historical charging behavior data. This generative prediction method provides richer and more accurate prediction results, improving the interpretability of the prediction results. Furthermore, the use of a low-rank adaptive supervised fine-tuning method to train the large language model ensures that the original weights of the large language model are not changed during fine-tuning, significantly reducing the required storage and computational resources and further avoiding prediction errors.

[0059] This invention also provides a charging behavior prediction method and system based on a large model, including acquiring multiple sets of historical charging data of a user to be predicted; using the multiple sets of historical charging data of the user to be predicted as input, and using a pre-constructed charging behavior prediction model to predict the charging data for the user's next charge; the charging behavior prediction model is constructed based on the aforementioned charging behavior prediction model construction method; by using the charging behavior prediction model to predict charging behavior, the method and system enable the output charging data for the next charge to integrate multi-source information, improving the accuracy of the prediction results. Furthermore, the charging behavior prediction model utilizes the generative capabilities of a large language model to predict the charging behavior of electric vehicle users, and performs comprehensive prediction by deeply understanding the user's historical charging behavior data. This generative prediction method can provide richer and more accurate prediction results, improving the interpretability of the prediction results. Attached Figure Description

[0060] Figure 1 A schematic diagram illustrating the process of constructing a charging behavior prediction model based on a large model, as provided by this invention;

[0061] Figure 2 A schematic diagram of a charging behavior prediction model construction system based on a large model provided by the present invention;

[0062] Figure 3 A schematic diagram of the charging behavior prediction method based on a large model provided by the present invention;

[0063] Figure 4 A schematic diagram of the overall process of a charging behavior prediction method based on a large model provided by the present invention;

[0064] Figure 5 A schematic diagram of a charging behavior prediction system based on a large model provided by the present invention;

[0065] Figure 6 This is a schematic diagram of an electronic device structure provided by the present invention. Detailed Implementation

[0066] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0067] Example 1:

[0068] This invention provides a method for constructing a charging behavior prediction model based on a large model, such as... Figure 1 As shown, it includes:

[0069] S1. Obtain multiple sets of historical charging data from electric vehicle users;

[0070] S2. Arrange multiple sets of historical charging data in time sequence. Based on the incremental window method, select the corresponding number of historical charging data sets in time sequence according to the window width of the incremental window method to construct several samples and obtain a sample set; wherein, the window width gradually increases with time.

[0071] S3. Based on the sample set, a low-rank adaptive supervised fine-tuning method is used to train the large language model, resulting in a well-constructed charging behavior prediction model.

[0072] Existing methods for predicting electric vehicle (EV) user charging behavior primarily rely on regression analysis and neural networks to model historical order data and vehicle driving data. However, charging behavior is not only linked to historical statistics but also to factors such as weather conditions and preferential policies, leading to difficulties in integrating multi-source data. Furthermore, training network models based on statistical data lacks expert knowledge and suffers from poor interpretability. This invention addresses this issue by acquiring multiple sets of historical charging behavior data and constructing a sample set using an incremental windowing method. This allows the prediction model to integrate multi-source information, consolidating rich data and improving its application value and accuracy. Simultaneously, it utilizes large-scale language models—the generative capabilities of large models—to predict EV user charging behavior. By deeply understanding historical charging behavior data, it provides comprehensive predictions, offering richer and more accurate results and improving interpretability. Personalized predictions for individual users also contribute to improved operational efficiency for charging service providers. Furthermore, a low-rank adaptive supervised fine-tuning method is used to train the large language model. This training method does not change the original weights of the large language model during the fine-tuning process, which greatly reduces the required storage and computing resources and further avoids prediction errors.

[0073] Considering that user charging behavior is not only related to charging time, but also that the relevant data may vary significantly at different times, in this embodiment, S1 specifically includes: data collection and preprocessing: collecting data related to the charging behavior of electric vehicle users from multiple data sources, including the user's historical charging time, geographical location (i.e., the charging station where the charging is located), weather conditions, etc., and cleaning and standardizing the collected data.

[0074] Specifically, information related to electric vehicle user charging behavior from all data sources is integrated, and data aggregation is performed with user ID as the core to ensure that each user's charging records are collected in a single dataset, i.e., the sample set.

[0075] This invention focuses on summarizing and analyzing behavioral data generated by users during the charging process when constructing the dataset, such as the specific time of charging, duration, amount of electricity charged, and specific location of charging. Furthermore, the method also involves incorporating other related information, such as weather factors, into the dataset, thereby creating a comprehensive training dataset containing multi-dimensional information to support the model fine-tuning training process.

[0076] Considering that the samples generated by the traditional sliding window method have a relatively uniform time span and are insufficient to capture the diversity of data at different time scales, in S2 above, an incremental window method is used to construct the sample set to better capture the long-term dependencies in time series data. In this embodiment, S2 may include:

[0077] Multiple sets of historical charging data are arranged in chronological order from earliest to latest, and the arranged sets of historical charging data are numbered.

[0078] In the sorted sets of historical charging data, based on the preset initial window width of the incremental window, starting from the first set of historical charging data, select a number of sets of historical charging data corresponding to the initial window width as the input data of the first sample, and use the next set of historical charging data outside the current incremental window as the output data of the first sample.

[0079] Then, the window width of the incremental window is increased according to the set increment. Based on the current window width and position of the incremental window, several sets of historical charging data of corresponding quantity and position are selected as the input data of the second sample, and the next set of historical charging data outside the current incremental window is used as the output data of the second sample.

[0080] Then, the incremental window is updated again according to the set increment until the output data of the sample is the last set of historical charging data in the sorted set of historical charging data, thus obtaining multiple samples and constructing a sample set.

[0081] Specifically, when sorting the data, all user data is sorted by user ID, and each user's multiple charging data is sorted by historical charging time.

[0082] Furthermore, after data sorting and before sample construction, the process includes: calculating and filtering the sorted user data to identify the most relevant indicators for predicting user charging behavior, such as charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions. The values ​​of the aforementioned indicators corresponding to a historical charging time constitute a set of historical charging data for electric vehicle users.

[0083] In this embodiment, obtaining multiple sets of historical charging data from electric vehicle users includes:

[0084] Acquire information on the charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to multiple charging behaviors of electric vehicle users;

[0085] The charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to a single charging behavior are compiled into a set of historical charging data.

[0086] Among them, charging time is the start time of the user's current charging; charging duration is the length of the user's current charging time, i.e., start charging time - end charging time; charging station is the name of the charging station for the user's current charging; transaction volume is the amount of electricity charged by the user for this charging; transaction amount is the amount paid by the user for this charging; discount type is the type of charging discount used by the user for this charging, such as no minimum purchase discount, full discount, or no discount; weather condition is the weather at the time of the user's charging.

[0087] For example, when constructing the sample set, assume that each electric vehicle user has n sets of historical charging data. For each user's n historical charging data points sorted by charging time, m data samples (m = n-1) are constructed sequentially for training and testing the large language model. Subsequently, samples from multiple users are used for model training.

[0088] The specific construction rules are as follows: 1. The input of the first data sample is the first charging data, that is, the first set of historical charging data, and the output is the second charging data, that is, the first set of historical charging data;

[0089] 2. The input for the second data sample is the data from the first two charges, and the output is the data from the third charge;

[0090] 3. Construct sequentially, that is, the input of the m-th data sample is the data from the previous m charging times, and the output is the data from the (m+1)-th charging time;

[0091] 4. Divide the constructed m data samples into m-2 training data samples and the last 2 test data samples.

[0092] Using the incremental window method to construct the sample set has the following advantages:

[0093] Incremental windowing, by gradually increasing the window width, can construct data samples of different sizes. This method helps capture the characteristics of data at different time scales, thereby enhancing the model's generalization ability.

[0094] In the incremental window method, the construction of each new sample only requires adding new data points based on the previous sample, instead of reselecting all data points in the window each time as in the sliding window method, thereby reducing redundant calculations and improving the efficiency of sample construction.

[0095] Increasing window method can better capture long-term dependencies in time series data. As the window width increases, the model can learn data patterns over a longer range, which is beneficial for predicting long-term trends.

[0096] Since the incremental window method does not require frequent recalculation of the data within the entire window, it can make more efficient use of computing resources, especially when dealing with large-scale datasets.

[0097] Compared to the sliding window method, which may lead to model overfitting due to overly similar data points within the window, the incremental window method can avoid this situation by gradually increasing the window width, enabling large language models to learn a wider range of data features.

[0098] In summary, constructing a sample set using the incremental window method to train a large language model can effectively improve the accuracy of predicting electric vehicle user charging behavior.

[0099] Large Language Models (LLMs) are commonly used in natural language processing tasks such as text generation, machine translation, and sentiment analysis. LLMs can handle unstructured data and demonstrate strong transfer learning capabilities across multiple tasks. First, LLMs excel at handling complex contextual relationships, capturing intricate relationships within user behavior, giving them an advantage over traditional prediction models. Second, LLMs possess rich cross-domain knowledge, extracting information from data across multiple domains to aid in predicting user behavior. Applying LLMs to electric vehicle charging behavior prediction effectively utilizes large-scale historical charging data to accurately predict future user charging behavior.

[0100] Compared to existing technologies, the above-mentioned S3 utilizes a large language model to perform more accurate analysis and prediction of complex data. Based on the knowledge contained in the large language model, it not only integrates historical charging behavior data such as the specific charging time, duration, and amount of electricity charged, but also considers related factors such as weather conditions and preferential policies, making it more consistent with the actual charging situation of electric vehicle users. Therefore, this invention can improve the charging experience for electric vehicle users and help charging stations optimize resource allocation. Furthermore, it has good scalability, adapting to different user information inputs and accurately predicting future charging behavior.

[0101] In this embodiment, S3 may include:

[0102] Set prompt words for the large language model based on historical charging data in the sample set;

[0103] The sample set is divided into a training set and a test set according to a set ratio;

[0104] Based on the training set, using the input data of the samples in the training set as input and the corresponding output data as output, the low-rank matrix in the low-rank adaptive supervised fine-tuning method is trained. The low-rank matrix is ​​then introduced into the large language model, and the parameters of the large language model are fine-tuned in a supervised manner. With the goal of minimizing the loss function of the large language model, the gradient descent method is used to optimize and update the low-rank matrix. The large language model during the training process is evaluated using a test set until the preset number of iterations is reached, resulting in the constructed charging behavior prediction model.

[0105] In this embodiment, a low-rank matrix is ​​introduced into the large language model to perform supervised fine-tuning of the model's parameters, including:

[0106] A low-rank matrix is ​​introduced into the query matrix and value matrix of the large language model to perform supervised fine-tuning updates on the query matrix and value matrix of the large language model.

[0107] A key aspect of this invention lies in how to effectively utilize the background knowledge base of a large language model. This includes task-specific introductions and the integration of expert knowledge or domain-specific knowledge into the model. By setting targeted prompts to mine and utilize this background knowledge, the accuracy and reliability of the model in a specific domain, namely, predicting the charging behavior of electric vehicle users, can be significantly improved.

[0108] For example, the Qwen-1.5-chat large language model (also known as the Qwen large model) is used to train the processed sample set to learn the patterns and rules of user charging behavior. The Qwen large model is chosen for training electric vehicle user charging behavior prediction. The Qwen large model has powerful representation capabilities, capturing rich semantic information, which helps improve the effectiveness of fine-tuning training. Furthermore, the Qwen large model has achieved excellent performance on multiple natural language processing tasks, indicating its strong generalization ability and suitability as a base model for fine-tuning training.

[0109] Specifically, when setting prompt words for the large language model in S3 based on historical charging data in the sample set, it includes:

[0110] Setting prompts for large language models effectively improves the accuracy and satisfaction of model output by clearly defining task objectives and specifying formats. Specifically, the prompt could be: "Based on the given historical charging data of a user, predict the user's next charging data. The format of your predicted next charging data should be the same as the historical charging data, including charging time, charging duration, charging station, transaction amount, transaction amount, discount type, and weather conditions."

[0111] Subsequently, in S3, the constructed electric vehicle user charging data, i.e., the sample set, is used for LoRA (Low-Rank Adaptation) supervised fine-tuning model training, that is, training the large language model using a low-rank adaptive supervised fine-tuning method. In this stage, the model parameters are updated based on the electric vehicle user charging data to optimize the model's performance in predicting user charging behavior. Specifically, this includes the following steps:

[0112] (1) Initialization

[0113] The self-attention mechanism in the Qwen large model is one of its core components, allowing the model to consider the entire input sequence when processing each token. The self-attention mechanism is implemented through three matrices: query (Q), key (K), and value (V), which extract relevant information from the input data.

[0114] Query (Q) Matrix: The Q matrix represents the relationship between the token the model is currently processing and all other tokens. In the self-attention layer, the Q matrix is ​​used to calculate the attention score, which indicates how much attention should be given to other tokens when generating the representation of the current token.

[0115] Key (K) matrix: The K matrix is ​​used to determine the importance of each token in the input sequence. The dot product of Q and K is used to calculate the attention score, which reflects the strength of the relationship between different elements in the input sequence.

[0116] Value (V) matrix: The V matrix contains the value of each token in the input sequence, which is weighted according to the attention scores calculated by Q and K. Finally, the sum of the weighted V matrices forms the output sequence, where the representation of each token takes into account information from the entire input sequence.

[0117] We chose to introduce the LoRA module for fine-tuning on the query (Q) and value (V) matrices. This is because adjusting the Q and V matrices directly changes the model's focus on the input data and how it integrates this information, without having to adjust the key (K) matrix, which is primarily responsible for determining the similarity between the input data.

[0118] The mathematical representation of LoRA is:

[0119] W' = W + AB T ;

[0120] Where: W is the original weight matrix; W' is the updated weight matrix; A and B are both trainable low-rank matrices;

[0121] Therefore, it is necessary to initialize two low-rank matrices A and B of LoRA. We initialize matrix A with a uniform distribution, and matrix B is initialized as a zero matrix.

[0122] (2) Forward propagation

[0123] During the forward propagation of the model, the updated weights are used for computation. Specifically, the original weight matrix W is replaced with W'. In this way, LoRA only updates A and B, while the original remains unchanged. The advantage of this is that it reduces the number of parameters that need to be trained, thereby reducing computational costs and storage requirements.

[0124] (3) Calculate the loss

[0125] This process uses the mean squared error function, which measures the error between the model output and the actual charging behavior data. The mean squared error function can be expressed as:

[0126]

[0127] in: is the loss function, representing the model's predicted value. The error between the actual value y and the actual value y. N is the number of data points in the training set, i.e., the number of samples in the sample set. i It is the actual charging data of the i-th data, including charging time, charging duration, charging station, transaction volume, transaction amount, discount type and weather conditions. It is the predicted charging data for the i-th data point, including charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions.

[0128] (4) Backpropagation

[0129] The gradient is calculated based on the loss function, and the values ​​of LoRA matrices A and B are updated. During backpropagation, the original pre-trained weights W are fixed and do not participate in the gradient calculation.

[0130] (5) Parameter update

[0131] Update the LoRA matrices A and B using gradient descent. The formula for parameter update can be expressed as:

[0132]

[0133] Where: θ t These are the model parameters at time step t. η is the learning rate, which controls the size of the learning step. It is the gradient of the loss function L at time step t-1 with respect to the model parameters θ.

[0134] For example, in LoRA fine-tuning, we set the following parameters:

[0135] Training batch size per device: 4; gradient accumulation steps: 8; learning rate: 1e-4; number of training epochs: 5.0;

[0136] Through continuous iteration of this process, the model's parameters are gradually adjusted until a pre-set number of training epochs are reached. Using the MSE loss function, the model learns how to adjust the parameters of the Q and V matrices to better predict sequential data of user charging behavior.

[0137] In summary, to improve the accuracy and interpretability of electric vehicle user charging behavior predictions, this invention integrates data from multiple sources, including not only historical user charging behavior data but also factors such as weather conditions, providing a more comprehensive data perspective. Therefore, the model can be trained and inferred based on a richer information set, significantly improving the accuracy and reliability of predictions. Furthermore, this invention utilizes the natural language generation capabilities of a large language model, allowing the decision-making logic for charging behavior prediction to be expressed in natural language, thereby improving the acceptability, interpretability, and trustworthiness of the predictions. Operations departments can then more effectively propose targeted operational and marketing strategies based on the prediction results and analysis, achieving better operational efficiency and cost control.

[0138] Example 2:

[0139] Based on the same inventive concept, this invention also provides a charging behavior prediction model construction system based on a large model, such as... Figure 2 As shown, it includes:

[0140] The historical data acquisition module is used to acquire multiple sets of historical charging data from electric vehicle users;

[0141] The sample construction module is used to arrange multiple sets of historical charging data in time sequence. Based on the incremental window method, it selects the corresponding number of historical charging data sets in time sequence according to the window width of the incremental window method to construct several samples and obtain a sample set; wherein, the window width gradually increases over time.

[0142] The model training module is used to train a large language model based on a sample set using a low-rank adaptive supervised fine-tuning method, resulting in a well-constructed charging behavior prediction model.

[0143] In this embodiment, the sample construction module is specifically used for:

[0144] Multiple sets of historical charging data are arranged in chronological order from earliest to latest, and the arranged sets of historical charging data are numbered.

[0145] In the sorted sets of historical charging data, based on the preset initial window width of the incremental window, starting from the first set of historical charging data, select a number of sets of historical charging data corresponding to the initial window width as the input data of the first sample, and use the next set of historical charging data outside the current incremental window as the output data of the first sample.

[0146] Then, the window width of the incremental window is increased according to the set increment. Based on the current window width and position of the incremental window, several sets of historical charging data of corresponding quantity and position are selected as the input data of the second sample, and the next set of historical charging data outside the current incremental window is used as the output data of the second sample.

[0147] Then, the incremental window is updated again according to the set increment until the output data of the sample is the last set of historical charging data in the sorted set of historical charging data, thus obtaining multiple samples and constructing a sample set.

[0148] In this embodiment, the model training module is specifically used for:

[0149] Set prompt words for the large language model based on historical charging data in the sample set;

[0150] The sample set is divided into a training set and a test set according to a set ratio;

[0151] Based on the training set, using the input data of the samples in the training set as input and the corresponding output data as output, the low-rank matrix in the low-rank adaptive supervised fine-tuning method is trained. The low-rank matrix is ​​then introduced into the large language model, and the parameters of the large language model are fine-tuned in a supervised manner. With the goal of minimizing the loss function of the large language model, the gradient descent method is used to optimize and update the low-rank matrix. The large language model during the training process is evaluated using a test set until the preset number of iterations is reached, resulting in the constructed charging behavior prediction model.

[0152] In this embodiment, the model training module is also specifically used for:

[0153] A low-rank matrix is ​​introduced into the query matrix and value matrix of the large language model to perform supervised fine-tuning updates on the query matrix and value matrix of the large language model.

[0154] In this embodiment, obtaining multiple sets of historical charging data from electric vehicle users includes:

[0155] Acquire information on the charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to multiple charging behaviors of electric vehicle users;

[0156] The charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to a single charging behavior are compiled into a set of historical charging data.

[0157] Example 3:

[0158] Based on the same inventive concept, this invention also provides a charging behavior prediction method based on a large model, such as... Figure 3 As shown, it includes:

[0159] A1. Obtain multiple sets of historical charging data for the user to be predicted;

[0160] A2. Using multiple sets of historical charging data of the user to be predicted as input, a pre-built charging behavior prediction model is used to predict the charging data of the user to be predicted for the next charging.

[0161] like Figure 4 As shown, the charging behavior prediction model is constructed based on the large model-based charging behavior prediction model construction method in the above embodiments.

[0162] This invention uses the aforementioned charging behavior prediction model to predict charging behavior, enabling the output charging data for the next charge to integrate multi-source information, thereby improving the accuracy of the prediction results. Furthermore, the charging behavior prediction model utilizes the generative capabilities of a large language model to predict the charging behavior of electric vehicle users. By deeply understanding the user's historical charging behavior data, it makes comprehensive predictions. This generative prediction method can provide richer and more accurate prediction results, improving the interpretability of the prediction results.

[0163] Specifically, in A2 above, predicting a user's future charging behavior based on a trained charging behavior prediction model can include:

[0164] Based on the constructed electric vehicle user charging test data, namely the multiple sets of historical charging data of the users to be predicted, the model is tested on the supervised fine-tuned Qwen-1.5-chat model, and the prediction success rate of the model is calculated.

[0165] By using n (n>2) historical charging data points for each user to be predicted as input, the system can predict the user's next charging time, charging duration, charging station, transaction amount, transaction value, discount type, and weather conditions. Each prediction input consists of all the user's existing historical charging data.

[0166] Based on the predicted data, targeted advertising of charging services and specific types of discounts can be pushed to different users at specific times.

[0167] In summary, to improve the accuracy and interpretability of electric vehicle (EV) user charging behavior predictions, this invention utilizes a large language model. This model goes beyond simply predicting based on statistical data; it employs a comprehensive approach by deeply understanding users' historical charging patterns, timing, frequency, location, and other relevant factors such as discount types and weather conditions. This generative prediction method provides richer and more accurate prediction results. It not only learns from historical data but also adapts to new changes and trends. This capability ensures the model can still provide accurate predictions when facing different users, thus enabling better personalized predictions and customized services for users.

[0168] Example 4:

[0169] Based on the same inventive concept, this invention also provides a charging behavior prediction system based on a large model, such as... Figure 5 As shown, it includes:

[0170] The data acquisition module is used to acquire multiple sets of historical charging data for the user to be predicted;

[0171] The prediction module is used to take multiple sets of historical charging data of the user to be predicted as input, and use a pre-built charging behavior prediction model to predict the charging data of the user to be predicted for the next charging.

[0172] The charging behavior prediction model is constructed based on the large model-based charging behavior prediction model construction method in the above embodiments.

[0173] Example 5

[0174] like Figure 6 As shown, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, processor, and transceiver component are connected via a bus; the memory can be used to store executable programs, and an exemplary executable program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be accessed and / or modified when instructions are executed.

[0175] The processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, and it is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to realize the corresponding method flow or corresponding function, so as to realize the steps of the charging behavior prediction model construction method based on a large model or the charging behavior prediction method based on a large model in the above embodiments.

[0176] Example 6

[0177] Based on the same inventive concept, this invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory). This readable storage medium is a memory device within an electronic device used to store programs and data. It is understood that the storage medium here can include both built-in storage media within the electronic device and extended storage media supported by the electronic device. The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Loading and executing one or more instructions stored in the storage medium by the processor can implement the steps of a charging behavior prediction model construction method based on a large model or a charging behavior prediction method based on a large model in the above embodiments.

[0178] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0180] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0181] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a charging behavior prediction model based on a large model, characterized in that, include: Obtain multiple sets of historical charging data from electric vehicle users; The historical charging data from multiple sets are arranged in time sequence. Based on the incremental window method, a number of samples are constructed by selecting the corresponding number of historical charging data sets in time sequence according to the window width of the incremental window method, thus obtaining a sample set. The window width gradually increases over time. Based on the sample set, a large language model is trained using a low-rank adaptive supervised fine-tuning method to obtain a well-constructed charging behavior prediction model.

2. The method as described in claim 1, characterized in that, The process involves arranging multiple sets of historical charging data in time sequence. Based on the incremental window method, several samples are constructed by selecting the corresponding number of historical charging data sets in time sequence according to the window width of the incremental window method, resulting in a sample set, including: The multiple sets of historical charging data are arranged in chronological order from earliest to latest, and the arranged sets of historical charging data are numbered. In the multiple sets of historical charging data after arrangement, according to the preset initial window width of the incremental window, starting from the first set of historical charging data, a number of sets of historical charging data corresponding to the initial window width are selected as the input data of the first sample, and the next set of historical charging data outside the current incremental window is used as the output data of the first sample. Then, the window width of the incrementing window is increased according to the set increment. Based on the current window width and position of the incrementing window, several sets of historical charging data with corresponding quantity and position are selected as the input data of the second sample. The next set of historical charging data outside the current incrementing window is used as the output data of the second sample. Then, the incrementing window is updated again according to the set increment until the output data of the sample is the last set of historical charging data in the sorted set of historical charging data, so as to obtain multiple samples and construct the sample set.

3. The method as described in claim 2, characterized in that, The process involves training a large language model based on the sample set using a low-rank adaptive supervised fine-tuning method to obtain a well-constructed charging behavior prediction model, including: The prompt words for the large language model are set based on the historical charging data in the sample set; The sample set is divided into a training set and a test set according to a set ratio; Based on the training set, using the input data of the samples in the training set as input and the corresponding output data as output, the low-rank matrix in the low-rank adaptive supervised fine-tuning method is trained. The low-rank matrix is ​​then introduced into the large language model, and the parameters of the large language model are supervisedly fine-tuned to minimize the loss function of the large language model. Gradient descent is used to optimize and update the low-rank matrix. The large language model during training is evaluated using the test set until a preset number of iterations is reached, resulting in the constructed charging behavior prediction model.

4. The method as described in claim 3, characterized in that, The step of introducing the low-rank matrix into the large language model and performing supervised fine-tuning of the parameters of the large language model includes: The low-rank matrix is ​​incorporated into the query matrix and value matrix of the large language model, and the query matrix and value matrix of the large language model are then subjected to supervised fine-tuning updates.

5. The method as described in claim 1, characterized in that, The acquisition of multiple sets of historical charging data from electric vehicle users includes: The system obtains the charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to the multiple charging behaviors of the electric vehicle users. The charging time, charging duration, charging station, transaction volume, transaction amount, discount type, and weather conditions corresponding to a single charging behavior are collected as a set of historical charging data.

6. A system for constructing a charging behavior prediction model based on a large model, characterized in that, include: The historical data acquisition module is used to acquire multiple sets of historical charging data from electric vehicle users; The sample construction module is used to arrange multiple sets of historical charging data in time sequence. Based on the incremental window method, it selects the corresponding number of historical charging data in time sequence according to the window width of the incremental window method to construct several samples and obtain a sample set; wherein, the window width gradually increases with time. The model training module is used to train a large language model based on the sample set using a low-rank adaptive supervised fine-tuning method, thereby obtaining a well-constructed charging behavior prediction model.

7. A charging behavior prediction method based on a large model, characterized in that, include: Obtain multiple sets of historical charging data for the user to be predicted; Using multiple sets of historical charging data of the user to be predicted as input, a pre-built charging behavior prediction model is used to predict the charging data of the user to be predicted for the next charging. The charging behavior prediction model is constructed based on the charging behavior prediction model construction method based on a large model as described in any one of claims 1-5.

8. A charging behavior prediction system based on a large model, characterized in that, include: The data acquisition module is used to acquire multiple sets of historical charging data for the user to be predicted; The prediction module is used to take multiple sets of historical charging data of the user to be predicted as input, and use a pre-built charging behavior prediction model to predict the charging data of the user to be predicted for the next charging. The charging behavior prediction model is constructed based on the charging behavior prediction model construction method based on a large model as described in any one of claims 1-5.

9. An electronic device, characterized in that, include: At least one processor and memory; The memory and processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, a method for constructing a charging behavior prediction model based on a large model as described in any one of claims 1 to 5, or a method for predicting charging behavior based on a large model as described in claim 7, is implemented.

10. A readable storage medium, characterized in that, It contains an execution program, which, when executed, implements a charging behavior prediction model construction method based on a large model as described in any one of claims 1 to 5, or a charging behavior prediction method based on a large model as described in claim 7.

Images