Electric vehicle charging load prediction method based on pre-training language model

Abstract

The invention provides an electric vehicle charging load prediction method based on a pre-training language model. According to the method, a supervised learning sample is constructed for hour-level granularity load data according to a sliding window, and reversible instance normalization is adopted to relieve sequence non-stationarity. Two-channel prompt input is further constructed, wherein a workday / weekend mark is determined through a numerical sequence channel according to a prediction anchor point, a time backtracking token (lexical element) is introduced, and the workday / weekend mark and the normalized load value form a structured sequence; and the calendar-semantic channel injects station description and calendar characteristics such as hours and weeks. And applying time decay and same-type daily enhancement weight to the time backtracking token, cutting, generating a time-position identifier, and overlapping learnable time embedding in an embedding layer to realize time sequence adaptation. And sharing a pre-training trunk 'Bert', respectively coding dual-channel vectors, and outputting future multi-step loads through gating fusion regression. The method can improve the prediction precision and enhance the cross-day type generalization ability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of electric power load forecasting, and in particular to an electric vehicle charging load forecasting method based on a pre-trained language model. BACKGROUND

[0002] Electric vehicle (EV) charging load is significantly influenced by user behavior randomness, calendar periodicity, holiday effects and local events, and presents burstiness, spikiness and non-stationarity. Traditional statistical models (such as the Autoregressive Integrated Moving Average model, ARIMA) and some deep learning models (Long Short-Term Memory network, LSTM) have limitations in cross-scene generalization, semantic information fusion and small sample transfer.

[0003] In recent years, pre-trained language models have been tried to be applied to time series forecasting due to their strong representation ability, but there are still three problems in directly inputting numerical sequences into pre-trained language models: first, the original position encoding of pre-trained language models cannot describe the time-dependent relationship between "backtracking steps and prediction anchors", resulting in insufficient time-position semantics; second, there is a lack of explicit and structured injection mechanism for key prior knowledge in EV charging scenarios (such as weekday / weekend mode difference); third, a single input channel cannot effectively fuse numerical dynamics and high-level semantic information such as calendar and time, limiting the collaborative modeling ability of multi-source knowledge.

[0004] Therefore, an electric vehicle charging load forecasting method based on a pre-trained language model is proposed, which can embed EV charging scenario prior knowledge into model input in a structured manner, enhance the position learnability of time tokens, and balance training stability and model interpretability. SUMMARY

[0005] An electric vehicle charging load forecasting method based on a pre-trained language model, the method comprising the following operations:

[0006] First, using a charging session dataset of a certain site, the dataset is converted into charging load data at an hourly granularity.

[0007] Second, reversible instance normalization processing is performed on the supervised learning samples to eliminate the influence of dimensional differences on model training.

[0008] Subsequently, the two-channel prompt input is customized for the EV charging scenario, and the workday / weekend token and the time backtracking token are introduced to encode the day type information and the backtracking step information of the historical sequence into the model input in a structured form.

[0009] Again, the time semantics of the embedding layer is re-calibrated, the time decay weight is introduced to the time backtracking token, which decreases with the backtracking step, and the marked token consistent with the predicted anchor day type is enhanced, and the inconsistent one is appropriately suppressed, to explicitly incorporate the scenario prior of "the most recent date is the most important" and "the same type of day is more similar", and improve the stability of the training process through weight clipping and other mechanisms.

[0010] Then, the temporal adapter is constructed to introduce a learnable time-position embedding for the time token, enhancing the model's ability to learn the semantics of the backtracking step.

[0011] Finally, the prediction result is output after the two-channel gated fusion, and the charging load data of Caltech Laboratory is used to verify the model, and the effectiveness of the model is judged by the evaluation index, and the specific process is as follows.

[0012] The application is a kind of electric vehicle charging load prediction method based on pre-training language model, using charging session data set of a certain site, the data set is discrete charging session record, for the first charging session, which includes charging session start time , end time and total energy consumption , so it needs to be converted into hour granularity load data suitable for time series modeling, the process specifically includes:

[0013] Step (1) splits each session by hour, assuming that the charging power during the session is approximately uniform, and according to the overlap length proportion of the session in each hour, the total energy consumption is proportionally allocated to the corresponding hour interval: ; Among them, represents the linear proportion of total energy consumption to hour timestamp, represents the duration of the first session record: ; Among them, represents the overlap length of session and hour interval : ; Among them, denotes the time index, and the time axis is divided into time stamps: ; wherein, denotes the th small time interval.

[0014] Step (2) aggregates the hourly energy consumption after processing all sessions to obtain the hourly load sequence: ; wherein, denotes the load sequence at the hour granularity;

[0015] Step (3) fills the missing time period with zero energy consumption to ensure the continuity of time: ; wherein, denotes the complete continuous hour index set: ; is the starting time of the time series, is the end time of the time series; After the above steps, the electric vehicle charging load data at the hour granularity is obtained. Then, the data is arranged in ascending order according to the time stamp, and a sliding window is used to construct a supervised learning sample: a historical load sequence with a length of is used as the model input, and a load sequence with a length of is used as the prediction target; the data is divided into training set, validation set and test set in proportion.

[0016] Secondly, the EV charging scenario has obvious non-stationarity, including significant amplitude differences in different day types and different seasons. A reversible instance normalization mechanism is introduced at the sample level. Specifically, the historical window sequence of the th sample is marked as: ; wherein denotes the load value at the th time in the historical window, the average value and the standard deviation of the load in the historical window are calculated: ; ; wherein, to prevent the denominator from being 0 when denormalizing, take ;

[0017] Subsequently, in order to explicitly inject the periodicity, day type difference of the EV charging scene and the backtracking step information into the model input, a double-channel prompt input is constructed and structured annotation information is generated synchronously, and the specific steps are as follows:

[0018] Step (1), a numerical sequence channel (channel A) prompt is constructed, the last time of the historical window is taken as the prediction anchor point, and the day type token is determined: workday token (WD) or weekend token (WE). Then the historical sequence is spliced according to time from the nearest to the farthest time backtracking token and the corresponding normalized load value token, wherein The backtracking step coding adopts the coding mode of "the closer, the smaller": the nearest historical time corresponds to , and the earliest historical time corresponds to ; accordingly, the structured token sequence is formed, so that the model can establish a one-to-one correspondence between the backtracking step and the numerical value at the token level;

[0019] Step (2), a calendar-semantic channel (channel B) prompt is constructed, which adopts natural language text to construct semantic prompts, including station background description and calendar element information such as hour, whether it is a weekend, month and date, and can be extended to holiday identification or external event information. This channel is used to supplement the "calendar priori and scene semantics" to improve the adaptability of the model under different running scenarios;

[0020] Step (3), label structured information, further generate two types of annotations aligned with the token position in the numerical sequence channel: one is a time-position identification sequence, which is used to explicitly mark the time index token, and 0 is left for all other non-time tokens; the other is an embedded scaling weight sequence, which is used for time semantic re-labeling in the subsequent embedding layer; the channel A and the channel B are subjected to word segmentation and indexing processing respectively, so as to meet the input format of the pre-trained language model.

[0021] Again, in order to explicitly integrate the scene priori of "the nearer the stronger" and "the more similar the same type of day", and enhance the learnable expression ability of the encoder to the backtracking step, the invention introduces semantic weight scaling in the embedding layer, and the specific steps are as follows:

[0022] Step (1), the time backtracking token in channel A is introduced into the weight function which decreases with the backtracking step: ; wherein, represents an exponential function, is a decay coefficient, and The larger the value, the greater weight the model assigns to the recent history window, thus more inclined to highlight the impact of recent data on the prediction result;

[0023] Step (2), introduce the same type of day enhancement weight , let the prediction anchor point day type be , multiply the token consistent with the anchor point day type by the enhancement coefficient , and multiply the inconsistent token by the weakening coefficient , where ; this mechanism is used to highlight the difference between weekdays / weekends in load distribution, making it easier for the model to learn the similarity between the same type of day.

[0024] Step (3), to avoid excessive scaling of weights leading to unstable training values, the maximum weight is clipped to a preset interval: ; where, and represent the upper and lower bounds of the weight, respectively. Clipping ensures that the weight always remains within a reasonable range, thereby improving the numerical stability and convergence of the model training.

[0025] Then, construct a temporal adapter for the pre-trained language model, which is a learnable time-location embedding mechanism used to inject time series information into the pre-trained encoding backbone "Bert", thereby enhancing the model's sensitivity to historical observations. This mechanism constructs a time embedding table and superimposes and scales the time series information on the basis of embedding to achieve temporal adaptation. The construction and encoding process are described in detail below:

[0026] Step (1) construct the time embedding table, the embedding dimension of the pre-trained encoding backbone is , reconstruct a learnable time embedding table: ; represents a set of real numbers; is a time embedding table, a matrix with a dimension of , each row vector is a learnable embedding vector that is optimized through gradient updates during training. It should be noted that index 0 corresponds to non-time tokens, used to process elements without explicit time-location; index (where ) corresponds to the backtracking step , i.e., the relative position from the current prediction time, the smaller the value, the more recent the data, The larger the value, the further back in time it is. This design ensures that the temporal embedding can capture semantic differences at different historical depths.

[0027] Step (2) performs time-adapted embedding computation on the first... samples, sequence position is The token, after time-series adaptation, is embedded as follows: ; in, This represents the final temporal adaptation embedding vector, which integrates the original semantics and temporal context; It represents the basic embedding vector, which only captures the semantic content of the token and does not contain temporal information; Represents a time-location identifier used to inject specific patterns of relative time-location; Indicates the embedded scaling weight and This is an element-wise multiplicative vector used to control the overall magnitude of the embedding vector. This weight can be calculated using the following formula: ; in, Represents the clipping function, when the... Each token is hour, Cut to within; when Each token is hour, Cut to Inside; in other cases =1;

[0028] Step (3) embeds the temporal adaptation vectors of all sequence positions. The input is fed into the backbone encoding layer of the pre-trained language model "Bert" to obtain the hidden representation sequence of the sequence. ,in It is a contextual representation processed by attention mechanisms and feedforward networks; the advantage of the temporal adaptation structure is that, through learnable temporal embeddings and weight scaling, the model can adaptively capture the dynamic patterns of time series, improving the accuracy of EV charging load prediction.

[0029] Finally, after the dual-channel gating fusion mechanism, the model can adaptively organize the representation information of the two channels. After inverse normalization, inference output can be achieved. The specific steps are as follows:

[0030] Step (1), perform dual-channel gating fusion, for the first For each sample, a representation vector is obtained from channel A and channel B respectively. and These vectors are extracted by the aforementioned encoding process, capturing the static or dynamic characteristics of the time series; subsequently, a gating network generates gating coefficients , whose formula is ; wherein, represents a sigmoid function, is a multi-layer perceptron, represents a two-channel vector splicing, and the fusion representation vector is calculated based on the gating coefficients , whose formula is ; wherein, represents the gating coefficient of channel A, represents the gating coefficient of channel B, represents element-wise multiplication;

[0031] Step (2), entering the training phase, using multi-step mean square error as the main loss function to optimize the model parameters, and defining the loss function as: ; wherein, represents the average mean square error, which is used to quantify the overall deviation of the prediction and promote the model to learn long-term dependencies; represents the normalized prediction value of the model for the th sample at the th step, represents the normalized true load value corresponding to the actual load value at the th step;

[0032] Step (3), entering the optimization process, using AdamW optimizer (a variant of Adam with weight decay) to efficiently handle high-dimensional parameter space, and integrating four strategies to improve the stability and convergence of training, including learning rate adaptive adjustment, early stopping strategy, gradient norm clipping, and outlier skip protection, which will together ensure an efficient training process and avoid overfitting and numerical anomaly problems;

[0033] Step (4), entering the inference phase, after optimization in the previous step, the optimal parameter combination of the model can be obtained, and after inverse normalization, the model outputs the predicted value of the EV charging load , which can be used for actual decision-making such as power grid dispatching;

[0034] Step (5), perform prediction index evaluation, to quantify the model performance, in the present invention, mean absolute error (MAE), root mean square error (RMSE) and determination coefficient (R 2As evaluation indexes, the formulas of the three are: ; ; ; Wherein, is the total number of samples in the test set, represents the load prediction value of the first sample after the inverse normalization processing in the first step, represents the true load value of the first sample in the first step, represents the true load value of the first sample in the first step, represents the load average of all samples. Compared with the prior art, the beneficial effects of the present application are: (1) The present application explicitly injects temporal prior in the embedding layer, improving the sensitivity of the model to recent historical data; (2) The present application introduces workday / weekend labels and same-type day enhancement coefficients, making the model more accurately capture the distribution difference between workdays and weekends; (3) The present application uses Temporal Adapter to replace the traditional static position encoding with a learnable time embedding, better adapting to the semantic needs of different backtracking steps; (4) The dual-channel gating mechanism used in the present application adaptively selects a more reliable channel in the sample, significantly enhancing the model's generalization ability and noise resistance.

[0035] BRIEF DESCRIPTION OF DRAWINGS

[0036] is the overall flowchart of the electric vehicle charging load prediction method based on the pre-trained language model of the present application;

[0037] Figure 1 is the time-position and value prompt embedding structure diagram in the numerical sequence channel of the present application;

[0038] Figure 2 is the time-position and value prompt embedding structure diagram in the numerical sequence channel of the present application;

[0039] Figure 3 is the embedding layer structure diagram of the temporal adaptation encoder of the present application;

[0040] Figure 4 is the structure diagram of the dual-channel gating fusion mechanism of the present application. DETAILED DESCRIPTION

[0041] The present application will be further described below in conjunction with the drawings, and the examples are used to more clearly describe the technical solutions of the present application, and the scope of application of the present application is not limited thereto, and can be applied to multiple fields.

[0042] As Figure 1 ​​As shown, the present application provides an electric vehicle charging load prediction method based on a pre-trained language model, which is specifically divided into the following steps:

[0043] First, the ACN data of Caltech Laboratory Charging Session Dataset (ACN data) is used, which records discrete charging sessions. The first charging session is selected, which includes the start time , end time and total energy consumption , so it needs to be converted into hourly granularity load data suitable for time series modeling, which specifically includes:

[0044] Step (1) splits each session by hour, assuming that the charging power during the session is approximately uniform, and according to the proportion of the overlapping length of the session in each hour, the total energy consumption is proportionally allocated to the corresponding hour interval: ; wherein, represents the proportionally linear allocation of total energy consumption to the hour timestamp, represents the duration of the th session record: ; wherein, represents the overlapping length of the session and the hour interval : ; wherein, represents the time index, which divides the time axis into time stamps per hour: ; wherein, represents the th small time interval.

[0045] Step (2) aggregates the hourly energy consumption after processing all sessions to obtain the hourly load sequence: ; wherein, represents the hourly granularity load sequence;

[0046] Step (3) fills the missing time period with zero energy consumption to ensure the continuity of time: ; wherein, represents the complete and continuous hour index set: ; In this example, is 0 o'clock on January 1, 2019, is 23 o'clock on December 31, 2019; after the above steps, the electric vehicle charging load data with hourly granularity is obtained. Then, the data is arranged in ascending order according to the time stamp, and a sliding window is used to construct a supervised learning sample: a historical load sequence with a length of is taken as the model input, and a load sequence with a length of is taken as the prediction target; in this example, , are taken as example parameters, which can be configured according to application requirements; the data is divided into a training set, a validation set and a test set according to time sequence, and a ratio of 70%:15%:15% is used to avoid data leakage.

[0047] Secondly, the EV charging scenario in this example has obvious non-stationarity, specifically including significant amplitude differences in different day types and different seasons. A reversible instance normalization mechanism is introduced at the sample level. Specifically, the historical window sequence of the i-th sample is marked as: ; wherein represents the load value at the i-th time point in the historical window, the average value and the standard deviation of the load in the historical window are calculated: ; ; wherein, to prevent the denominator from being 0 when denormalizing, take ;

[0048] Subsequently, as shown in Figure 2 , in order to explicitly inject the periodicity, day type difference and backtracking step information of the EV charging scenario into the model input, this example constructs a double-channel prompt input and synchronously generates structured annotation information, and the specific steps are as follows:

[0049] Step (1), construct a numerical sequence channel (channel A) prompt. Take the last time point of the historical window as the prediction anchor point to determine the day type token: weekday token (WD) or weekend token (WE). Then concatenate the time backtracking token and the corresponding normalized load value token in chronological order from the most recent to the most distant, wherein is the backtracking step encoding, which adopts the encoding mode of “the closer, the smaller”: the most recent historical time point corresponds to , and the earliest historical time point corresponds to ​​According to the structured token sequence, the model can establish a one-to-one correspondence between the "backtracking step-value" at the token level;

[0050] Step (2), construct a calendar-semantic channel (channel B) prompt, which uses natural language text to construct a semantic prompt containing station background description and calendar element information such as hours, whether it is a weekend, month, and date, and can be extended to holiday identification or external event information. This channel is used to supplement the "calendar priori and scene semantics" to improve the adaptability of the model in different running scenarios;

[0051] Step (3), label structured information, further generate two types of labels aligned with token positions in the numerical sequence channel: one is a time-position identification sequence, which is used to explicitly mark time index tokens, leaving 0 to all other non-time tokens; The other is an embedded scaling weight sequence, which is used for subsequent time semantic re-labeling of the embedding layer; Perform word segmentation and indexing processing on channels A and B to meet the input format of the pre-trained language model.

[0052] Again, to explicitly incorporate the scene priori of "stronger recent cause" and "similar to the same type of day", and to enhance the learnable expression ability of the encoder to the backtracking step, this example introduces semantic weight scaling in the embedding layer, as follows:

[0053] Step (1), for the time backtracking tokens in channel A Introduce a weight function that decreases with the backtracking step: ; Where, is the decay coefficient, and The larger the value, the greater the weight the model gives to the more recent historical window, thus giving more weight to recent data on the prediction result;

[0054] Step (2), introduce the same type of day enhancement weight , let the day type of the prediction anchor point be , multiply the token consistent with the anchor point day type by the enhancement coefficient , and multiply the inconsistent one by the weakening coefficient , where ; This mechanism is used to highlight the differences in load distribution between weekdays / weekends, making it easier for the model to learn the similarities between the same type of day.

[0055] Step (3), to avoid excessive scaling of weights leading to unstable training values, the maximum weight is clipped to a preset interval: ; where, and respectively represent the upper and lower bounds of the weight, and the clipping ensures that the weights are always kept within a reasonable range, thus improving the numerical stability and convergence of the model training.

[0056] Then, a temporal adapter is constructed for the pre-trained language model, which is a learnable time-position embedding mechanism for injecting time series information into the pre-trained encoding backbone "Bert", thereby enhancing the model's sensitivity to historical observations. This mechanism achieves temporal adaptation by constructing a time embedding table and superimposing and scaling the time series information on the basic embedding, as shown in Figure 3 The construction and encoding process are described in detail below:

[0057] Step (1) constructs a time embedding table. The embedding dimension of the pre-trained encoding backbone is , and a learnable time embedding table is reconstructed: ; denotes a set of real numbers; is a time embedding table, which is a matrix with a dimension of Each row vector is a learnable embedding vector that is optimized through gradient updates during training. It is worth noting that the index 0 corresponds to non-time tokens, which are used to process elements without explicit time-position; the index (where ) corresponds to the backtracking step , i.e., the relative historical position from the current prediction time, , the smaller the value, the more recent the period, , the larger the value, the more distant the period. This design ensures that the time embedding can capture the semantic differences of different historical depths;

[0058] Step (2) calculates the embedding after temporal adaptation. For the th sample, the token at sequence position , the embedding after temporal adaptation is represented as: ; where, denotes the final temporal adaptation embedding vector, which integrates the original semantics and temporal context; denotes the basic embedding vector, which only captures the semantic content of the token and does not contain temporal information; denotes the time-position identifier, which is used to inject specific patterns of relative time-position; denotes the embedding scaling weight and , which is an element-wise product vector used to control the overall magnitude of the embedding vector, and the weight can be calculated by the following formula: ; wherein, represents a clipping function, when the first token is , it is clipped to ; when the first token is , it is clipped to ; otherwise is 1;

[0059] Step (3) inputs the time-adaptive embedding vector of all sequence positions into the backbone encoding layer of the pre-trained language model "Bert" to obtain the hidden representation sequence of the sequence, wherein is the context representation processed by the attention mechanism and the feedforward network; the advantage of the time-adaptive structure is that the model can adaptively capture the dynamic pattern of the time series through the learnable time embedding and weight scaling, thereby improving the accuracy of the EV charging load prediction.

[0060] Finally, after the double-channel gating fusion mechanism, the model can adaptively organize the representation information of the two channels, and after the inverse normalization processing, the inference output can be realized, as shown in Figure 4 , the specific steps are as follows:

[0061] Step (1), double-channel gating fusion is performed, for the first sample, the representation vectors and are obtained from channel A and channel B respectively, which are extracted through the foregoing encoding process and capture the static or dynamic features of the time series; then, a gating network is used to generate the gating coefficient , and the calculation formula is as follows: ; wherein, represents a sigmoid function, is a multilayer perceptron, represents the vector splicing of the two channels, and the fused representation vector is calculated based on the gating coefficient, and the formula is as follows: ; wherein, represents the gating coefficient of channel A, represents the gating coefficient of channel B, ​​represents element-wise multiplication;

[0062] Step (2), entering the training stage, adopts multi-step mean square error as the main loss function to optimize the model parameters, and the loss function is defined as: ; wherein, represents the average mean square error, which is used to quantify the overall deviation of the prediction and promote the model to learn long-term dependencies; represents the normalized prediction value of the model to the th sample at the th step, represents the normalized true load value corresponding to the actual load value at the th step;

[0063] Step (3), entering the optimization process, adopts AdamW optimizer (a variant of Adam with weight decay) to efficiently process high-dimensional parameter space. To improve the stability and convergence of training, four strategies are integrated, including learning rate adaptive adjustment, early stopping strategy, gradient norm clipping, and outlier skip protection, which will together ensure an efficient training process and avoid overfitting and numerical anomaly problems;

[0064] Step (4), entering the inference stage, the optimal parameter combination of the model can be obtained after optimization in the previous step. After inverse normalization, the model outputs Step EV charging load prediction value , which can be used for actual decision-making such as power grid dispatching;

[0065] Step (5), perform prediction index evaluation. To quantify the model performance, in this example, mean absolute error (MAE), root mean square error (RMSE), and coefficient of determination (R 2 ) are selected as evaluation indexes, and their formulas are: ; ; ; wherein, is the total number of samples in the test set, represents the load prediction value of the th sample after inverse normalization at the th step, represents the true load value of the th sample at the th step, represents the average load value of all samples.

[0066] The prediction result is shown in Table 1. In the case that other conditions of the model are the same, the model is compared with ARIMA and LSTM models, EV charging load in a test set is predicted, and the effectiveness of the model is judged by evaluation indexes;

[0067] Table 1 shows the results of the model evaluation index. Model MAE RMSE [R 2 ]] ARIMA 4.75 8.49 0.4726 LSTM 3.52 5.37 0.7887 Invention prediction model 3.18 5.18 0.8036 As can be seen from the results in Table 1, the effect achieved by the prediction model of the application is optimal, and the benefits of the application can be proved.

Claims

1. A method for predicting electric vehicle charging load based on a pre-trained language model, characterized in that, A temporal prediction network is constructed based on the pre-trained language model "Bert", and prediction is achieved through dual-channel cueing and a temporal adapter structure. The method includes the following steps: Step 1: Perform data preprocessing, convert the raw electric vehicle (EV) charging session data into hourly load sequences, and remove outliers and handle missing values ​​in the sequences. Step 2: A sliding time window is used to construct supervised learning samples. The input sequence consists of multiple consecutive historical load values, and the prediction target is the load values ​​at multiple consecutive subsequent time points. The mean and standard deviation of the load values ​​are calculated and then reversible instance normalization is performed to eliminate the influence of dimensional differences on model training. Step 3: A customized dual-channel prompt is constructed for the EV charging scenario. The numerical sequence channel prompt uses the last moment of the historical load input sequence as the prediction time anchor point, and introduces an explicit time-location identifier for each historical moment. Based on the day type of the prediction anchor point, a workday identifier WD (Workday) or a weekend identifier WE (Weekend) is generated and embedded into the text prompt. The normalized historical load value, time-location identifier, and day type identifier are combined to form the first prompt channel for describing the numerical sequence information. The second channel is a calendar-semantic prompt channel, which mainly includes semantic description text of the charging station and calendar feature text. Then, word segmentation and indexing are performed to make it meet the input format of the pre-trained language model. Step 4: Introduce semantic weight scaling in the embedding layer. For the numerical sequence channel prompt, generate a token (lexicon, the smallest semantic or syntactic unit used by the pre-trained language model when processing text) weight matrix and a time-location identifier. The token weight matrix assigns weights to the time index token and the day type marker token. The time-location identifier can make the time step number corresponding to the time index token a non-zero value, and the non-time token a zero value. Step 5: Construct a temporal adaptation structure with learnable time-position embedding capabilities for the pre-trained language model, which is used to inject the dual-channel cues into the pre-trained language model "Bert" encoder with shared parameters to obtain dual-channel representation vectors; Step six: Perform dual-channel representation vector gating fusion, and output multi-step load prediction values ​​at a normalized scale through a multi-step regression prediction head; use the mean and standard deviation described in step two to inversely normalize the prediction values ​​to obtain the multi-step prediction results of EV charging load at the actual scale.

2. The electric vehicle charging load prediction method based on a pre-trained language model according to claim 1, characterized in that, The dual-channel prompt construction process includes: Step (1): Construct a numerical sequence channel (channel A) hint, using the last moment of the historical window as the prediction anchor point, and determine the day type token: weekday token (WD) or weekend token (WE); then, splice the historical sequence with time backtracking tokens from most recent to oldest. With the corresponding normalized load value token, where Represents a time index marker. For backtracking step size encoding, a "closer, smaller" encoding method is adopted: the most recent historical moment corresponds to... The earliest historical moment corresponds to This forms a structured token sequence, enabling the model to establish a one-to-one correspondence between "backtracking step size and numerical value" at the token level. Step (2) Construct a calendar-semantic channel (channel B) prompt. This channel uses natural language text to construct semantic prompts, which include a description of the site background and calendar element information, such as hour, whether it is a weekend, month and date, and can be extended with holiday identifiers or external event information. This channel is used to supplement "calendar priors and scene semantics" to improve the model's adaptability in different operating scenarios. Step (3) Label the structured information and further generate two types of labels aligned with the token position in the numerical sequence channel: one is a time-position identifier sequence, which is used to explicitly mark the time index token and leave 0 for all other non-time tokens; the other is an embedding scaling weight sequence, which is used for the time semantic recalibration of the subsequent embedding layer.

3. The electric vehicle charging load prediction method based on a pre-trained language model according to claim 1, characterized in that, The semantic weight scaling step described in step four includes: Step (1), for the time backtracking token in channel A Introduce a weight function that decreases with the backtracking step size: ； in, Represents an exponential function. It is the attenuation coefficient. and The larger the value, the greater the weight the model assigns to more recent historical windows, thus tending to highlight the impact of recent data on the prediction results; Step (2), introduce daily enhanced weights of the same type. Let the daily type of the prediction anchor point be... Multiply the token with the same date type as the anchor point by an enhancement factor. Multiply the inconsistent ones by a weakening factor ,in ; Step (3): To avoid excessive scaling of weights leading to unstable training values, the final weights are... Crop to preset area: ； in, and These represent the upper and lower bounds of the allowed weights, respectively. Trimming ensures that the weights are always kept within a reasonable range. The numerical sequence channel prompt is generated according to the structure of "daily type token, historical sequence structure fragment, multiple sets of time index tokens and corresponding normalized load value tokens"; and the token weight matrix weights the structure tokens, but does not force the numerical tokens to be weighted one by one, in order to reduce the risk of training instability.

4. The electric vehicle charging load prediction method based on a pre-trained language model according to claim 1, characterized in that, The construction steps of the timing adaptation structure include: Step (1) involves constructing the time embedding table. For the historical step size, the embedding dimension of the pre-trained encoding backbone is... Reconstruct a learnable time embedding table: ； Represents the set of real numbers; The time-embedded table is a dimension-based table. The matrix, where each row vector is a learnable embedding vector, is optimized through gradient updates during training. It should be noted that index 0 corresponds to a non-time token, used to handle elements without a clear time-position relationship; index 1... (in ), corresponding backtracking step size That is, the relative historical position from the current prediction time. The smaller the value, the more recent the information. The larger the value, the further into the future; Step (2) performs time-adapted embedding computation on the first... samples, sequence position is The token, after time-series adaptation, is embedded as follows: ； in, This represents the final temporal adaptation embedding vector, which integrates the original semantics and temporal context; It represents the basic embedding vector, which only captures the semantic content of the token and does not contain temporal information; Represents a time-location identifier used to inject specific patterns of relative time-location; Indicates the embedded scaling weight and This is an element-wise multiplicative vector used to control the overall magnitude of the embedding vector. This weight can be calculated using the following formula: ； in, Represents the clipping function, when the... Each token is hour, Cut to within; when Each token is hour, Cut to Inside; in other cases =1; Step (3) embeds the temporal adaptation vectors of all sequence positions. The input is fed into the backbone encoding layer of the pre-trained language model "Bert" to obtain the hidden representation sequence of the sequence. ,in It is a contextual representation processed by attention mechanisms and feedforward networks.

5. The electric vehicle charging load prediction method based on a pre-trained language model according to claim 1, characterized in that, In the dual-channel representation vector gating fusion process, for the first... For each sample, a representation vector is obtained from channel A and channel B respectively. and These vectors, extracted through the aforementioned encoding process, can capture the static or dynamic features of time series data. This represents the embedding dimension of the pre-trained encoding backbone; subsequently, gating coefficients are generated through a gating network. The calculation formula is as follows: ； in, This represents the sigmoid function. It is a multilayer perceptron. This represents the concatenation of vectors from two channels, with the fused representation vector calculated based on the gating coefficients. The formula is: ; in, This represents the gating coefficient for channel A. This represents the gating coefficient for channel B. This indicates element-wise multiplication.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 5.

Images