Source load prediction method, device, equipment and medium based on multimodal data

Abstract

The embodiment of the present invention discloses a source load prediction method, device, equipment and medium based on multimodal data. The method includes: collecting multimodal initial data, and preprocessing the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data; fusing the text data and cloud image video data to obtain fused feature data, and position-encoding the fused features through an informer model to obtain position-encoded data; using the position-encoded data and the fused feature data together as a training data set, and grouping the training data set to obtain multiple groups of training sub-data sets; using the multiple groups of training sub-data sets to train the informer model to obtain a source load prediction model; inputting the obtained multimodal data to be predicted into the source load prediction model to obtain a prediction result. The embodiment of the present application occupies less memory and has high computational efficiency when performing source load prediction.

Description

Technical Field

[0001] The present invention relates to the field of source load prediction technology, and in particular to a source load prediction method, device, equipment and medium based on multimodal data. Background Art

[0002] Photovoltaic (PV) power generation uses the photovoltaic effect of semiconductor materials to directly convert solar energy into electricity. However, its volatility, grid adaptability, and sustainability issues still need to be addressed through technological innovation and system optimization. PV power generation and load curve forecasting is a key technology for new energy power systems, and its accuracy directly impacts the economic viability and security of the grid. With advances in AI and the integration of multi-source data, PV power generation and load curve forecasting methods are evolving from traditional statistical models to deep learning and physical information fusion. In the future, PV power generation and load forecasting will become more intelligent and automated, supporting the stable operation of power grids with a high proportion of renewable energy.

[0003] Time Series Networks (TSNs) are neural network structures specifically designed for processing time series data in deep learning. They effectively capture the temporal dependencies and dynamic changes in photovoltaic power generation data. From the initial exploration of RNNs (recurrent neural networks), to the optimization of LSTMs (long short-term memory) and GRUs (gated recurrent units), and finally to the innovations of Transformers and diffusion models, TSNs have consistently centered around two core issues: modeling and dynamic adaptability. The Transformer, a deep learning model based on the self-attention mechanism, was originally developed for natural language processing and has recently demonstrated outstanding performance in time series forecasting. However, this model also has limitations, such as high computational resource consumption, high computational complexity of self-attention, low computational efficiency, high training costs, and memory usage. Summary of the Invention

[0004] The embodiments of the present invention provide a source load prediction method, apparatus, device and medium based on multimodal data, aiming to solve the problems of large memory usage and low computational efficiency of existing source load prediction methods.

[0005] In a first aspect, an embodiment of the present invention provides a source load prediction method based on multimodal data, which includes:

[0006] Collecting multimodal initial data and preprocessing the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data;

[0007] Performing feature fusion on the text data and the cloud image video data to obtain fused feature data, and position encoding the fused feature using an informer model to obtain position encoding data;

[0008] Taking the position coding data and the fused feature data together as a training data set, and grouping the training data set to obtain multiple groups of training sub-data sets;

[0009] Using multiple sets of training sub-data sets to train the Informer model to obtain a source-load prediction model;

[0010] The obtained multimodal data to be predicted is input into the source-load prediction model for prediction to obtain a prediction result.

[0011] In a second aspect, an embodiment of the present invention further provides a source load prediction device based on multimodal data, comprising:

[0012] An acquisition and processing unit, configured to acquire multimodal initial data and preprocess the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data;

[0013] A fusion encoding unit, configured to perform feature fusion on the text data and the cloud image video data to obtain fused feature data, and position encode the fused feature using an informer model to obtain position encoded data;

[0014] a grouping unit, configured to use the position coding data and the fused feature data together as a training data set, and group the training data set to obtain a plurality of training sub-data sets;

[0015] A training unit, configured to train the Informer model using multiple sets of training sub-data sets to obtain a source-load prediction model;

[0016] The prediction unit is used to input the acquired multimodal data to be predicted into the source-load prediction model to perform prediction to obtain a prediction result.

[0017] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0019] The embodiment of the present invention provides a source load prediction method, device, equipment and medium based on multimodal data. The method includes: collecting multimodal initial data and preprocessing the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data; fusing the text data and the cloud image video data to obtain fused feature data, and position-encoding the fused feature through an informer model to obtain position-encoded data; using the position-encoded data and the fused feature data as a training data set, and grouping the training data set to obtain multiple sets of training sub-data sets; using the multiple sets of training sub-data sets to train the informer model to obtain a source load prediction model; and inputting the acquired multimodal data to be predicted into the source load prediction model to obtain a prediction result. The technical solution of the embodiment of the present invention first performs feature fusion on the text data and cloud image video data in the multimodal target data to obtain fused feature data, and position encodes the fused feature data through the informer model to obtain position encoded data; then, multiple sets of training sub-data sets are generated according to the position encoded data and the fused feature data, and the informer model is trained with the multiple sets of training sub-data sets to obtain a source-load prediction model; finally, the acquired multimodal data to be predicted is input into the source-load prediction model for prediction to obtain a prediction result. Since the source-load prediction model is a model obtained by training the informer model with the multiple sets of training sub-data sets generated by the multimodal target data, the generalization ability and robustness of the model are improved. When the source-load prediction model is used to perform source-load prediction, less memory is occupied and the computational efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic flow chart of a source load prediction method based on multimodal data provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a sub-process of a source load prediction method based on multimodal data provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of another sub-process of a source load prediction method based on multimodal data provided by an embodiment of the present invention;

[0024] Figure 4A schematic diagram of another sub-process of a source load prediction method based on multimodal data provided by an embodiment of the present invention;

[0025] Figure 5 This is an overall schematic diagram of the source-load prediction model training provided by an embodiment of the present invention;

[0026] Figure 6 A schematic block diagram of a source load prediction device based on multimodal data provided by an embodiment of the present invention;

[0027] Figure 7 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0030] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0031] It should be further understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0032] As used in this specification and the appended claims, the term “if” can be interpreted as “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [described condition or event] is detected” can be interpreted as meaning “upon determination” or “in response to determining” or “upon detection of [described condition or event]” or “in response to detecting [described condition or event],” depending on the context.

[0033] See also Figure 1 , Figure 1 FIG is a flow chart of a method for predicting source load based on multimodal data provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110-S150.

[0034] S110 , collecting multimodal initial data, and preprocessing the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data.

[0035] In an embodiment of the present invention, multimodal initial data is collected, wherein the multimodal initial data includes initial text data and initial cloud image video data. The multimodal data is preprocessed to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data. It can be understood that the text data is obtained by preprocessing the initial text data, and the cloud image video data is obtained by preprocessing the initial cloud image video data. It should be noted that the preprocessing includes deduplication, denoising, data anomaly removal, data missingness removal, and normalization. The initial text data includes user energy profile data, date attribute information, historical source-load curves, equipment inventory, and externally accessed weather forecast data. The text data is the preprocessed user energy profile data, date attribute information, historical source-load curves, equipment inventory, and externally accessed weather forecast data. It should also be noted that the collection of the initial text data and initial cloud image video data is performed on a daily basis. The initial cloud image video data is cloud image video data related to weather.

[0036] S120, performing feature fusion on the text data and the cloud image video data to obtain fused feature data, and position encoding the fused feature through an Informer model to obtain position encoded data.

[0037] In an embodiment of the present invention, after obtaining the text data and the cloud image video data, the text data and the cloud image video data are subjected to feature fusion to obtain fused feature data, and the fused feature data is positionally encoded using an Informer model to obtain positionally encoded data. Specifically, the fused feature data is positionally encoded using the PositionalEncoding algorithm in the Informer model to obtain positionally encoded data. It should be noted that Informer is a deep learning model based on an improved Transformer architecture, designed specifically for Long Sequence Time-Series Forecasting (LSTF) tasks.

[0038] In one embodiment, if Figure 2 As shown, step S120 may specifically include steps S121-S122:

[0039] S121. Extracting features of the cloud image video data through a CNN network to obtain cloud image video features;

[0040] S122: Fusing the cloud image video features with the text data to obtain the fused feature data.

[0041] In an embodiment of the present invention, the CNN network is a convolutional neural network, a deep learning model specifically designed for processing grid-like data (such as images, videos, audio spectrograms, etc.). The CNN network includes convolutional layers and pooling layers. The convolutional layers include a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer. Extracting features from the cloud image video data using the CNN network to obtain cloud image video features includes: sequentially performing dimensionality reduction processing on the cloud image video data using the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer to obtain reduced-dimensional cloud image video data; and performing maximum pooling processing on the reduced-dimensional cloud image video data using the pooling layer to obtain the cloud image video features. It should be noted that the first convolutional layer is a 3*48 convolutional layer, the second convolutional layer is a 48*128 convolutional layer, the third convolutional layer is a 128*256 convolutional layer, and the fourth convolutional layer is a 256*512 convolutional layer. The pooling layer is a 2*2 MaxPool pooling layer. It should also be noted that the fusion of the cloud image video features and the text data is specifically to splice the cloud image video features and the text data.

[0042] S130: Use the position coding data and the fused feature data together as a training data set, and group the training data set to obtain multiple training sub-data sets.

[0043] In an embodiment of the present invention, after obtaining the position coding data, the position coding data and the fusion feature data are used together as a training data set, and the training data set is grouped to obtain multiple groups of training sub-data sets. It should be noted that in this embodiment, the training data set is grouped into groups of 8 data to obtain multiple groups of training sub-data sets, and the training sub-data sets include source-load training data and label data. In the training sub-data set, the first 7 data are the source-load training data, and the last data, that is, the 8th data, is the label data. It is understandable that the last data can be predicted by the first 7 data. It should also be noted that in other embodiments, the training data set is also grouped into groups of 10 data to obtain multiple groups of training sub-data sets. In the training sub-data set, the first 9 data are the source-load training data, and the last data, that is, the 10th data, is the label data. That is, the number of data in the training sub-data set is set according to actual needs and is not specifically limited.

[0044] S140: Using multiple sets of the training sub-data sets to train the Informer model to obtain a source-load prediction model.

[0045] In an embodiment of the present invention, the training sub-dataset includes source-load training data and label data, and both the source-load training data and the label data include the fusion feature data and the position coding data. Figure 3 As shown, step S140 may specifically include steps S141-S142: S141, for each group of the training sub-datasets, concatenate the fused feature data and the position coding data in the source-load training data to obtain target source-load training data; S142, train the informer model using the target source-load training data and the label data to obtain the source-load prediction model. It can be understood that when training the informer model, the data sets are input in groups, that is, each time a group of the training sub-datasets is input, the attention mechanism in the informer model will calculate the similarity between each data vector in the training sub-dataset and other vectors, and then obtain the final prediction result based on the similarity. The prediction result and the label data in the training sub-dataset are used to calculate the source-load loss. The parameters in the informer model are updated through backpropagation based on the source-load loss, so that the informer model effectively fits the training data, thereby making the prediction result more consistent with the expected result. It should be noted that in this embodiment, the informer model includes a self-attention mechanism module, a sparse attention module, a multi-head attention module, and a cross-attention module.

[0046] In one embodiment, if Figure 4As shown, step S142 may specifically include steps S1421-S1426:

[0047] S1421, inputting the target source load training data into the self-attention mechanism module to perform data fusion to obtain a dictionary set;

[0048] S1422, inputting the dictionary set into the sparse attention module to perform dimensionality reduction to obtain a reduced dimensionality dictionary set;

[0049] S1423, inputting the dimension reduction dictionary set into the multi-head attention module to perform information aggregation to obtain an aggregated dictionary set;

[0050] S1424, inputting the aggregated dictionary set into the cross attention module to calculate the similarity between the aggregated dictionary set and the target source-load training data to obtain source-load similarity;

[0051] S1425, calculating a source-load loss value according to the source-load similarity, the target source-load training data, and the label data;

[0052] S1426. Update the parameters of the Informer model through back propagation according to the source-load loss value to obtain the source-load prediction model.

[0053] In an embodiment of the present invention, for each group of the training sub-data sets, the number of training epochs is set to 20 times. During the training process, the model with the smallest source load loss is saved, and the initial learning rate is defined as 0.00001. After the model training starts, the fused feature data and the position code data in the source load training data are first spliced to obtain the target source load training data and input into the self-attention mechanism module. The self-attention mechanism module performs data aggregation to obtain a dictionary set, and the dictionary set is input into the sparse attention module for data dimensionality reduction to obtain a reduced dimensionality dictionary set. The reduced dimensionality dictionary set is then input into the multi-head attention module. The multi-head attention module performs information aggregation on the reduced dimensionality dictionary set to obtain an aggregated dictionary set, and the aggregated dictionary set is input into the cross-attention module to calculate the similarity between the aggregated dictionary set and the target source load training data to obtain source load similarity. A source load prediction result is determined based on the source load similarity and the target source load training data. The error loss between the source load prediction result and the labeled result in the label data is calculated to obtain the source load loss value. Based on the source load loss value, the parameters of the informer model are updated through back propagation to obtain the source load prediction model.

[0054] S150: Input the acquired multimodal data to be predicted into the source-load prediction model for prediction to obtain a prediction result.

[0055] In this embodiment of the present invention, after the source-load prediction model is trained, it is saved and a Python RESTful service is written to load the model. An external system then calls the RESTful service interface, passing in the multimodal data to be predicted, to obtain a prediction result. It should be noted that in this embodiment, the multimodal data to be predicted includes text data to be predicted and cloud image video data to be predicted. The prediction results include a photovoltaic power generation curve and a load curve.

[0056] See also Figure 5 , Figure 5 This is an overall schematic diagram of the source-load prediction model training provided by an embodiment of the present invention. Figure 5 As shown, the features of cloud image video data are extracted through a CNN network to obtain cloud image video features, the cloud image video features and text data are fused to obtain fused feature data, the fused feature data are position-encoded to obtain position-encoded data, the position-encoded data and the fused feature data are used together as a training data set, and the training data set is grouped to obtain multiple groups of training sub-data sets. The Informer model is trained using the multiple groups of training sub-data sets to obtain a source-load prediction model.

[0057] To sum up, in this embodiment, since the source load prediction model is a model obtained by training the Informer model based on multiple sets of training sub-data sets generated by multimodal target data, the generalization ability and robustness of the model are improved. When the source load prediction model is used for source load prediction, it occupies less memory and has high computational efficiency.

[0058] Figure 6 FIG is a schematic block diagram of a source load prediction device 200 based on multimodal data provided by an embodiment of the present invention. Figure 6 As shown, corresponding to the above source load prediction method based on multimodal data, the present invention also provides a source load prediction device 200 based on multimodal data. The source load prediction device 200 based on multimodal data includes a unit for executing the above source load prediction method based on multimodal data, and the device can be configured in a computer device. Specifically, please refer to Figure 6 The source load prediction device 200 based on multimodal data includes an acquisition and processing unit 201, a fusion coding unit 202, a grouping unit 203, a training unit 204 and a prediction unit 205.

[0059] Among them, the acquisition and processing unit 201 is used to acquire multimodal initial data, and preprocess the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data; the fusion coding unit 202 is used to perform feature fusion on the text data and the cloud image video data to obtain fused feature data, and position encode the fused feature through the informer model to obtain position encoding data; the grouping unit 203 is used to use the position encoding data and the fused feature data as a training data set, and group the training data set to obtain multiple groups of training sub-data sets; the training unit 204 is used to train the informer model using multiple groups of training sub-data sets to obtain a source-load prediction model; the prediction unit 205 is used to input the acquired multimodal data to be predicted into the source-load prediction model for prediction to obtain a prediction result.

[0060] In some embodiments, such as this embodiment, the training unit 204 includes a splicing unit and a training sub-unit.

[0061] Among them, the splicing unit is used to splice the fused feature data and the position coding data in the source load training data for each group of the training sub-data sets to obtain the target source load training data; the training sub-unit is used to use the target source load training data and the label data to train the Informer model to obtain the source load prediction model.

[0062] In some embodiments, such as the present embodiment, the training sub-units include a data fusion unit, a dimensionality reduction unit, an information aggregation unit, a first calculation unit, a second calculation unit, and an update unit.

[0063] Among them, the data fusion unit is used to input the target source load training data into the self-attention mechanism module for data fusion to obtain a dictionary set; the dimensionality reduction unit is used to input the dictionary set into the sparse attention module for dimensionality reduction to obtain a reduced dimensionality dictionary set; the information aggregation unit is used to input the reduced dimensionality dictionary set into the multi-head attention module for information aggregation to obtain an aggregated dictionary set; the first calculation unit is used to input the aggregated dictionary set into the cross-attention module to calculate the similarity between the aggregated dictionary set and the target source load training data to obtain source load similarity; the second calculation unit is used to calculate the source load loss value based on the source load similarity, the target source load training data and the label data; the update unit is used to update the parameters of the Informer model through back propagation according to the source load loss value to obtain the source load prediction model.

[0064] In some embodiments, such as this embodiment, the second calculation unit includes a determination unit and a third calculation unit.

[0065] Among them, the determination unit is used to determine the source load prediction result based on the source load similarity and the target source load training data; the third calculation unit is used to calculate the error loss between the source load prediction result and the annotation result in the label data to obtain the source load loss value.

[0066] In some embodiments, such as this embodiment, the fusion encoding unit 202 includes a feature extraction unit and a fusion unit.

[0067] Among them, the feature extraction unit is used to extract the features of the cloud image video data through the CNN network to obtain the cloud image video features; the fusion unit is used to fuse the cloud image video features with the text data to obtain the fused feature data.

[0068] In some embodiments, such as this embodiment, the feature extraction unit includes a dimensionality reduction processing unit and a pooling processing unit.

[0069] Among them, the dimensionality reduction processing unit is used to perform dimensionality reduction processing on the cloud image video data through the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer in sequence to obtain reduced dimensionality cloud image video data; the pooling processing unit is used to perform maximum pooling processing on the reduced dimensionality cloud image video data through the pooling layer to obtain the cloud image video features.

[0070] The specific implementation of the source load prediction device 200 based on multimodal data in the embodiment of the present invention corresponds to the above-mentioned source load prediction method based on multimodal data, which will not be described in detail here.

[0071] The above-mentioned source load prediction device based on multimodal data can be implemented in the form of a computer program. The computer program can be used in Figure 7 Runs on the computer equipment shown.

[0072] See also Figure 7 , Figure 7 3 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 300 is a device having a source load prediction function based on multimodal data.

[0073] See Figure 7 The computer device 300 includes a processor 302 , a memory, and a network interface 305 connected via a system bus 301 , wherein the memory may include a storage medium 303 and an internal memory 304 .

[0074] The storage medium 303 may store an operating system 3031 and a computer program 3032. When the computer program 3032 is executed, the processor 302 may execute a source load prediction method based on multimodal data.

[0075] The processor 302 is used to provide computing and control capabilities to support the operation of the entire computer device 300.

[0076] The internal memory 304 provides an environment for the operation of the computer program 3032 in the storage medium 303. When the computer program 3032 is executed by the processor 302, the processor 302 can execute a source load prediction method based on multimodal data.

[0077] The network interface 305 is used to communicate with other devices through the network. Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 300 to which the solution of the present application is applied. The specific computer device 300 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0078] The processor 302 is configured to run a computer program 3032 stored in the memory to implement any embodiment of the above-mentioned source load prediction method based on multimodal data.

[0079] It should be understood that in the embodiment of the present application, the processor 302 may be a central processing unit (CPU), and the processor 302 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0080] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0081] Therefore, the present invention further provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform any embodiment of the above-mentioned source load prediction method based on multimodal data.

[0082] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0083] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0084] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0085] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0086] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (such as a personal computer, terminal, or network device) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0087] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, to the extent such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to encompass such changes and modifications.

[0089] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A source load prediction method based on multimodal data, characterized in that: include: Collecting multimodal initial data and preprocessing the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data, wherein the text data includes preprocessed user energy consumption profile data, date attribute information, historical source-load curves, equipment records, and externally called weather forecast data, and the cloud image video data includes preprocessed weather data; Performing feature fusion on the text data and the cloud image video data to obtain fused feature data, and performing position encoding on the fused feature data through an informer model to obtain position encoding data; Taking the position coding data and the fused feature data together as a training data set, and grouping the training data set to obtain multiple groups of training sub-data sets; Using multiple sets of training sub-data sets to train the Informer model to obtain a source-load prediction model; Inputting the acquired multimodal data to be predicted into the source-load prediction model to perform prediction to obtain a prediction result; The training sub-dataset includes source-load training data and label data, and both the source-load training data and the label data include the fusion feature data and the position coding data. The using multiple sets of the training sub-datasets to train the Informer model to obtain a source-load prediction model includes: For each set of the training sub-data sets, the fused feature data and the position coding data in the source-load training data are spliced to obtain target source-load training data; The target source load training data and the label data are used to train the Informer model to obtain the source load prediction model; The informer model includes a self-attention mechanism module, a sparse attention module, a multi-head attention module, and a cross-attention module. The target source load training data and the label data are used to train the informer model to obtain the source load prediction model, including: Inputting the target source load training data into the self-attention mechanism module to perform data fusion to obtain a dictionary set; Inputting the dictionary set into the sparse attention module to perform dimensionality reduction to obtain a reduced dimensionality dictionary set; Inputting the dimension reduction dictionary set into the multi-head attention module to perform information aggregation to obtain an aggregated dictionary set; Inputting the aggregate dictionary set into the cross attention module to calculate the similarity between the aggregate dictionary set and the target source-load training data to obtain source-load similarity; Calculating a source-load loss value according to the source-load similarity, the target source-load training data, and the label data; The source load prediction model is obtained by updating the parameters of the Informer model through back propagation according to the source load loss value.

2. The method according to claim 1, characterized in that The calculating the loss value according to the source-load similarity, the target source-load training data, and the label data includes: Determining a source load prediction result according to the source load similarity and the target source load training data; The error loss between the source load prediction result and the labeling result in the label data is calculated to obtain the source load loss value.

3. The method according to claim 1, characterized in that The step of fusing the text data and the cloud image video data to obtain fused feature data includes: Extracting features of the cloud image video data through a CNN network to obtain cloud image video features; The cloud image video features are fused with the text data to obtain the fused feature data.

4. The method according to claim 3, characterized in that The CNN network includes a convolution layer and a pooling layer, and the convolution layer includes a first convolution layer, a second convolution layer, a third convolution layer, and a fourth convolution layer. The cloud image video features are obtained by extracting the features of the cloud image video data through the CNN network, including: Performing dimensionality reduction processing on the cloud image video data through the first convolution layer, the second convolution layer, the third convolution layer, and the fourth convolution layer in sequence to obtain reduced-dimensional cloud image video data; The cloud image video features are obtained by performing maximum pooling processing on the dimension-reduced cloud image video data through the pooling layer.

5. The method according to claim 1, wherein The preprocessing includes deduplication, denoising, data anomaly, data missing and normalization.

6. A source load prediction device based on multimodal data, characterized in that: include: An acquisition and processing unit, configured to acquire multimodal initial data and preprocess the multimodal data to obtain multimodal target data, wherein the multimodal target data includes text data and cloud image video data; A fusion encoding unit, configured to perform feature fusion on the text data and the cloud image video data to obtain fused feature data, and position encode the fused feature using an informer model to obtain position encoded data; a grouping unit, configured to use the position coding data and the fused feature data together as a training data set, and group the training data set to obtain a plurality of training sub-data sets; A training unit, configured to train the Informer model using multiple sets of training sub-data sets to obtain a source-load prediction model; A prediction unit, configured to input the acquired multimodal data to be predicted into the source-load prediction model to perform prediction and obtain a prediction result; The training sub-dataset includes source-load training data and label data, and both the source-load training data and the label data include the fusion feature data and the position coding data. The training unit includes: a splicing unit, configured to splice the fused feature data and the position coding data in the source-load training data for each set of the training sub-data sets to obtain target source-load training data; A training subunit, configured to train the Informer model using the target source load training data and the label data to obtain the source load prediction model; The Informer model includes a self-attention mechanism module, a sparse attention module, a multi-head attention module, and a cross-attention module. The training subunit includes: A data fusion unit, configured to input the target source load training data into the self-attention mechanism module to perform data fusion to obtain a dictionary set; A dimensionality reduction unit, configured to input the dictionary set into the sparse attention module for dimensionality reduction to obtain a dimensionality reduction dictionary set; An information aggregation unit, configured to input the dimension reduction dictionary set into the multi-head attention module to perform information aggregation to obtain an aggregated dictionary set; A first calculation unit is configured to input the aggregated dictionary set into the cross attention module to calculate the similarity between the aggregated dictionary set and the target source-load training data to obtain source-load similarity; A second calculation unit is used to calculate a source-load loss value according to the source-load similarity, the target source-load training data and the label data; An updating unit is used to update the parameters of the Informer model through back propagation according to the source-load loss value to obtain the source-load prediction model.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images