A wave energy power generation prediction method and system based on multimodal data

Abstract

This invention discloses a wave energy power prediction method and system based on multimodal data. Currently, the core structure of an integrated floating breakwater and wave energy power generation device includes a float assembly consisting of high-strength, corrosion-resistant buoys, a connecting rod structure connecting the float assembly to a variable speed gear mechanism, and a modular structure designed according to surge levels. To address the technical problem of large fluctuations in the real-time output power of wave energy power generation devices, which increases the pressure on power regulation and energy storage scheduling, this solution obtains environmental factors such as wind speed, wind direction, temperature, and humidity at various locations in the watershed, and utilizes a combination of training data sets and algorithm models to predict the next output power of the power generation device in advance, providing data support for energy storage scheduling and supply and demand balance of the entire power grid.

Description

Technical Field

[0001] The present application relates to the field of wave energy power generation and machine learning technology, and in particular to a wave energy power generation power prediction method and system based on multimodal data. Background Art

[0002] With rapid economic development both domestically and internationally, energy demand continues to rise. Excessive consumption of traditional fossil fuels has triggered serious energy crises and environmental problems. Against this backdrop, the development of renewable energy has become a top priority. Wave energy, a clean, renewable, and potentially vast marine energy source, offers broad application prospects. It is estimated that the theoretical global wave energy reserves are approximately 2TW to 3TW, with a technically exploitable capacity of approximately 200GW to 300GW. Furthermore, coastal protection projects are crucial for the safety and stability of coastal areas. Breakwaters, as common coastal protection structures, primarily serve to protect against wave intrusion, maintain port stability, and reduce sedimentation. However, traditional breakwaters are expensive to construct and maintain, offer only a single protective function, and suffer from deficiencies in energy utilization. This innovative design, combining wave energy generators with breakwaters, has emerged. This innovative approach not only effectively utilizes wave energy and reduces dependence on traditional energy sources, but also reduces breakwater construction and maintenance costs, thereby improving their overall efficiency. It holds significant research value and application significance in the fields of energy and marine engineering, offering new approaches and approaches for achieving the dual goals of sustainable energy development and coastal protection.

[0003] However, when using wave energy for power generation, the real-time output power of a generator typically depends on the current wave volume and wave fluctuations in the area where the generator is located. These wave volume and wave fluctuations are subject to significant uncertainty due to various environmental factors. This leads to significant fluctuations in the real-time output power of a wave energy generator, increasing the pressure on power system regulation and energy storage scheduling. Summary of the Invention

[0004] In response to the above technical problems, the present application provides a wave energy power generation power prediction method and system based on multimodal data. By integrating and utilizing multimodal data, the power generation power of the wave energy power generation device in the future time period is predicted, providing data support for the overall energy storage scheduling and supply and demand balance of the power grid, thereby ensuring the stable operation of the power system.

[0005] In a first aspect, an embodiment of the present application provides a method for predicting wave energy power generation based on multimodal data, comprising:

[0006] Acquiring, within a first time period, an environmental dataset, a wave dataset, and an image dataset at each preset location in a target watershed, where the wave energy power generation device is located;

[0007] Based on the time sequence, the environmental data set, the wave data set and the image data set are fused to construct multimodal time series data;

[0008] Inputting the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period;

[0009] The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0010] The embodiment of the present application provides a method for predicting wave energy power generation based on multimodal data. Multimodal time series data is constructed by using an environmental data set, a wave data set, and an image data set. The power generation in the second time period in the future is predicted using a power prediction model constructed based on an autoformer model, thereby achieving an advance prediction of wave energy power generation. Considering that the power generation of wave energy is usually affected by multiple factors, and each factor also affects each other, for example, environmental data may affect wave data, and wave data may affect image data. Therefore, in terms of data input, the embodiment of the present application comprehensively captures the dynamic influencing factors of wave energy generation by fusing multi-source data, avoiding the limitations of a single data source and improving the accuracy of subsequent model predictions. At the same time, the embodiment of the present application accordingly uses a power prediction model constructed based on an autoformer to process multimodal time series data. The deep decomposition architecture of the autoformer model can effectively separate the long-term trend of wave energy (such as tidal cycles) and short-term fluctuations (such as instantaneous surges), enhancing the model's modeling capabilities for non-stationary time series data. Finally, predicting power generation in advance can provide accurate charging and discharging references for the energy storage system, reduce grid shocks caused by power fluctuations, provide data support for the overall energy storage scheduling and supply and demand balance of the grid, and ensure the stable operation of the power system.

[0011] In one possible implementation, obtaining the environmental dataset, wave dataset, and image dataset at each preset location in the target watershed includes:

[0012] Acquiring the environmental data set through environmental sensors at each first preset position, the environmental data set including wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed;

[0013] acquiring the wave data set by wave detectors at each second preset position, wherein the wave data set includes a significant wave height, a wave period, and a wave direction at each preset position in the target watershed;

[0014] The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

[0015] The present application provides a method for acquiring a data set, which uses environmental sensors, wave detectors, and camera devices to respectively acquire wind force, wave parameter, and water surface image data. Multi-dimensional data acquisition (such as wind force, wave height, and wave motion direction) can cover the physical factors that affect wave energy generation, thereby diversifying the input data sources of the model, improving the completeness of the input data, and thus improving the accuracy of the model prediction. Regarding the use of sensors, the present application uses various types of specialized sensors to collect multimodal data. For example, a wave detector can provide accurate wave period and direction data, and a camera device can intuitively reflect the water surface conditions, thereby avoiding the error accumulation of traditional single-point measurements.

[0016] In one possible implementation, the fusing of the environmental dataset, the wave dataset, and the image dataset into multimodal time series data based on time sequence includes:

[0017] Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental dataset, wave dataset and image dataset respectively;

[0018] Converting the environmental feature set, wave feature set, and image feature set into corresponding environmental feature time series data, wave feature time series data, and image feature time series data respectively based on the time sequence;

[0019] After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0020] The embodiment of the present application provides a method for fusing multi-source data into multi-modal time series data. First, feature extraction is performed on each data set. Since the data types of each data set are different, for example, the environmental data set is physical parameter type data, and the image data set is image data, feature extraction is first performed to unify the data dimensions of the multi-source data to facilitate subsequent data fusion processing. Then, based on the time sequence, each feature set is converted into time series data, and the image features (such as wave texture) and physical parameters (such as wind speed) are unified into time series data to facilitate the model to efficiently process heterogeneous information. Finally, each time series data is fused and constructed into the multi-modal time series data through timestamp alignment to avoid data misalignment caused by differences in sensor sampling frequencies, ensure the synchronization of multi-modal data, and improve the accuracy of model prediction.

[0021] Furthermore, extracting a corresponding image feature set from the image dataset includes:

[0022] Locating each wave contour from the image data set using an edge detection algorithm, and then extracting the wave morphological features of each image;

[0023] determining the motion direction and speed of waves in each image based on adjacent frame images in the image data set, and then extracting wave motion features of each image;

[0024] performing texture analysis on each image in the image data set according to each of the wave morphological features, thereby extracting wave texture features of each image;

[0025] The wave morphology features, wave motion features and wave texture features are combined to construct the image feature set.

[0026] The present embodiment provides a method for extracting an image feature set. This method uses edge detection and texture analysis to quantify wave geometric features (such as crest height), supplementing visual information that traditional sensors cannot capture. Analysis of adjacent image frames can track wave motion in real time, improving the model's response to transient changes. The present embodiment utilizes three different image processing methods to extract wave morphological features, wave motion features, and wave texture features, respectively. This improves the model's visual perception of waves in the water, thereby increasing the accuracy of the model's predictions.

[0027] In one possible implementation, the power prediction model extracts trend items and period items from the multimodal time series data based on a deep decomposition architecture, generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including:

[0028] Performing layer-by-layer sequence decomposition and data prediction on the multimodal time series data through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results, and period prediction results for each layer;

[0029] In any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic term and trend term of the current layer; the decoder of the current layer predicts the periodic term and trend term of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic term of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data;

[0030] According to the weight parameters within the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0031] The present application provides a model prediction method that uses a multi-layer encoder-decoder structure to perform layer-by-layer sequence decomposition on the input multimodal time series data. This method effectively utilizes the characteristics of the Autoformer model, extracts periodic terms to obtain the periodic variation pattern of the input data in short time scales, and extracts trend terms to obtain the variation trend of the input data in long time scales. This sequence decomposition improves the model's ability to model multi-scale time series. During the prediction process, each layer of decoder generates a prediction result for its own layer based on the periodic terms and trend terms of each layer, and weightedly fuses the prediction results of each layer in the final output. The weighted fusion mechanism can suppress single-layer prediction errors and improve the stability of the output results.

[0032] In one possible implementation, the training of the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model includes:

[0033] Acquire a historical environmental dataset, a historical wave dataset, a historical image dataset, and a historical power generation dataset of the wave energy power generation device at each preset location in the target watershed;

[0034] Based on the time sequence, the historical environmental data set, the historical wave data set and the historical image data set are fused to construct the historical multimodal time series data;

[0035] Constructing a training data set based on the historical multimodal time series data and the historical power generation data set;

[0036] Inputting the training data set into the initial power prediction model so that the initial power prediction model adjusts internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters;

[0037] The initial power prediction model is updated according to the training parameters to obtain the power prediction model.

[0038] The embodiment of the present application provides a training method for a power prediction model. Similar to the data processing method in the prediction process, the historical environmental data set, the historical wave data set, and the historical image data set are fused to construct historical multimodal time series data. The historical multimodal time series data is then combined with the historical power generation data set to construct a training data set. Finally, through supervised learning, the initial power prediction model is trained using the training data set and a preset loss function, so that the model learns the variation pattern of power generation under different environments. The loss function drives the adjustment of model parameters to ensure that the error between the prediction result and the actual power is minimized, thereby obtaining the final power prediction model.

[0039] In a second aspect, accordingly, an embodiment of the present application provides a wave energy power generation prediction system based on multimodal data, including an acquisition module, a data fusion module, and a prediction module;

[0040] The acquisition module is used to acquire, within a first time period, an environmental data set, a wave data set, and an image data set at each preset location in a target watershed, wherein the target watershed is the watershed where the wave energy power generation device is located;

[0041] The data fusion module is used to fuse the environmental data set, the wave data set and the image data set into multimodal time series data based on time sequence;

[0042] The prediction module is used to input the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period;

[0043] The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0044] In one possible implementation, the data fusion module fuses the environmental dataset, the wave dataset, and the image dataset into multimodal time series data based on a time sequence, including:

[0045] Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental dataset, wave dataset and image dataset respectively;

[0046] Converting the environmental feature set, wave feature set, and image feature set into corresponding environmental feature time series data, wave feature time series data, and image feature time series data respectively based on the time sequence;

[0047] After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0048] In one possible implementation, the power prediction model extracts trend items and period items from the multimodal time series data based on a deep decomposition architecture, generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including:

[0049] Performing layer-by-layer sequence decomposition and data prediction on the multimodal time series data through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results, and period prediction results for each layer;

[0050] In any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic term and trend term of the current layer; the decoder of the current layer predicts the periodic term and trend term of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic term of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data;

[0051] According to the weight parameters within the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0052] In one possible implementation, the wave energy power generation power prediction system further includes a model training module, which is used to train an initial power prediction model according to historical multimodal time series data to obtain the power prediction model, and includes an acquisition unit, a time series data construction unit, a training set construction unit, a training unit, and a model update unit;

[0053] The acquisition unit is used to acquire a historical environmental data set, a historical wave data set, a historical image data set, and a historical power generation data set of the wave energy power generation device at each preset location in the target watershed;

[0054] The time series data construction unit is used to fuse the historical environment data set, the historical wave data set and the historical image data set into the historical multimodal time series data based on the time sequence;

[0055] The training set construction unit is used to construct a training data set based on the historical multimodal time series data and the historical power generation data set;

[0056] The training unit is used to input the training data set into the initial power prediction model, so that the initial power prediction model adjusts the internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters;

[0057] The model updating unit is used to update the initial power prediction model according to the training parameters to obtain the power prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 : A flow chart of a method for predicting wave energy power generation based on multimodal data provided in an embodiment of the present application.

[0059] Figure 2 : A structural diagram of a wave energy power generation prediction system based on multimodal data provided in an embodiment of the present application.

[0060] Figure 3 : A structural diagram of a model training module in a wave energy power generation prediction system based on multimodal data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0062] It should be noted that the step numbers herein are for convenience of explanation of the specific embodiments and do not serve to define the order in which the steps are to be performed. In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature designated "first" or "second" may explicitly or implicitly include one or more of such features.

[0063] Example 1:

[0064] like Figure 1 As shown, embodiment 1 provides a wave energy power prediction method based on multimodal data, including steps S1 to S3:

[0065] Step S1: acquiring, within a first time period, an environmental dataset, a wave dataset, and an image dataset at each preset location in a target watershed, where the target watershed is where a wave energy power generation device is located;

[0066] Step S2: fusing the environmental dataset, the wave dataset, and the image dataset into multimodal time series data based on a time sequence;

[0067] Step S3: inputting the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period;

[0068] The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0069] The embodiment of the present application provides a method for predicting wave energy power generation based on multimodal data. Multimodal time series data is constructed by using an environmental data set, a wave data set, and an image data set. The power generation in the second time period in the future is predicted using a power prediction model constructed based on an autoformer model, thereby achieving an advance prediction of wave energy power generation. Considering that the power generation of wave energy is usually affected by multiple factors, and each factor also affects each other, for example, environmental data may affect wave data, and wave data may affect image data. Therefore, in terms of data input, the embodiment of the present application comprehensively captures the dynamic influencing factors of wave energy generation by fusing multi-source data, avoiding the limitations of a single data source and improving the accuracy of subsequent model predictions. At the same time, the embodiment of the present application accordingly uses a power prediction model constructed based on an autoformer to process multimodal time series data. The deep decomposition architecture of the autoformer model can effectively separate the long-term trend of wave energy (such as tidal cycles) and short-term fluctuations (such as instantaneous surges), enhancing the model's modeling capabilities for non-stationary time series data. Finally, predicting power generation in advance can provide accurate charging and discharging references for the energy storage system, reduce grid shocks caused by power fluctuations, provide data support for the overall energy storage scheduling and supply and demand balance of the grid, and ensure the stable operation of the power system.

[0070] In one possible implementation, in step S1, obtaining an environmental dataset, a wave dataset, and an image dataset at each preset location in the target watershed includes:

[0071] Acquiring the environmental data set through environmental sensors at each first preset position, the environmental data set including wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed;

[0072] acquiring the wave data set by wave detectors at each second preset position, wherein the wave data set includes a significant wave height, a wave period, and a wave direction at each preset position in the target watershed;

[0073] The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

[0074] The present application provides a method for acquiring a data set, which uses environmental sensors, wave detectors, and camera devices to respectively acquire wind force, wave parameter, and water surface image data. Multi-dimensional data acquisition (such as wind force, wave height, and wave motion direction) can cover the physical factors that affect wave energy generation, thereby diversifying the input data sources of the model, improving the completeness of the input data, and thus improving the accuracy of the model prediction. Regarding the use of sensors, the present application uses various types of specialized sensors to collect multimodal data. For example, a wave detector can provide accurate wave period and direction data, and a camera device can intuitively reflect the water surface conditions, thereby avoiding the error accumulation of traditional single-point measurements.

[0075] In a preferred embodiment, in step S1, anemometers and thermo-hygrometers are pre-installed at 10 first-preset locations in the target watershed (e.g., an offshore floating breakwater area) to collect real-time wind speed data (m / s), wind direction (angle), temperature (°C), and humidity (%). Wave sensors are simultaneously installed on buoys at five second-preset locations to record significant wave height (m), wave period (s), and wave direction (angle) every 5 seconds. High-definition cameras are installed at three third-preset locations along the coast of the target watershed to capture water surface images at a 10Hz frame rate, capturing wave morphology and motion. All devices synchronize timestamps via GPS, and data is stored in a cloud database at 5-minute intervals to ensure time series alignment.

[0076] In one possible implementation, in step S2, the step of fusing the environmental dataset, the wave dataset, and the image dataset into multimodal time series data based on a time sequence includes:

[0077] Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental dataset, wave dataset and image dataset respectively;

[0078] Converting the environmental feature set, wave feature set, and image feature set into corresponding environmental feature time series data, wave feature time series data, and image feature time series data respectively based on the time sequence;

[0079] After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0080] The embodiment of the present application provides a method for fusing multi-source data into multi-modal time series data. First, feature extraction is performed on each data set. Since the data types of each data set are different, for example, the environmental data set is physical parameter type data, and the image data set is image data, feature extraction is first performed to unify the data dimensions of the multi-source data to facilitate subsequent data fusion processing. Then, based on the time sequence, each feature set is converted into time series data, and the image features (such as wave texture) and physical parameters (such as wind speed) are unified into time series data to facilitate the model to efficiently process heterogeneous information. Finally, each time series data is fused and constructed into the multi-modal time series data through timestamp alignment to avoid data misalignment caused by differences in sensor sampling frequencies, ensure the synchronization of multi-modal data, and improve the accuracy of model prediction.

[0081] Furthermore, extracting a corresponding image feature set from the image dataset includes:

[0082] Locating each wave contour from the image data set using an edge detection algorithm, and then extracting the wave morphological features of each image;

[0083] determining the motion direction and speed of waves in each image based on adjacent frame images in the image data set, and then extracting wave motion features of each image;

[0084] performing texture analysis on each image in the image data set according to each of the wave morphological features, thereby extracting wave texture features of each image;

[0085] The wave morphology features, wave motion features and wave texture features are combined to construct the image feature set.

[0086] The present embodiment provides a method for extracting an image feature set. This method uses edge detection and texture analysis to quantify wave geometric features (such as crest height), supplementing visual information that traditional sensors cannot capture. Analysis of adjacent image frames can track wave motion in real time, improving the model's response to transient changes. The present embodiment utilizes three different image processing methods to extract wave morphological features, wave motion features, and wave texture features, respectively. This improves the model's visual perception of waves in the water, thereby increasing the accuracy of the model's predictions.

[0087] In a preferred embodiment, the Canny edge detection algorithm is used to locate wave contours, extract peak height and wavelength, and thus extract wave morphological features. The optical flow method (Lucas-Kanade algorithm) is used to calculate wave displacement between adjacent frames, derive wave velocity, and thus extract wave motion features. The gray-level co-occurrence matrix (GLCM) is used to extract image texture contrast and entropy values, thereby extracting wave texture features.

[0088] In one possible implementation, in step S3, the power prediction model extracts trend items and period items from the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for the second time period, including:

[0089] Performing layer-by-layer sequence decomposition and data prediction on the multimodal time series data through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results, and period prediction results for each layer;

[0090] In any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic term and trend term of the current layer; the decoder of the current layer predicts the periodic term and trend term of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic term of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data;

[0091] According to the weight parameters within the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0092] The present application provides a model prediction method that uses a multi-layer encoder-decoder structure to perform layer-by-layer sequence decomposition on the input multimodal time series data. This method effectively utilizes the characteristics of the Autoformer model, extracts periodic terms to obtain the periodic variation pattern of the input data in short time scales, and extracts trend terms to obtain the variation trend of the input data in long time scales. This sequence decomposition improves the model's ability to model multi-scale time series. During the prediction process, each layer of decoder generates a prediction result for its own layer based on the periodic terms and trend terms of each layer, and weightedly fuses the prediction results of each layer in the final output. The weighted fusion mechanism can suppress single-layer prediction errors and improve the stability of the output results.

[0093] In a preferred embodiment, a four-layer encoder-decoder structure is designed based on Autoformer to decompose trend terms and periodic terms layer by layer. Specifically, in each encoder layer, the input data is decomposed into periodic terms and trend terms using a moving average decomposition method. The specific formula is:

[0094]

[0095]

[0096] in, For the l The trend item after layer decomposition, Represents a sliding average (moving average) operation, used to extract low-frequency trends. Indicates that a padding operation is performed on the sequence to avoid boundary effects. For the l The input data of the layer, For the l The periodic term after layer decomposition. The periodic term output by each layer encoder serves as the input of the next layer encoder. The specific formula is:

[0097]

[0098] In the decoder part, after each layer obtains the decomposed trend term and period term, the decoder of each layer predicts the trend term and period term of its layer based on the self-attention mechanism to obtain the trend prediction results and period prediction results.

[0099] Finally, the prediction results of each layer are weighted and fused through the linear layer to generate the final power generation prediction result. The specific formula is:

[0100]

[0101] in, and For the l The weight parameters of the layer trend term and the period term are obtained by model training. is the power generation prediction result, and For the l The trend prediction results and period prediction results of the layer, L is the total number of encoder-decoder layers.

[0102] In one possible implementation, the training of the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model includes:

[0103] Acquire a historical environmental dataset, a historical wave dataset, a historical image dataset, and a historical power generation dataset of the wave energy power generation device at each preset location in the target watershed;

[0104] Based on the time sequence, the historical environmental data set, the historical wave data set and the historical image data set are fused to construct the historical multimodal time series data;

[0105] Constructing a training data set based on the historical multimodal time series data and the historical power generation data set;

[0106] Inputting the training data set into the initial power prediction model so that the initial power prediction model adjusts internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters;

[0107] The initial power prediction model is updated according to the training parameters to obtain the power prediction model.

[0108] The embodiment of the present application provides a training method for a power prediction model. Similar to the data processing method in the prediction process, the historical environmental data set, the historical wave data set, and the historical image data set are fused to construct historical multimodal time series data. The historical multimodal time series data is then combined with the historical power generation data set to construct a training data set. Finally, through supervised learning, the initial power prediction model is trained using the training data set and a preset loss function, so that the model learns the variation pattern of power generation under different environments. The loss function drives the adjustment of model parameters to ensure that the error between the prediction result and the actual power is minimized, thereby obtaining the final power prediction model.

[0109] In a preferred embodiment, historical data (environment, wave, image, and power generation) from the past year are used to construct a training set (80%), a validation set (10%), and a test set (10%). The mean square error (MSE) is used to calculate the loss between the true value and the predicted value. The optimizer is Adam to train the initial power prediction model to obtain the power prediction model.

[0110] Example 2:

[0111] Accordingly, the second embodiment provides a wave energy power prediction system based on multimodal data, such as Figure 2 As shown, it includes an acquisition module 10, a data fusion module 20 and a prediction module 30;

[0112] The acquisition module 10 is used to acquire, within a first time period, an environmental data set, a wave data set, and an image data set at each preset location in a target watershed, where the target watershed is where the wave energy power generation device is located.

[0113] The data fusion module 20 is used to fuse the environmental data set, the wave data set and the image data set into multimodal time series data based on time sequence;

[0114] The prediction module 30 is used to input the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for the second time period;

[0115] The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0116] In one possible implementation, the acquisition module 10 acquires the environmental dataset, wave dataset, and image dataset of each preset location in the target watershed, including:

[0117] Acquiring the environmental data set through environmental sensors at each first preset position, the environmental data set including wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed;

[0118] acquiring the wave data set by wave detectors at each second preset position, wherein the wave data set includes a significant wave height, a wave period, and a wave direction at each preset position in the target watershed;

[0119] The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

[0120] In one possible implementation, the data fusion module 20 fuses the environmental dataset, the wave dataset, and the image dataset into multimodal time series data based on a time sequence, including:

[0121] Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental dataset, wave dataset and image dataset respectively;

[0122] Converting the environmental feature set, wave feature set, and image feature set into corresponding environmental feature time series data, wave feature time series data, and image feature time series data respectively based on the time sequence;

[0123] After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0124] Furthermore, extracting a corresponding image feature set from the image dataset includes:

[0125] Locating each wave contour from the image data set using an edge detection algorithm, and then extracting the wave morphological features of each image;

[0126] determining the motion direction and speed of waves in each image based on adjacent frame images in the image data set, and then extracting wave motion features of each image;

[0127] performing texture analysis on each image in the image data set according to each of the wave morphological features, thereby extracting wave texture features of each image;

[0128] The wave morphology features, wave motion features and wave texture features are combined to construct the image feature set.

[0129] In one possible implementation, the power prediction model extracts trend items and period items from the multimodal time series data based on a deep decomposition architecture, generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including:

[0130] Performing layer-by-layer sequence decomposition and data prediction on the multimodal time series data through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results, and period prediction results for each layer;

[0131] In any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic term and trend term of the current layer; the decoder of the current layer predicts the periodic term and trend term of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic term of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data;

[0132] According to the weight parameters within the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0133] In one possible implementation, Figure 3As shown, the wave energy power generation power prediction system further includes a model training module 40, which is used to train the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model, including an acquisition unit 401, a time series data construction unit 402, a training set construction unit 403, a training unit 404 and a model updating unit 405;

[0134] The acquisition unit 401 is used to acquire a historical environmental dataset, a historical wave dataset, a historical image dataset, and a historical power generation dataset of the wave energy power generation device at each preset location in the target watershed;

[0135] The time series data construction unit 402 is used to fuse the historical environment dataset, the historical wave dataset, and the historical image dataset into the historical multimodal time series data based on a time sequence;

[0136] The training set construction unit 403 is used to construct a training data set based on the historical multimodal time series data and the historical power generation data set;

[0137] The training unit 404 is configured to input the training data set into the initial power prediction model so that the initial power prediction model adjusts internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters;

[0138] The model updating unit 405 is configured to update the initial power prediction model according to the training parameters to obtain the power prediction model.

[0139] The embodiment of the present application provides a wave energy power generation prediction system based on multimodal data. It constructs multimodal time series data through environmental data sets, wave data sets, and image data sets, and uses a power prediction model constructed based on an autoformer model to predict the power generation in the second time period in the future, thereby realizing the advance prediction of wave energy power generation. Considering that the power generation of wave energy is usually affected by multiple factors, and each factor also affects each other, for example, environmental data may affect wave data, and wave data may affect image data. Therefore, in terms of data input, the embodiment of the present application comprehensively captures the dynamic influencing factors of wave energy power generation by fusing multi-source data, avoiding the limitations of a single data source and improving the accuracy of subsequent model predictions. At the same time, the embodiment of the present application accordingly uses a power prediction model constructed based on an autoformer to process multimodal time series data. The deep decomposition architecture of the autoformer model can effectively separate the long-term trend of wave energy (such as tidal cycle) and short-term fluctuations (such as instantaneous surge), enhancing the model's modeling ability for non-stationary time series data. Finally, predicting power generation in advance can provide accurate charging and discharging references for the energy storage system, reduce grid shocks caused by power fluctuations, provide data support for the overall energy storage scheduling and supply and demand balance of the grid, and ensure the stable operation of the power system.

[0140] The more detailed working principle and process flow of this embodiment can be referred to, but not limited to, the relevant records of the first embodiment.

[0141] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application by those skilled in the art should be included within the scope of protection of this application.

Claims

1. A wave energy power generation prediction method based on multimodal data, characterized in that: include: Acquiring, within a first time period, an environmental dataset, a wave dataset, and an image dataset at each preset location in a target watershed, where the wave energy power generation device is located; Based on the time sequence, the environmental data set, the wave data set and the image data set are fused to construct multimodal time series data; Inputting the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model; The step of fusing the environmental dataset, the wave dataset, and the image dataset into multimodal time series data based on a time sequence includes: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental dataset, wave dataset and image dataset respectively; Converting the environmental feature set, wave feature set, and image feature set into corresponding environmental feature time series data, wave feature time series data, and image feature time series data respectively based on the time sequence; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, the data are fused to form the multimodal time series data; Extracting a corresponding image feature set from the image dataset includes: Locating each wave contour from the image data set using an edge detection algorithm, and then extracting the wave morphological features of each image; determining the motion direction and speed of waves in each image based on adjacent frame images in the image data set, and then extracting wave motion features of each image; performing texture analysis on each image in the image data set according to each of the wave morphological features, thereby extracting wave texture features of each image; Combining the wave morphology features, wave motion features, and wave texture features to construct the image feature set; The power prediction model extracts trend items and period items from the multimodal time series data based on a deep decomposition architecture, generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including: Performing layer-by-layer sequence decomposition and data prediction on the multimodal time series data through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results, and period prediction results for each layer; In any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic term and trend term of the current layer; the decoder of the current layer predicts the periodic term and trend term of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic term of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters within the power prediction model, the trend prediction results and the period prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate a power generation prediction result for the second time period; The Canny edge detection algorithm is used to locate the wave contour, extract the peak height and wavelength, and then extract the wave morphological characteristics; the optical flow method is used to calculate the wave displacement of adjacent frames, deduce the wave movement speed, and then extract the wave motion characteristics; the image texture contrast and entropy value are extracted based on the gray-level co-occurrence matrix, and then extract the wave texture characteristics.

2. The method for predicting wave energy power generation based on multimodal data according to claim 1, characterized in that: The step of obtaining the environmental dataset, wave dataset, and image dataset of each preset location in the target watershed includes: Acquiring the environmental data set through environmental sensors at each first preset position, the environmental data set including wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed; acquiring the wave data set by wave detectors at each second preset position, wherein the wave data set includes a significant wave height, a wave period, and a wave direction at each preset position in the target watershed; The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

3. The method for predicting wave energy power generation based on multimodal data according to claim 1, characterized in that: The training of the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model includes: Acquire a historical environmental dataset, a historical wave dataset, a historical image dataset, and a historical power generation dataset of the wave energy power generation device at each preset location in the target watershed; Based on the time sequence, the historical environmental data set, the historical wave data set and the historical image data set are fused to construct the historical multimodal time series data; Constructing a training data set based on the historical multimodal time series data and the historical power generation data set; Inputting the training data set into the initial power prediction model so that the initial power prediction model adjusts internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The initial power prediction model is updated according to the training parameters to obtain the power prediction model.

4. A wave energy power generation prediction system based on multimodal data, characterized in that: Including acquisition module, data fusion module and prediction module; The acquisition module is used to acquire, within a first time period, an environmental data set, a wave data set, and an image data set at each preset location in a target watershed, wherein the target watershed is the watershed where the wave energy power generation device is located; The data fusion module is used to fuse the environmental data set, the wave data set and the image data set into multimodal time series data based on time sequence; The prediction module is used to input the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model; The data fusion module fuses the environmental dataset, the wave dataset, and the image dataset into multimodal time series data based on a time sequence, including: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental dataset, wave dataset and image dataset respectively; Converting the environmental feature set, wave feature set, and image feature set into corresponding environmental feature time series data, wave feature time series data, and image feature time series data respectively based on the time sequence; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, the data are fused to form the multimodal time series data; Extracting a corresponding image feature set from the image dataset includes: Locating each wave contour from the image data set using an edge detection algorithm, and then extracting the wave morphological features of each image; determining the motion direction and speed of waves in each image based on adjacent frame images in the image data set, and then extracting wave motion features of each image; performing texture analysis on each image in the image data set according to each of the wave morphological features, thereby extracting wave texture features of each image; Combining the wave morphology features, wave motion features, and wave texture features to construct the image feature set; The power prediction model extracts trend items and period items from the multimodal time series data based on a deep decomposition architecture, generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including: Performing layer-by-layer sequence decomposition and data prediction on the multimodal time series data through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results, and period prediction results for each layer; In any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic term and trend term of the current layer; the decoder of the current layer predicts the periodic term and trend term of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic term of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters within the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

5. The wave energy power generation prediction system based on multimodal data according to claim 4, characterized in that: The wave energy power generation power prediction system further includes a model training module, which is used to train the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model, including an acquisition unit, a time series data construction unit, a training set construction unit, a training unit and a model update unit; The acquisition unit is used to acquire a historical environmental data set, a historical wave data set, a historical image data set, and a historical power generation data set of the wave energy power generation device at each preset location in the target watershed; The time series data construction unit is used to fuse the historical environment data set, the historical wave data set and the historical image data set into the historical multimodal time series data based on the time sequence; The training set construction unit is used to construct a training data set based on the historical multimodal time series data and the historical power generation data set; The training unit is used to input the training data set into the initial power prediction model, so that the initial power prediction model adjusts the internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The model updating unit is used to update the initial power prediction model according to the training parameters to obtain the power prediction model.

Images