Wave power generation power prediction method and system based on multi-modal data

Abstract

The invention discloses a wave power generation power prediction method and system based on multi-modal data, and the core structure of an existing floating breakwater and wave power generation integrated device comprises a floating body device which is composed of a high-strength and corrosion-resistant floating cylinder, a connecting rod structure is connected with a floating body and a change gear mechanism, and a modular structure is combined to be designed according to the surge grade. In order to solve the technical problem that the real-time output power of a wave energy power generation device fluctuates greatly, so that the pressure of power regulation and control and energy storage dispatching is increased, according to the scheme, environmental factors such as wind power, wind direction, temperature and humidity of all positions of a drainage basin are obtained, and training data set algorithm models are combined; the output power of the power generation device at the next time is predicted in advance, and data support is provided for overall energy storage scheduling and supply-demand balance of a power grid.

Description

Technical Field

[0001] The present application relates to the field of wave energy generation and machine learning technology, and in particular to a wave energy generation power prediction method and system based on multimodal data. Background Art

[0002] At present, the domestic and foreign economies are developing rapidly, and the energy demand continues to rise. The excessive consumption of traditional fossil energy has caused serious energy crises and environmental problems. In this context, the development of renewable energy has become a top priority. As a clean, renewable and huge marine energy, wave energy is estimated to have a theoretical reserve of about 2TW-3TW and a technically exploitable amount of about 200GW-300GW, which has broad application prospects. At the same time, coastal protection projects are crucial to the safety and stability of coastal areas. As a common coastal protection structure, the main functions of breakwaters are to defend against wave invasion, protect the stability of port waters, and reduce siltation. However, the construction and maintenance costs of traditional breakwaters are high, and they only have a single protection function, which is insufficient in energy utilization. The integrated design concept of combining wave energy generation devices with breakwaters came into being. This innovative design can not only effectively utilize wave energy and reduce dependence on traditional energy, but also reduce the construction and operation and maintenance costs of breakwaters and improve their comprehensive benefits. It has extremely important research value and application significance in the field of energy and marine engineering, and provides new ideas and new ways to achieve the dual goals of sustainable energy development and coastal protection.

[0003] However, in the process of using wave energy to generate electricity, the real-time output power of the power generation device usually depends on the current number of waves and the degree of wave fluctuation in the area where the power generation device is located. The number of waves and the degree of wave fluctuation are affected by various environmental factors and have great uncertainty, which leads to large fluctuations in the real-time output power of the wave energy power generation device, increasing the pressure on power control and energy storage scheduling of the power system. Summary of the invention

[0004] In response to the above technical problems, the present application provides a wave energy power generation prediction method and system based on multimodal data. By integrating and utilizing multimodal data to predict the power generation power of the wave energy power generation device in the future time period, data support is provided for the overall energy storage scheduling and supply and demand balance of the power grid, thereby ensuring the stable operation of the power system.

[0005] In a first aspect, an embodiment of the present application provides a wave energy power prediction method based on multimodal data, comprising: In a first time period, an environmental data set, a wave data set, and an image data set are obtained at each preset location in a target watershed, where the target watershed is where the wave energy power generation device is located; Based on the time sequence, the environmental data set, the wave data set and the image data set are fused to construct multimodal time series data; Inputting the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0006] The embodiment of the present application provides a wave energy power generation prediction method based on multimodal data, constructs multimodal time series data through environmental data sets, wave data sets and image data sets, and uses a power prediction model constructed based on an Autoformer model to predict the power generation in the second time period in the future, thereby realizing the advance prediction of wave energy power generation. Considering that the power generation of wave energy is usually affected by multiple factors, and each factor also affects each other, for example, environmental data may affect wave data, and wave data may affect image data. Therefore, in terms of data input, the embodiment of the present application comprehensively captures the dynamic influencing factors of wave energy power generation by fusing multi-source data, avoiding the limitations of a single data source, and improving the accuracy of subsequent model predictions. At the same time, the embodiment of the present application accordingly uses a power prediction model constructed based on Autoformer to process multimodal time series data. The deep decomposition architecture of the Autoformer model can effectively separate the long-term trend (such as tidal cycle) and short-term fluctuation (such as instantaneous surge) of wave energy, and enhance the modeling ability of the model for non-stationary time series data. Finally, predicting power generation in advance can provide accurate charging and discharging references for the energy storage system, reduce grid shocks caused by power fluctuations, provide data support for the overall grid energy storage scheduling and supply and demand balance, and ensure the stable operation of the power system.

[0007] In a possible implementation, the step of obtaining the environmental dataset, wave dataset, and image dataset of each preset location in the target watershed includes: Acquire the environmental data set through environmental sensors at each first preset position, wherein the environmental data set includes wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed; Acquire the wave data set by wave detectors at each second preset position, wherein the wave data set includes the effective wave height, wave period and wave direction at each preset position of the target basin; The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

[0008] The embodiment of the present application provides a method for obtaining a data set, which obtains wind force, wave parameters and water surface image data through environmental sensors, wave detectors, and camera devices. The physical factors that affect wave energy generation can be covered through multi-dimensional data collection (such as wind force, wave height, and wave movement direction), so that the input data sources of the model are diversified, the completeness of the input data is improved, and the accuracy of the model prediction is improved. In terms of the use of sensors, the embodiment of the present application uses various types of special sensors to collect multimodal data, such as wave detectors that can provide accurate wave period and direction data, and camera devices that can intuitively reflect water surface conditions, etc., to avoid the error accumulation of traditional single-point measurements.

[0009] In a possible implementation, the step of fusing the environment dataset, the wave dataset, and the image dataset into multimodal time series data based on time order includes: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental data set, wave data set and image data set respectively; Based on the time sequence, the environmental feature set, the wave feature set and the image feature set are respectively converted into corresponding environmental feature time series data, wave feature time series data and image feature time series data; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0010] The embodiment of the present application provides a method for fusing multi-source data into multi-modal time series data. First, feature extraction is performed on each data set. Since the data types of each data set are different, for example, the environmental data set is physical parameter type data, and the image data set is image data, feature extraction is first performed to unify the data dimensions of the multi-source data to facilitate subsequent data fusion processing. Then, based on the time sequence, each feature set is converted into time series data, and the image features (such as wave texture) and physical parameters (such as wind speed) are unified into time series data to facilitate the model to efficiently process heterogeneous information. Finally, each time series data is fused and constructed into the multi-modal time series data through timestamp alignment to avoid data misalignment caused by differences in sensor sampling frequencies, ensure the synchronization of multi-modal data, and improve the accuracy of model prediction.

[0011] Furthermore, extracting a corresponding image feature set from the image data set includes: Locating each wave contour from the image data set by an edge detection algorithm, and then extracting the wave morphological features of each image; Determining the movement direction and speed of waves in each image according to adjacent frame images in the image data set, and then extracting wave movement features of each image; Performing texture analysis on each image in the image data set according to each of the wave morphology features, thereby extracting wave texture features of each image; The wave morphology features, wave motion features and wave texture features are combined to construct the image feature set.

[0012] The embodiment of the present application provides a method for extracting an image feature set, in which the geometric features of waves (such as wave crest height) are quantified through edge detection and texture analysis to supplement the visual information that traditional sensors cannot capture. Adjacent frame image analysis can track the trajectory of wave motion in real time and improve the model's response speed to transient changes. The embodiment of the present application uses three different image processing methods to respectively extract and obtain wave morphological features, wave motion features, and wave texture features, thereby improving the model's perception of waves in the water from a visual level, thereby improving the accuracy of the model's predictions.

[0013] In a possible implementation, the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including: The multimodal time series data is subjected to layer-by-layer sequence decomposition and data prediction by a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results and period prediction results of each layer; Among them, in any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic item and trend item of the current layer; the decoder of the current layer predicts the periodic item and trend item of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic item of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters inside the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0014] The embodiment of the present application provides a prediction method of a model, which performs layer-by-layer sequence decomposition of the input multimodal time series data through a multi-layer encoder-decoder structure, effectively utilizes the characteristics of the Autoformer model, obtains the periodic change law of the input data in a short time scale by extracting the periodic term, obtains the change trend of the input data in a long time scale by extracting the trend term, and improves the modeling ability of the model for multi-scale time series through sequence decomposition. In the prediction process, each layer of decoders generates the prediction results of their respective layers according to the periodic terms and trend terms of each layer, and weights and fuses the prediction results of each layer in the final output. The weighted fusion mechanism can suppress the single-layer prediction error and improve the stability of the output result.

[0015] In a possible implementation, the training of the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model includes: Acquire a historical environmental data set, a historical wave data set, a historical image data set, and a historical power generation data set of the wave energy power generation device at each preset location in the target river basin; Based on the time sequence, the historical environment data set, the historical wave data set and the historical image data set are fused to construct the historical multimodal time series data; Constructing a training data set based on the historical multimodal time series data and the historical power generation data set; Inputting the training data set into the initial power prediction model so that the initial power prediction model adjusts internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The initial power prediction model is updated according to the training parameters to obtain the power prediction model.

[0016] The embodiment of the present application provides a training method for a power prediction model. The historical environment data set, the historical wave data set and the historical image data set are fused to construct historical multimodal time series data in the same way as the data processing method in the prediction process. The historical multimodal time series data is then combined with the historical power generation data set to construct a training data set. Finally, through supervised learning, the initial power prediction model is trained using the training data set and a preset loss function, so that the model learns the law of power generation changes under different environments, and the model parameters are adjusted by the loss function to ensure that the error between the prediction result and the actual power is minimized, thereby obtaining the final power prediction model.

[0017] In a second aspect, accordingly, an embodiment of the present application provides a wave energy power generation prediction system based on multimodal data, including an acquisition module, a data fusion module and a prediction module; The acquisition module is used to acquire the environmental data set, wave data set and image data set of each preset position in the target watershed within the first time period, and the target watershed is the watershed where the wave energy power generation device is located; The data fusion module is used to fuse the environmental data set, the wave data set and the image data set into multi-modal time series data based on the time sequence; The prediction module is used to input the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0018] In a possible implementation, the data fusion module fuses the environment data set, the wave data set, and the image data set into multimodal time series data based on a time sequence, including: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental data set, wave data set and image data set respectively; Based on the time sequence, the environmental feature set, the wave feature set and the image feature set are respectively converted into corresponding environmental feature time series data, wave feature time series data and image feature time series data; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0019] In a possible implementation, the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including: The multimodal time series data is subjected to layer-by-layer sequence decomposition and data prediction by a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results and period prediction results of each layer; Among them, in any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic item and trend item of the current layer; the decoder of the current layer predicts the periodic item and trend item of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic item of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters inside the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0020] In a possible implementation, the wave energy power generation power prediction system further includes a model training module, which is used to train an initial power prediction model according to historical multimodal time series data to obtain the power prediction model, and includes an acquisition unit, a time series data construction unit, a training set construction unit, a training unit, and a model update unit; Wherein, the acquisition unit is used to acquire the historical environment data set, the historical wave data set, the historical image data set and the historical power generation data set of the wave energy power generation device at each preset location in the target watershed; The time series data construction unit is used to fuse the historical environment data set, the historical wave data set and the historical image data set into the historical multimodal time series data based on the time sequence; The training set construction unit is used to construct a training data set according to the historical multimodal time series data and the historical power generation data set; The training unit is used to input the training data set into the initial power prediction model, so that the initial power prediction model adjusts the internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The model updating unit is used to update the initial power prediction model according to the training parameters to obtain the power prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 : A flow chart of a wave energy power generation prediction method based on multimodal data provided in an embodiment of the present application.

[0022] Figure 2 : A structural schematic diagram of a wave energy power generation prediction system based on multimodal data provided in an embodiment of the present application.

[0023] Figure 3 : A structural diagram of a model training module in a wave energy power generation prediction system based on multimodal data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] It should be noted that the step numbers in the text are only for the convenience of explanation of the specific embodiments and do not serve to limit the order in which the steps are executed. In the description of this application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0026] Embodiment 1: like Figure 1 As shown, embodiment 1 provides a wave energy power prediction method based on multi-modal data, including steps S1 to S3: Step S1, within a first time period, obtaining an environmental data set, a wave data set, and an image data set at each preset location in a target watershed, wherein the target watershed is a watershed where a wave energy power generation device is located; Step S2, fusing the environmental data set, the wave data set and the image data set into multimodal time series data based on the time sequence; Step S3, inputting the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0027] The embodiment of the present application provides a wave energy power generation prediction method based on multimodal data, constructs multimodal time series data through environmental data sets, wave data sets and image data sets, and uses a power prediction model constructed based on an Autoformer model to predict the power generation in the second time period in the future, thereby realizing the advance prediction of wave energy power generation. Considering that the power generation of wave energy is usually affected by multiple factors, and each factor also affects each other, for example, environmental data may affect wave data, and wave data may affect image data. Therefore, in terms of data input, the embodiment of the present application comprehensively captures the dynamic influencing factors of wave energy power generation by fusing multi-source data, avoiding the limitations of a single data source, and improving the accuracy of subsequent model predictions. At the same time, the embodiment of the present application accordingly uses a power prediction model constructed based on Autoformer to process multimodal time series data. The deep decomposition architecture of the Autoformer model can effectively separate the long-term trend (such as tidal cycle) and short-term fluctuation (such as instantaneous surge) of wave energy, and enhance the modeling ability of the model for non-stationary time series data. Finally, predicting power generation in advance can provide accurate charging and discharging references for the energy storage system, reduce grid shocks caused by power fluctuations, provide data support for the overall grid energy storage scheduling and supply and demand balance, and ensure the stable operation of the power system.

[0028] In a possible implementation, in step S1, obtaining an environmental dataset, a wave dataset, and an image dataset at each preset location in the target watershed includes: Acquire the environmental data set through environmental sensors at each first preset position, wherein the environmental data set includes wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed; Acquire the wave data set by wave detectors at each second preset position, wherein the wave data set includes the effective wave height, wave period and wave direction at each preset position of the target basin; The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

[0029] The embodiment of the present application provides a method for obtaining a data set, which obtains wind force, wave parameters and water surface image data through environmental sensors, wave detectors, and camera devices. The physical factors that affect wave energy generation can be covered through multi-dimensional data collection (such as wind force, wave height, and wave movement direction), so that the input data sources of the model are diversified, the completeness of the input data is improved, and the accuracy of the model prediction is improved. In terms of the use of sensors, the embodiment of the present application uses various types of special sensors to collect multimodal data, such as wave detectors that can provide accurate wave period and direction data, and camera devices that can intuitively reflect water surface conditions, etc., to avoid the error accumulation of traditional single-point measurements.

[0030] In a preferred embodiment, in step S1, anemometers and thermometers are pre-installed at 10 first preset positions in the target basin (such as offshore floating breakwater areas) to collect wind data (m / s), wind direction (angle), temperature (℃), and humidity (%) in real time. At the same time, wave sensors are installed on buoys at 5 second preset positions to record effective wave height (m), wave period (s), and wave direction (angle) every 5 seconds. High-definition cameras are installed at 3 third preset positions along the coast of the target basin to capture water surface images at a frame rate of 10Hz to capture wave morphology and motion trajectory. All devices synchronize timestamps via GPS, and data is stored in the cloud database at intervals of 5 minutes to ensure time series alignment.

[0031] In a possible implementation, in step S2, the environmental data set, the wave data set, and the image data set are fused and constructed into multimodal time series data based on the time sequence, including: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental data set, wave data set and image data set respectively; Based on the time sequence, the environmental feature set, the wave feature set and the image feature set are respectively converted into corresponding environmental feature time series data, wave feature time series data and image feature time series data; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0032] The embodiment of the present application provides a method for fusing multi-source data into multi-modal time series data. First, feature extraction is performed on each data set. Since the data types of each data set are different, for example, the environmental data set is physical parameter type data, and the image data set is image data, feature extraction is first performed to unify the data dimensions of the multi-source data to facilitate subsequent data fusion processing. Then, based on the time sequence, each feature set is converted into time series data, and the image features (such as wave texture) and physical parameters (such as wind speed) are unified into time series data to facilitate the model to efficiently process heterogeneous information. Finally, each time series data is fused and constructed into the multi-modal time series data through timestamp alignment to avoid data misalignment caused by differences in sensor sampling frequencies, ensure the synchronization of multi-modal data, and improve the accuracy of model prediction.

[0033] Furthermore, extracting a corresponding image feature set from the image data set includes: Locating each wave contour from the image data set by an edge detection algorithm, and then extracting the wave morphological features of each image; Determining the movement direction and speed of waves in each image according to adjacent frame images in the image data set, and then extracting wave movement features of each image; Performing texture analysis on each image in the image data set according to each of the wave morphology features, thereby extracting wave texture features of each image; The wave morphology features, wave motion features and wave texture features are combined to construct the image feature set.

[0034] The embodiment of the present application provides a method for extracting an image feature set, in which the geometric features of waves (such as wave crest height) are quantified through edge detection and texture analysis to supplement the visual information that traditional sensors cannot capture. Adjacent frame image analysis can track the trajectory of wave motion in real time and improve the model's response speed to transient changes. The embodiment of the present application uses three different image processing methods to respectively extract and obtain wave morphological features, wave motion features, and wave texture features, thereby improving the model's perception of waves in the water from a visual level, thereby improving the accuracy of the model's predictions.

[0035] In a preferred embodiment, the Canny edge detection algorithm is used to locate the wave contour, extract the wave crest height and wavelength, and then extract the wave morphology characteristics. The optical flow method (Lucas-Kanade algorithm) is used to calculate the wave displacement of adjacent frames, deduce the wave movement speed, and then extract the wave movement characteristics. The image texture contrast and entropy value are extracted based on the gray level co-occurrence matrix (GLCM), and then the wave texture characteristics are extracted.

[0036] In a possible implementation, in step S3, the power prediction model extracts trend items and period items in the multimodal time series data based on the deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for the second time period, including: The multimodal time series data is subjected to layer-by-layer sequence decomposition and data prediction by a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results and period prediction results of each layer; Among them, in any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic item and trend item of the current layer; the decoder of the current layer predicts the periodic item and trend item of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic item of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters inside the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0037] The embodiment of the present application provides a prediction method of a model, which performs layer-by-layer sequence decomposition of the input multimodal time series data through a multi-layer encoder-decoder structure, effectively utilizes the characteristics of the Autoformer model, obtains the periodic change law of the input data in a short time scale by extracting the periodic term, obtains the change trend of the input data in a long time scale by extracting the trend term, and improves the modeling ability of the model for multi-scale time series through sequence decomposition. In the prediction process, each layer of decoders generates the prediction results of their respective layers according to the periodic terms and trend terms of each layer, and weights and fuses the prediction results of each layer in the final output. The weighted fusion mechanism can suppress the single-layer prediction error and improve the stability of the output result.

[0038] In a preferred embodiment, a 4-layer encoder-decoder structure is designed based on Autoformer to decompose trend items and period items layer by layer. Specifically, in each layer of encoder, the input data is decomposed into period items and trend items using a moving average decomposition method. The specific formula is:

[0039]

[0040] in, For the l The trend item after layer decomposition, Represents a sliding average (moving average) operation, which is used to extract low-frequency trends. Indicates padding of the sequence to avoid boundary effects. For the l The input data of the layer, For the l The periodic term after layer decomposition. The periodic term output by each layer encoder is used as the input of the next layer encoder. The specific formula is:

[0041] In the decoder part, after each layer obtains the decomposed trend terms and period terms, the decoder of each layer predicts the trend terms and period terms of its layer based on the self-attention mechanism to obtain the trend prediction results and period prediction results.

[0042] Finally, the prediction results of each layer are weighted and fused through the linear layer to generate the final power generation prediction result. The specific formula is:

[0043] in, and For the l The weight parameters of the layer trend term and the period term are obtained by model training. is the power generation prediction result, and For the l The trend prediction results and period prediction results of the layer, L is the total number of encoder-decoder layers.

[0044] In a possible implementation, the training of the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model includes: Acquire a historical environmental data set, a historical wave data set, a historical image data set, and a historical power generation data set of the wave energy power generation device at each preset location in the target river basin; Based on the time sequence, the historical environment data set, the historical wave data set and the historical image data set are fused to construct the historical multimodal time series data; Constructing a training data set based on the historical multimodal time series data and the historical power generation data set; Inputting the training data set into the initial power prediction model so that the initial power prediction model adjusts internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The initial power prediction model is updated according to the training parameters to obtain the power prediction model.

[0045] The embodiment of the present application provides a training method for a power prediction model. The historical environment data set, the historical wave data set and the historical image data set are fused to construct historical multimodal time series data in the same way as the data processing method in the prediction process. The historical multimodal time series data is then combined with the historical power generation data set to construct a training data set. Finally, through supervised learning, the initial power prediction model is trained using the training data set and a preset loss function, so that the model learns the law of power generation changes under different environments, and the model parameters are adjusted by the loss function to ensure that the error between the prediction result and the actual power is minimized, thereby obtaining the final power prediction model.

[0046] In a preferred embodiment, historical data (environment, wave, image and power generation) of the past year are used to construct a training set (80%), a validation set (10%) and a test set (10%), and the mean square error (MSE) is used to calculate the loss between the true value and the predicted value. The optimizer is Adam to train the initial power prediction model to obtain the power prediction model.

[0047] Embodiment 2: Accordingly, the second embodiment provides a wave energy power prediction system based on multi-modal data, such as Figure 2 As shown, it includes an acquisition module 10, a data fusion module 20 and a prediction module 30; The acquisition module 10 is used to acquire the environmental data set, wave data set and image data set of each preset position of the target watershed within a first time period, and the target watershed is the watershed where the wave energy power generation device is located; The data fusion module 20 is used to fuse the environmental data set, the wave data set and the image data set into multi-modal time series data based on the time sequence; The prediction module 30 is used to input the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

[0048] In a possible implementation, the acquisition module 10 acquires the environmental data set, wave data set and image data set of each preset position in the target watershed, including: Acquire the environmental data set through environmental sensors at each first preset position, wherein the environmental data set includes wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed; Acquire the wave data set by wave detectors at each second preset position, wherein the wave data set includes the effective wave height, wave period and wave direction at each preset position of the target basin; The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

[0049] In a possible implementation, the data fusion module 20 fuses the environment data set, the wave data set, and the image data set into multimodal time series data based on a time sequence, including: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental data set, wave data set and image data set respectively; Based on the time sequence, the environmental feature set, the wave feature set and the image feature set are respectively converted into corresponding environmental feature time series data, wave feature time series data and image feature time series data; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

[0050] Furthermore, extracting a corresponding image feature set from the image data set includes: Locating each wave contour from the image data set by an edge detection algorithm, and then extracting the wave morphological features of each image; Determining the movement direction and speed of waves in each image according to adjacent frame images in the image data set, and then extracting wave movement features of each image; Performing texture analysis on each image in the image data set according to each of the wave morphology features, thereby extracting wave texture features of each image; The wave morphology features, wave motion features and wave texture features are combined to construct the image feature set.

[0051] In a possible implementation, the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including: The multimodal time series data is subjected to layer-by-layer sequence decomposition and data prediction by a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results and period prediction results of each layer; Among them, in any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic item and trend item of the current layer; the decoder of the current layer predicts the periodic item and trend item of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic item of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters inside the power prediction model, the trend prediction results and cycle prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

[0052] In one possible implementation, Figure 3 As shown, the wave energy power generation power prediction system also includes a model training module 40, which is used to train the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model, including an acquisition unit 401, a time series data construction unit 402, a training set construction unit 403, a training unit 404 and a model update unit 405; The acquisition unit 401 is used to acquire a historical environmental data set, a historical wave data set, a historical image data set and a historical power generation data set of the wave energy power generation device at each preset location in the target watershed; The time series data construction unit 402 is used to fuse the historical environment data set, the historical wave data set and the historical image data set into the historical multimodal time series data based on the time sequence; The training set construction unit 403 is used to construct a training data set according to the historical multimodal time series data and the historical power generation data set; The training unit 404 is used to input the training data set into the initial power prediction model, so that the initial power prediction model adjusts the internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The model updating unit 405 is used to update the initial power prediction model according to the training parameters to obtain the power prediction model.

[0053] The embodiment of the present application provides a wave energy power generation prediction system based on multimodal data, which constructs multimodal time series data through environmental data sets, wave data sets and image data sets, and uses a power prediction model constructed based on an Autoformer model to predict the power generation in the second time period in the future, thereby realizing the advance prediction of wave energy power generation. Considering that the power generation of wave energy is usually affected by multiple factors, and each factor also affects each other, for example, environmental data may affect wave data, and wave data may affect image data. Therefore, in terms of data input, the embodiment of the present application comprehensively captures the dynamic influencing factors of wave energy power generation by fusing multi-source data, avoiding the limitations of a single data source, and improving the accuracy of subsequent model predictions. At the same time, the embodiment of the present application accordingly uses a power prediction model constructed based on Autoformer to process multimodal time series data. The deep decomposition architecture of the Autoformer model can effectively separate the long-term trend (such as tidal cycle) and short-term fluctuation (such as instantaneous surge) of wave energy, and enhance the modeling ability of the model for non-stationary time series data. Finally, predicting power generation in advance can provide accurate charging and discharging references for the energy storage system, reduce grid shocks caused by power fluctuations, provide data support for the overall grid energy storage scheduling and supply and demand balance, and ensure the stable operation of the power system.

[0054] The more detailed working principle and step flow of this embodiment can refer to, but are not limited to, the relevant records of Embodiment 1.

[0055] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A wave energy power generation prediction method based on multimodal data, characterized in that: include: In a first time period, an environmental data set, a wave data set, and an image data set are obtained at each preset location in a target watershed, where the target watershed is where the wave energy power generation device is located; Based on the time sequence, the environmental data set, the wave data set and the image data set are fused to construct multimodal time series data; Inputting the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

2. A method for predicting wave energy power generation based on multimodal data according to claim 1, characterized in that: The step of obtaining the environmental data set, wave data set and image data set at each preset location in the target watershed includes: Acquire the environmental data set through environmental sensors at each first preset position, wherein the environmental data set includes wind force data, wind direction data, temperature data, and humidity data at each preset position in the target watershed; Acquire the wave data set by wave detectors at each second preset position, wherein the wave data set includes the effective wave height, wave period and wave direction at each preset position of the target basin; The image data set is acquired by shooting devices at each third preset position, and the image data set includes water surface images at each preset position of the target watershed.

3. A method for predicting wave energy power generation based on multimodal data according to claim 1, characterized in that: The step of fusing the environment dataset, the wave dataset and the image dataset into multimodal time series data based on the time sequence includes: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental data set, wave data set and image data set respectively; Based on the time sequence, the environmental feature set, the wave feature set and the image feature set are respectively converted into corresponding environmental feature time series data, wave feature time series data and image feature time series data; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

4. A method for predicting wave energy power generation based on multimodal data as claimed in claim 3, characterized in that: The extracting the corresponding image feature set from the image data set comprises: Locating each wave contour from the image data set by an edge detection algorithm, and then extracting the wave morphological features of each image; Determining the movement direction and speed of waves in each image according to adjacent frame images in the image data set, and then extracting wave movement features of each image; Performing texture analysis on each image in the image data set according to each of the wave morphology features, thereby extracting wave texture features of each image; The wave morphology features, wave motion features and wave texture features are combined to construct the image feature set.

5. The method for predicting wave energy power generation based on multimodal data according to claim 1, characterized in that: The power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including: The multimodal time series data is subjected to layer-by-layer sequence decomposition and data prediction through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results and period prediction results of each layer; Among them, in any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic item and trend item of the current layer; the decoder of the current layer predicts the periodic item and trend item of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic item of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters inside the power prediction model, the trend prediction results and period prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

6. A method for predicting wave energy power generation based on multimodal data according to claim 1, characterized in that: The training of the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model includes: Acquire a historical environmental data set, a historical wave data set, a historical image data set, and a historical power generation data set of the wave energy power generation device at each preset location in the target river basin; Based on the time sequence, the historical environment data set, the historical wave data set and the historical image data set are fused to construct the historical multimodal time series data; Constructing a training data set based on the historical multimodal time series data and the historical power generation data set; Inputting the training data set into the initial power prediction model so that the initial power prediction model adjusts internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The initial power prediction model is updated according to the training parameters to obtain the power prediction model.

7. A wave energy power generation prediction system based on multimodal data, characterized in that: It includes acquisition module, data fusion module and prediction module; The acquisition module is used to acquire the environmental data set, wave data set and image data set of each preset position in the target watershed within the first time period, and the target watershed is the watershed where the wave energy power generation device is located; The data fusion module is used to fuse the environmental data set, the wave data set and the image data set into multi-modal time series data based on the time sequence; The prediction module is used to input the multimodal time series data into a preset power prediction model, so that the power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period; The power prediction model is obtained by training an initial power prediction model based on historical multimodal time series data, and the initial power prediction model is constructed based on an Autoformer model.

8. A wave energy power generation prediction system based on multimodal data as claimed in claim 7, characterized in that: The data fusion module fuses the environmental data set, the wave data set and the image data set into multimodal time series data based on the time sequence, including: Extracting corresponding environmental feature sets, wave feature sets and image feature sets from the environmental data set, wave data set and image data set respectively; Based on the time sequence, the environmental feature set, the wave feature set and the image feature set are respectively converted into corresponding environmental feature time series data, wave feature time series data and image feature time series data; After aligning the timestamps of the environmental feature time series data, the wave feature time series data, and the image feature time series data, they are fused and constructed into the multimodal time series data.

9. A wave energy power generation prediction system based on multimodal data as claimed in claim 7, characterized in that: The power prediction model extracts trend items and period items in the multimodal time series data based on a deep decomposition architecture, and generates corresponding trend prediction results and period prediction results, and then combines the trend prediction results and the period prediction results to generate a power generation prediction result for a second time period, including: The multimodal time series data is subjected to layer-by-layer sequence decomposition and data prediction through a preset multi-layer encoder-decoder to generate trend items, period items, trend prediction results and period prediction results of each layer; Among them, in any layer of encoder-decoder, the encoder of the current layer performs sequence decomposition on the input data of the current layer to obtain the periodic item and trend item of the current layer; the decoder of the current layer predicts the periodic item and trend item of the current layer to generate the trend prediction result and period prediction result of the current layer; the periodic item of the current layer is input to the encoder-decoder of the next layer; the input data of the encoder-decoder of the first layer is the multimodal time series data; According to the weight parameters inside the power prediction model, the trend prediction results and period prediction results of the encoder-decoder of each layer are weightedly fused through a preset linear layer fusion layer to generate the power generation prediction result of the second time period.

10. A wave energy power generation prediction system based on multimodal data as claimed in claim 7, characterized in that: The wave energy power generation power prediction system also includes a model training module, which is used to train the initial power prediction model according to the historical multimodal time series data to obtain the power prediction model, including an acquisition unit, a time series data construction unit, a training set construction unit, a training unit and a model update unit; Wherein, the acquisition unit is used to acquire the historical environment data set, the historical wave data set, the historical image data set and the historical power generation data set of the wave energy power generation device at each preset location in the target watershed; The time series data construction unit is used to fuse the historical environment data set, the historical wave data set and the historical image data set into the historical multimodal time series data based on the time sequence; The training set construction unit is used to construct a training data set according to the historical multimodal time series data and the historical power generation data set; The training unit is used to input the training data set into the initial power prediction model, so that the initial power prediction model adjusts the internal parameters of the initial power prediction model according to a preset loss function to obtain corresponding training parameters; The model updating unit is used to update the initial power prediction model according to the training parameters to obtain the power prediction model.

Images