New energy intelligent distribution and prediction system based on big data optimization

Abstract

This invention discloses a big data-optimized intelligent allocation and prediction system for new energy sources, comprising: a data acquisition module responsible for collecting data from multiple sources; a data processing and analysis module responsible for cleaning and integrating the collected data to form a dataset that can be used for analysis, constructing a prediction model, and making high-precision predictions of new energy output and grid load; an intelligent scheduling module responsible for automatically adjusting the allocation ratio of new energy and the charging and discharging strategy of the energy storage system based on the predicted new energy output and grid load; and a visualization monitoring and decision support module responsible for constructing a visualization interface to display the real-time status of new energy production, storage, allocation, and use, providing data support and decision-making basis for decision-makers. In this invention, LSTM is used to construct a prediction model for new energy output and grid load to make more accurate predictions of new energy output, and an intelligent scheduling module is developed to dynamically adjust energy allocation based on the prediction results to improve energy utilization.

Description

Technical Field

[0001] This invention relates to the field of intelligent new energy allocation technology, and in particular to an intelligent new energy allocation and prediction system based on big data optimization. Background Technology

[0002] In the process of energy transition, coal and coal-fired power generation will gradually shift from being the primary energy source to a backup energy source, a supporting energy source, and a peak-shaving energy source. New energy sources are characterized by being clean, renewable, and having low carbon emissions, and are of great significance for alleviating the energy crisis, reducing environmental pollution, and addressing climate change.

[0003] With increasing global emphasis on sustainable development, the application of new energy sources is becoming increasingly widespread. However, the supply of new energy sources is greatly affected by natural conditions (such as weather changes), resulting in intermittency and instability. Meanwhile, matching the supply and demand of new energy sources, their efficient utilization, and the stable operation of the power grid are the main challenges currently faced. The development of big data technology provides new ideas and methods for solving these problems. Therefore, a new energy intelligent allocation and prediction system based on big data optimization is proposed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies, such as the fact that the supply of new energy sources is greatly affected by natural conditions, resulting in intermittency and instability, and that matching the supply and demand of new energy sources, efficient utilization, and stable operation of the power grid are the main challenges currently faced. The invention proposes a new energy intelligent allocation and prediction system based on big data optimization.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A new energy intelligent allocation and forecasting system based on big data optimization includes:

[0007] Data acquisition module: responsible for collecting data from multiple sources, including real-time power generation, working status and fault information of new energy equipment (such as solar panels and wind turbines), meteorological data from meteorological stations (such as wind speed, wind direction, light intensity, temperature and humidity), and real-time data such as power grid load, voltage, current and frequency from power grid monitoring points.

[0008] Data processing and analysis module: responsible for cleaning and integrating the collected data to form a high-quality dataset that can be used for analysis, building predictive models, and making high-precision predictions of new energy output and grid load;

[0009] Intelligent scheduling module: Based on the predicted new energy output and grid load, it automatically adjusts the allocation ratio of new energy and the charging and discharging strategy of the energy storage system.

[0010] Visual monitoring and decision support module: responsible for building a visual interface to display the status of new energy production, storage, distribution and use in real time, providing data support and decision-making basis for decision-makers, and providing functions such as historical data analysis, alarms and early warnings, data analysis reports, simulation and remote control;

[0011] Data storage and management module: responsible for persistent storage and efficient access to data, and building knowledge bases and case libraries;

[0012] The data acquisition module transmits the collected multi-source heterogeneous data (such as new energy equipment data, meteorological data, power grid data, etc.) to the data processing and analysis module in real time. The data processing and analysis module receives real-time data from the data acquisition module and historical data from the data storage and management module. The data processing and analysis module provides the processed data (such as cleaned historical data, real-time data, prediction results, etc.) to the intelligent scheduling module and the data storage and management module. The intelligent scheduling module transmits the optimized scheduling strategy, prediction results, and related optimization parameters to the visualization monitoring and decision support module and the data storage and management module.

[0013] The above technical solution further includes:

[0014] Furthermore, the data acquisition module includes a multi-source heterogeneous data acquisition unit, a data format parsing unit, a data caching and queuing unit, and a data verification unit. The multi-source heterogeneous data acquisition unit is responsible for collecting real-time and diverse data from multiple sources (such as various new energy devices like solar panels, wind turbines, weather stations, and power grid monitoring points), covering key information such as power generation, wind speed, light intensity, and power grid load. The data format parsing unit parses the received data and converts it into a unified data format within the system. During the data acquisition process, a data caching and queuing mechanism is introduced. The data caching and queuing unit is responsible for temporarily storing the parsed data in the cache or placing it in the message queue for subsequent processing. The data verification unit performs preliminary verification on the acquired data. The multi-source heterogeneous data acquisition unit transmits the acquired raw data to the data format parsing unit for format parsing. The parsed data is transmitted to the data caching and queuing unit for temporary storage or queuing for processing. Data in the cache or queue is retrieved sequentially and enters the data verification unit for preliminary verification.

[0015] Furthermore, the data processing and analysis module includes a data receiving unit, a data processing unit, a quality control unit, and a data analysis unit. The data receiving unit is responsible for receiving the collected data transmitted from the data acquisition module. The data processing unit uses data cleaning algorithms, deduplication techniques, and transformation tools to process the data. The quality control unit performs quality checks on the preprocessed data to form a high-quality dataset that can be used for analysis. The data analysis unit constructs a prediction model to conduct in-depth analysis of the processed data to predict new energy output and grid load. The data analysis unit transmits the prediction results to the intelligent scheduling and optimization algorithm module. The data processing and analysis module adopts distributed computing, dividing the collected data transmitted from the data acquisition module into small blocks and processing them in parallel on multiple nodes.

[0016] Furthermore, the intelligent scheduling module includes an intelligent scheduling and optimization unit, a control execution unit, a user feedback collection unit, and a user feedback analysis unit. The intelligent scheduling and optimization unit receives the prediction results from the data analysis unit. Based on the prediction results and set optimization objectives (economy, environmental protection, reliability), the intelligent scheduling and optimization unit automatically calculates the optimal energy configuration scheme and transmits the scheduling instructions and optimization scheme to the control execution unit. The control execution unit receives the scheduling instructions and optimization scheme from the intelligent scheduling and optimization unit and controls the grid connection ratio of new energy equipment and the charging and discharging strategy of the energy storage system. The data acquisition module monitors the grid operation status after the control execution unit controls the charging and discharging strategy and feeds back the monitoring data to the data processing and analysis module for continuous optimization. The user feedback collection unit is responsible for collecting user feedback information. The user feedback analysis unit is responsible for analyzing the user feedback information and extracting key information related to new energy scheduling. The user feedback analysis unit receives the feedback information transmitted by the user feedback collection unit and outputs the key feedback information. The intelligent scheduling and optimization unit adjusts the scheduling using optimization algorithms based on the key information from user feedback.

[0017] Furthermore, the visualization monitoring and decision support module includes a visualization interface unit, a decision support unit, and a remote control unit. The visualization interface unit receives data from the data processing unit and displays the status of new energy production, storage, distribution, and use in real time through charts, maps, and other forms. Simultaneously, the visualization interface unit is connected to an alarm and early warning system to ensure timely notification to users in case of abnormal situations. The decision support unit, based on historical and real-time data in the data storage and management module, utilizes data analysis reports and simulations to provide decision-makers with comprehensive data support and decision-making basis. In addition, the decision support unit is connected to the remote control unit to achieve remote control of new energy equipment and energy storage systems. The remote control unit receives instructions from the decision support unit to achieve remote monitoring and control of new energy equipment and energy storage systems. Simultaneously, the remote control unit transmits the operating status and feedback information of the equipment to the data acquisition module and the visualization interface unit.

[0018] Furthermore, the data storage and management module includes a data storage unit, a data integration unit, a data access optimization unit, and a knowledge base and case study management unit. The data storage unit is responsible for data storage. The data integration unit is responsible for extracting data from multiple data sources, cleaning, transforming, and integrating it to build a unified data warehouse. The data warehouse includes a knowledge base and a case study library. The data access optimization unit is responsible for optimizing the data storage structure and query algorithms. The knowledge base and case study management unit builds and manages the knowledge base and case study library in the new energy field, supporting the accumulation, classification, indexing, and updating of knowledge. It utilizes an efficient distributed storage system to ensure that the cleaned and integrated data can be accessed and called quickly. The data integration unit extracts data from multiple data sources and writes the cleaned and transformed data into the data storage unit for persistent storage. The data access optimization unit designs and optimizes query algorithms and data indexes based on the data structure and storage characteristics in the data storage unit. The data integration unit transmits the integrated data to the data warehouse management unit.

[0019] Furthermore, the data analysis unit includes a data acquisition subunit, a feature extraction and selection subunit, a model training subunit, a model evaluation subunit, a model selection subunit, and a model deployment and application subunit. The data acquisition subunit is responsible for acquiring historical data from the data storage and management module and real-time data from the data acquisition module. The feature extraction and selection subunit is responsible for extracting and selecting features suitable for analysis and prediction from the processed data. The model training subunit is responsible for training the prediction model using the newly collected data. The model evaluation subunit is responsible for evaluating the newly trained model using evaluation metrics to verify its effectiveness under new conditions. The model selection subunit is responsible for selecting the optimal model as the updated model based on the model evaluation results. The model deployment and application subunit is responsible for deploying the updated model to the actual application environment and applying it to new data analysis tasks to predict new energy output (such as solar power generation and wind power generation). Based on feedback from actual applications and new data, the model is continuously optimized to improve prediction accuracy and performance.

[0020] Furthermore, in the model training subunit, LSTM is used for prediction. The specific steps are as follows:

[0021] Data preparation:

[0022] Time series feature data is obtained from the feature extraction and selection sub-unit. The time series feature data is divided into time windows of fixed length. Features are extracted from each time window. The feature dataset is divided into training set, validation set and test set with proportions of 70%, 15% and 15%, respectively.

[0023] Model building:

[0024] Building a prediction model using LSTM:

[0025] Forgotten Gate: f t =σ(W f ·[h t-1 ,x t ]+b f )

[0026] Among them, W f This is the weight matrix of the forget gate, b f It is the bias term, σ is the sigmoid function, [h t-1 ,x t ] indicates that h t-1 and x t Concatenate them into a single vector;

[0027] Input gate: The input gate is responsible for updating the cell state. The input gate consists of two parts: a sigmoid layer, which determines which information will be updated; and a tanh layer, which creates a new candidate value vector. The new candidate value vector is added to the cell state. Finally, the two pieces of information are multiplied to update the cell state.

[0028] sigmoid layer: i t =σ(W i ·[h t-1 ,x t ]+b i );

[0029] tanh layer:

[0030] Cell status update:

[0031] Output gate: The output gate determines which part of the information based on the cell state is used for output, according to the current input tx. t The state h of the hidden layer at the previous time step t-1 And the latest cell state C t The output h at the current time step is determined by the combined action of the sigmoid and tanh functions. t ;

[0032] sigmoid layer: o t =σ(W o ·[h t-1 ,x t ]+b o );

[0033] Output hidden state: h t =o t *tanh(C t );

[0034] Training process:

[0035] Forward propagation: For the input at each time step, calculate the hidden state and output according to the RNN;

[0036] Calculate the loss: Use a loss function to measure the difference between the model's predictions and the actual labels;

[0037] Backpropagation: Calculates the gradient of the loss with respect to the model parameters using the time backpropagation algorithm;

[0038] Parameter update: Use Adam to update model parameters based on gradients;

[0039] Iterative training: Repeat the process of forward propagation, loss calculation, backpropagation, and parameter update until the preset number of training rounds or error standard is reached;

[0040] Testing and Reasoning:

[0041] The processed new data is then input into the trained LSTM model to predict new energy output or grid load.

[0042] Furthermore, the intelligent scheduling and optimization unit calculates the optimal energy configuration scheme, specifically through the following steps:

[0043] Data collection and preprocessing: Obtain forecast data on new energy output (such as solar and wind power generation) and real-time data on grid load from the data processing and analysis module;

[0044] Multi-objective optimization analysis: Using multi-objective evolutionary algorithms, energy allocation schemes are optimized by comprehensively considering multiple objectives such as economic efficiency (e.g., cost minimization), environmental protection (e.g., carbon emission reduction), and grid stability.

[0045] Scheduling strategy formulation: Based on the results of optimization analysis, specific scheduling strategies are formulated, including the allocation ratio of new energy sources, storage strategies (such as the charging and discharging plan of battery energy storage systems), and backup energy scheduling plans.

[0046] Dispatch execution and monitoring: After the dispatch strategy is formulated, the system automatically executes the dispatch instructions and tracks the status of new energy power generation, grid load and energy storage system in real time through the monitoring system.

[0047] Furthermore, in the multi-objective optimization analysis, a multi-objective evolutionary algorithm is used for optimization analysis. The specific steps are as follows:

[0048] Problem definition and goal setting: Clearly define the goals of the optimization problem, namely, specific quantitative indicators of economy, environmental protection and power grid stability. For example, economy may be aimed at minimizing total cost, environmental protection may be aimed at reducing carbon emissions, and power grid stability may involve indicators such as voltage stability and frequency fluctuation.

[0049] Encoding and initializing the population: Encode the energy allocation scheme (such as the allocation ratio of various energy sources, the charging and discharging strategy of the energy storage system, etc.) into chromosomes (i.e. individuals) in the genetic algorithm, and randomly generate an initial population;

[0050] Fitness assessment: Based on the set objective function (or fitness function), the fitness value of each individual in the population is calculated. The fitness value reflects the performance of the energy allocation scheme represented by that individual in achieving the optimization objective.

[0051] Selection operation: Based on fitness values, select a subset of superior individuals from the current population as parents to generate the next generation of the population. The selection operation usually follows the principle of "survival of the fittest".

[0052] Crossover and mutation: Selected parent individuals are subjected to crossover (i.e., gene recombination) and mutation operations to produce new offspring individuals. Crossover operation helps to combine the superior genes of different individuals, while mutation operation helps to introduce new gene combinations and increase the diversity of the population.

[0053] New population generation and iteration: The offspring individuals produced by crossover and mutation are merged with the parent individuals (or a portion thereof) to form a new population. Then, fitness evaluation, selection, crossover, and mutation operations are repeated until the stopping condition is met (such as reaching a preset number of iterations, or the fitness value no longer significantly increasing, etc.).

[0054] Results Output and Evaluation: Output the best individual (or several best individuals) in the final population as the result of the optimization analysis. Simultaneously evaluate the feasibility and effectiveness of these results in practical applications.

[0055] The present invention has the following beneficial effects:

[0056] 1. In this invention, LSTM is used to construct a prediction model for new energy output and grid load, so as to make more accurate predictions of new energy output. An intelligent scheduling module is developed to dynamically adjust energy allocation based on the prediction results in order to improve energy utilization.

[0057] 2. In this invention, a multi-objective evolutionary algorithm is used to dynamically adjust the grid connection ratio of new energy sources and the charging and discharging strategy of energy storage systems based on real-time predicted new energy output and grid load, so as to ensure supply and demand balance and realize real-time monitoring and optimized control of grid operation status. Attached Figure Description

[0058] Figure 1 This is a system block diagram of the new energy intelligent allocation and prediction system based on big data optimization proposed in this invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Please see Figure 1 As shown, this invention is a new energy intelligent allocation and prediction system based on big data optimization, comprising:

[0061] Data acquisition module: responsible for collecting data from multiple sources, including real-time power generation, working status and fault information of new energy equipment (such as solar panels and wind turbines), meteorological data from meteorological stations (such as wind speed, wind direction, light intensity, temperature and humidity), and real-time data such as power grid load, voltage, current and frequency from power grid monitoring points.

[0062] Data processing and analysis module: responsible for cleaning and integrating the collected data to form a high-quality dataset that can be used for analysis, building predictive models, and making high-precision predictions of new energy output and grid load;

[0063] Intelligent scheduling module: Based on the predicted new energy output and grid load, it automatically adjusts the allocation ratio of new energy and the charging and discharging strategy of the energy storage system.

[0064] Visual monitoring and decision support module: responsible for building a visual interface to display the status of new energy production, storage, distribution and use in real time, providing data support and decision-making basis for decision-makers, and providing functions such as historical data analysis, alarms and early warnings, data analysis reports, simulation and remote control;

[0065] Data storage and management module: responsible for persistent storage and efficient access to data, and building knowledge bases and case libraries;

[0066] The data acquisition module transmits the collected multi-source heterogeneous data (such as new energy equipment data, meteorological data, power grid data, etc.) to the data processing and analysis module in real time. The data processing and analysis module receives real-time data from the data acquisition module and historical data from the data storage and management module. The data processing and analysis module provides the processed data (such as cleaned historical data, real-time data, prediction results, etc.) to the intelligent scheduling module and the data storage and management module. The intelligent scheduling module transmits the optimized scheduling strategy, prediction results, and related optimization parameters to the visualization monitoring and decision support module and the data storage and management module.

[0067] The working principle of the new energy intelligent allocation and prediction system based on big data optimization proposed in this invention is as follows: sensors are installed on new energy equipment (such as solar panels and wind turbines) to collect data such as power generation, operating status (such as operation / shutdown), and fault information in real time. At the same time, meteorological sensors are deployed at meteorological stations to collect meteorological data such as wind speed, wind direction, light intensity, temperature, and humidity, while power monitoring equipment is installed at power grid monitoring points to collect real-time data such as power grid load, voltage, current, and frequency.

[0068] Using IoT technology, the collected data is transmitted to the data center or cloud server via wireless or wired means (such as Wi-Fi, LoRa, NB-IoT, fiber optics, etc.) to ensure the real-time performance and security of data transmission, and to adopt encryption technology and data verification mechanisms.

[0069] After receiving data from the data acquisition module, the data processing and analysis module first performs data cleaning to remove duplicate, erroneous, and incomplete data records, identifies and processes outliers, and integrates the cleaned multi-source heterogeneous data (such as new energy equipment data, meteorological data, and power grid data) according to a unified data format and standard to form a high-quality dataset. Using LSTM, a prediction model for new energy production and power grid load is built based on historical and real-time data. Through continuous training and optimization of the model, the prediction accuracy is improved.

[0070] The intelligent scheduling module implements the calculation and optimization of the allocation ratio of new energy and the charging and discharging strategy of energy storage system based on the predicted new energy output and grid load, combined with the current status of energy storage system (such as power and charging and discharging efficiency). It uses a multi-objective evolutionary algorithm to analyze and optimize the new energy allocation ratio and the charging and discharging strategy of energy storage system. Based on real-time data feedback, it dynamically adjusts the scheduling strategy to ensure the maximum utilization of new energy and the stable operation of the grid. The optimized scheduling strategy is transformed into specific control commands and sent to new energy equipment and energy storage system for execution through remote control system.

[0071] The visualization monitoring and decision support module constructs a user-friendly visual interface, displaying real-time status diagrams, reports, dashboards, etc., of new energy production, storage, distribution, and use. It provides functions such as historical data analysis, alarms and warnings, data analysis reports, simulation, and remote control. Decision-makers can quickly understand the system status through the interface and make informed decisions.

[0072] The data storage and management module adopts a distributed database to achieve persistent data storage and efficient access. It organizes key information such as historical data, prediction models, and scheduling strategies into a knowledge base and case library to facilitate subsequent analysis and reuse.

[0073] In one embodiment, the data acquisition module includes a multi-source heterogeneous data acquisition unit, a data format parsing unit, a data caching and queuing unit, and a data verification unit. The multi-source heterogeneous data acquisition unit is responsible for collecting real-time and diverse data from multiple sources (such as various new energy devices like solar panels, wind turbines, weather stations, and power grid monitoring points), covering key information such as power generation, wind speed, light intensity, and power grid load. The data format parsing unit parses the received data and converts it into a unified data format within the system. During the data acquisition process, a data caching and queuing mechanism is introduced. The data caching and queuing unit is responsible for temporarily storing the parsed data in the cache or placing it in the message queue for subsequent processing. The data verification unit performs preliminary verification on the acquired data. The multi-source heterogeneous data acquisition unit transmits the acquired raw data to the data format parsing unit for format parsing. The parsed data is transmitted to the data caching and queuing unit for temporary storage or queuing for processing. Data in the cache or queue is retrieved sequentially and enters the data verification unit for preliminary verification.

[0074] In one embodiment, the data processing and analysis module includes a data receiving unit, a data processing unit, a quality control unit, and a data analysis unit. The data receiving unit receives the collected data transmitted from the data acquisition module. The data processing unit processes the data using data cleaning algorithms, deduplication techniques, and transformation tools. The quality control unit performs quality checks on the preprocessed data to form a high-quality dataset suitable for analysis. The data analysis unit constructs a prediction model to conduct in-depth analysis of the processed data, predicting new energy output and grid load. The data analysis unit transmits the prediction results to the intelligent scheduling and optimization algorithm module. The data processing and analysis module employs distributed computing, dividing the collected data transmitted from the data acquisition module into small blocks and processing them in parallel on multiple nodes.

[0075] In one embodiment, the intelligent scheduling module includes an intelligent scheduling and optimization unit, a control execution unit, a user feedback collection unit, and a user feedback analysis unit. The intelligent scheduling and optimization unit receives prediction results from the data analysis unit. Based on the prediction results and set optimization objectives (economy, environmental friendliness, reliability), the intelligent scheduling and optimization unit automatically calculates the optimal energy configuration scheme and transmits the scheduling instructions and optimization scheme to the control execution unit. The control execution unit receives the scheduling instructions and optimization scheme from the intelligent scheduling and optimization unit and controls the grid connection ratio of new energy equipment and the charging and discharging strategy of the energy storage system. The data acquisition module monitors the grid operation status after the control execution unit controls the charging and discharging strategy and feeds back the monitoring data to the data processing and analysis module for continuous optimization. The user feedback collection unit is responsible for collecting user feedback information. The user feedback analysis unit is responsible for analyzing the user feedback information and extracting key information related to new energy scheduling. The user feedback analysis unit receives the feedback information transmitted by the user feedback collection unit and outputs the key feedback information. The intelligent scheduling and optimization unit adjusts the scheduling based on the key information from the user feedback using an optimization algorithm.

[0076] In one embodiment, the aforementioned visualization monitoring and decision support module includes a visualization interface unit, a decision support unit, and a remote control unit. The visualization interface unit receives data from the data processing unit and displays the status of new energy production, storage, distribution, and use in real time through charts, maps, and other formats. Simultaneously, the visualization interface unit is connected to an alarm and early warning system to ensure timely notification to users in case of abnormal situations. The decision support unit, based on historical and real-time data from the data storage and management module, utilizes data analysis reports and simulations to provide decision-makers with comprehensive data support and decision-making basis. Furthermore, the decision support unit is connected to the remote control unit to achieve remote control of new energy equipment and energy storage systems. The remote control unit receives instructions from the decision support unit to achieve remote monitoring and control of new energy equipment and energy storage systems. Simultaneously, the remote control unit transmits the operating status and feedback information of the equipment to the data acquisition module and the visualization interface unit.

[0077] In one embodiment, the data storage and management module includes a data storage unit, a data integration unit, a data access optimization unit, and a knowledge base and case study management unit. The data storage unit is responsible for data storage. The data integration unit extracts data from multiple data sources, cleans, transforms, and integrates it to build a unified data warehouse. The data warehouse includes a knowledge base and a case study library. The data access optimization unit optimizes the data storage structure and query algorithms. The knowledge base and case study management unit builds and manages a knowledge base and case study library in the new energy field, supporting knowledge accumulation, classification, indexing, and updating. It utilizes an efficient distributed storage system to ensure that the cleaned and integrated data can be quickly accessed and retrieved. The data integration unit extracts data from multiple data sources and writes the cleaned and transformed data into the data storage unit for persistent storage. The data access optimization unit designs and optimizes query algorithms and data indexes based on the data structure and storage characteristics of the data storage unit. The data integration unit then transmits the integrated data to the data warehouse management unit.

[0078] In one embodiment, the data analysis unit includes a data acquisition subunit, a feature extraction and selection subunit, a model training subunit, a model evaluation subunit, a model selection subunit, and a model deployment and application subunit. The data acquisition subunit is responsible for acquiring historical data from the data storage and management module and real-time data from the data acquisition module. The feature extraction and selection subunit is responsible for extracting and selecting features suitable for analysis and prediction from the processed data. The model training subunit is responsible for training a prediction model using the newly collected data. The model evaluation subunit is responsible for evaluating the newly trained model using evaluation metrics to verify its effectiveness under new conditions. The model selection subunit is responsible for selecting the optimal model as the updated model based on the model evaluation results. The model deployment and application subunit is responsible for deploying the updated model to the actual application environment and applying it to new data analysis tasks to predict new energy output (such as solar power generation and wind power generation). Based on feedback from actual applications and new data, the model is continuously optimized to improve prediction accuracy and performance.

[0079] In one embodiment, for the above model training subunit, LSTM is used for prediction within the model training subunit, with the following specific steps:

[0080] Data preparation:

[0081] Time series feature data is obtained from the feature extraction and selection sub-unit. The time series feature data is divided into time windows of fixed length. Features are extracted from each time window. The feature dataset is divided into training set, validation set and test set with proportions of 70%, 15% and 15%, respectively.

[0082] Model building:

[0083] Building a prediction model using LSTM:

[0084] Forgotten Gate: f t =σ(W f ·[h t-1 ,x t ]+b f )

[0085] Among them, W f This is the weight matrix of the forget gate, b f It is the bias term, σ is the sigmoid function, [h t-1 ,x t ] indicates that h t-1 and x t Concatenate them into a single vector;

[0086] Input gate: The input gate is responsible for updating the cell state. The input gate consists of two parts: a sigmoid layer, which determines which information will be updated; and a tanh layer, which creates a new candidate value vector. The new candidate value vector is added to the cell state. Finally, the two pieces of information are multiplied to update the cell state.

[0087] sigmoid layer: i t =σ(W i ·[h t-1 ,x t ]+b i );

[0088] tanh layer:

[0089] Cell status update:

[0090] Output gate: The output gate determines which part of the cell state-based information is used for output, based on the current input. The state h of the hidden layer at the previous moment t-1 And the latest cell state C t The output h at the current time step is determined by the combined action of the sigmoid and tanh functions. t ;

[0091] sigmoid layer: o t =σ(W o ·[ht-1 ,x t ]+b o );

[0092] Output hidden state: h t =o t *tanh(C t );

[0093] Training process:

[0094] Forward propagation: For the input at each time step, calculate the hidden state and output according to the RNN;

[0095] Calculate the loss: Use a loss function to measure the difference between the model's predictions and the actual labels;

[0096] Backpropagation: Calculates the gradient of the loss with respect to the model parameters using the time backpropagation algorithm;

[0097] Parameter update: Use Adam to update model parameters based on gradients;

[0098] Iterative training: Repeat the process of forward propagation, loss calculation, backpropagation, and parameter update until the preset number of training rounds or error standard is reached;

[0099] Testing and Reasoning:

[0100] The processed new data is then input into the trained LSTM model to predict new energy output or grid load.

[0101] In one embodiment, for the aforementioned intelligent scheduling and optimization unit, the intelligent scheduling and optimization unit calculates the optimal energy configuration scheme, specifically through the following steps:

[0102] Data collection and preprocessing: Obtain forecast data on new energy output (such as solar and wind power generation) and real-time data on grid load from the data processing and analysis module;

[0103] Assuming forecast data shows that solar power generation will peak in the next 24 hours, while grid load is expected to peak in the afternoon;

[0104] Multi-objective optimization analysis: Using multi-objective evolutionary algorithms, energy allocation schemes are optimized by comprehensively considering multiple objectives such as economic efficiency (e.g., cost minimization), environmental protection (e.g., carbon emission reduction), and grid stability.

[0105] During the optimization process, we will consider increasing the allocation ratio of solar power generation to reduce reliance on traditional energy sources, while ensuring sufficient power supply during peak grid load periods.

[0106] Scheduling strategy formulation: Based on the results of optimization analysis, specific scheduling strategies are formulated, including the allocation ratio of new energy sources, storage strategies (such as the charging and discharging plan of battery energy storage systems), and backup energy scheduling plans.

[0107] For example, it is decided to store excess electricity in a battery energy storage system during peak solar power generation periods so that it can be released and used during peak grid load periods;

[0108] Dispatch execution and monitoring: After the dispatch strategy is formulated, the system automatically executes the dispatch instructions and tracks the status of new energy power generation, grid load and energy storage system in real time through the monitoring system;

[0109] When the grid load begins to rise, the battery energy storage system automatically starts discharging to supplement the grid power supply.

[0110] In one embodiment, for the above multi-objective optimization analysis, a multi-objective evolutionary algorithm is used for optimization analysis, with the following specific steps:

[0111] Problem definition and goal setting: Clearly define the goals of the optimization problem, namely, specific quantitative indicators of economy, environmental protection and power grid stability. For example, economy may be aimed at minimizing total cost, environmental protection may be aimed at reducing carbon emissions, and power grid stability may involve indicators such as voltage stability and frequency fluctuation.

[0112] The goal is to minimize total costs (including generation costs, maintenance costs, etc.), while reducing carbon emissions and ensuring that voltage fluctuations in the power grid do not exceed a certain range during peak load periods.

[0113] Encoding and initializing the population: Encode the energy allocation scheme (such as the allocation ratio of various energy sources, the charging and discharging strategy of the energy storage system, etc.) into chromosomes (i.e. individuals) in the genetic algorithm, and randomly generate an initial population;

[0114] Each individual represents a specific energy configuration plan, which includes the allocation ratio of different energy sources such as solar, wind, and hydropower, as well as the charging and discharging plan of the energy storage system at different times of the day.

[0115] Fitness assessment: Based on the set objective function (or fitness function), the fitness value of each individual in the population is calculated. The fitness value reflects the performance of the energy allocation scheme represented by that individual in achieving the optimization objective.

[0116] Calculate the total cost, carbon emissions, and grid stability index for each individual, and then comprehensively evaluate its fitness value based on these indicators;

[0117] Selection operation: Based on fitness values, a subset of superior individuals are selected from the current population to serve as parents for the next generation. Selection typically follows the principle of "survival of the fittest."

[0118] Strategies such as roulette wheel selection or tournament selection are used to select individuals with higher fitness values ​​as parents.

[0119] Crossover and mutation: Selected parent individuals are subjected to crossover (i.e., gene recombination) and mutation operations to produce new offspring individuals. Crossover operation helps to combine the superior genes of different individuals, while mutation operation helps to introduce new gene combinations and increase the diversity of the population.

[0120] In crossover, one or more gene segments are randomly selected from two parent individuals and exchanged; in mutation, certain positions in the genes of an individual are randomly changed.

[0121] New population generation and iteration: The offspring individuals produced by crossover and mutation are merged with the parent individuals (or a portion thereof) to form a new population. Then, fitness evaluation, selection, crossover, and mutation operations are repeated until the stopping condition is met (such as reaching a preset number of iterations, or the fitness value no longer significantly increasing, etc.).

[0122] After multiple iterations, the individuals in the population gradually approached the optimal solution, that is, an energy configuration scheme that can minimize total cost, reduce carbon emissions, and ensure grid stability was found.

[0123] Results Output and Evaluation: Output the best individual (or several best individuals) in the final population as the result of the optimization analysis. Simultaneously evaluate the feasibility and effectiveness of these results in practical applications.

[0124] Output the optimal energy configuration scheme, including the allocation ratio of various energy sources and the charging and discharging plan of the energy storage system, and evaluate the effectiveness of the scheme in reducing total cost, reducing carbon emissions and improving grid stability.

[0125] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A new energy intelligent allocation and prediction system based on big data optimization, characterized in that, include: Data acquisition module: responsible for collecting data from multiple sources; Data processing and analysis module: responsible for cleaning and integrating the collected data to form a dataset that can be used for analysis, building predictive models, and making high-precision predictions of new energy production and grid load; Intelligent scheduling module: Based on the predicted new energy output and grid load, it automatically adjusts the allocation ratio of new energy and the charging and discharging strategy of the energy storage system. Visual monitoring and decision support module: responsible for building a visual interface to display the status of new energy production, storage, distribution and use in real time, providing data support and decision-making basis for decision-makers; Data storage and management module: responsible for persistent storage and efficient access to data, and building knowledge bases and case libraries; The data acquisition module transmits the collected multi-source heterogeneous data to the data processing and analysis module in real time. The data processing and analysis module receives real-time data from the data acquisition module and historical data from the data storage and management module. The data processing and analysis module provides the processed data to the intelligent scheduling module and the data storage and management module. The intelligent scheduling module transmits the optimized data to the visualization monitoring and decision support module and the data storage and management module. The data processing and analysis module includes a data receiving unit, a data processing unit, a quality control unit, and a data analysis unit. The data receiving unit is responsible for receiving the collected data transmitted from the data acquisition module. The data processing unit uses data cleaning algorithms, deduplication techniques, and transformation tools to process the data. The quality control unit performs quality checks on the preprocessed data to form a high-quality dataset that can be used for analysis. The data analysis unit builds a prediction model to conduct in-depth analysis of the processed data to predict new energy output and grid load. The data analysis unit transmits the prediction results to the intelligent scheduling and optimization algorithm module. The data processing and analysis module adopts distributed computing, dividing the collected data transmitted from the data acquisition module into small blocks and processing them in parallel on multiple nodes. The intelligent scheduling module includes an intelligent scheduling and optimization unit, a control execution unit, a user feedback collection unit, and a user feedback analysis unit. The intelligent scheduling and optimization unit receives the prediction results from the data analysis unit. Based on the prediction results and set optimization objectives, the intelligent scheduling and optimization unit automatically calculates the optimal energy configuration scheme and transmits the scheduling instructions and optimization scheme to the control execution unit. The control execution unit receives the scheduling instructions and optimization scheme from the intelligent scheduling and optimization unit and controls the grid connection ratio of new energy equipment and the charging and discharging strategy of the energy storage system. The data acquisition module monitors the grid operation status after the control execution unit controls the charging and discharging strategy in real time and feeds back the monitoring data to the data processing and analysis module for continuous optimization. The user feedback collection unit is responsible for collecting user feedback information. The user feedback analysis unit is responsible for analyzing the user feedback information and extracting key information related to new energy scheduling. The user feedback analysis unit receives the feedback information transmitted by the user feedback collection unit and outputs the key feedback information. The intelligent scheduling and optimization unit adjusts the scheduling using optimization algorithms based on the key information from user feedback. The visualization monitoring and decision support module includes a visualization interface unit, a decision support unit, and a remote control unit. The visualization interface unit receives data from the data processing unit and displays the real-time status of new energy production, storage, distribution, and use. Simultaneously, the visualization interface unit is connected to an alarm and early warning system to ensure timely notification to users in case of abnormal situations. The decision support unit, based on historical and real-time data from the data storage and management module, utilizes data analysis reports and simulations to provide decision-makers with comprehensive data support and decision-making basis. Furthermore, the decision support unit is connected to the remote control unit to achieve remote control of new energy equipment and energy storage systems. The remote control unit receives instructions from the decision support unit to achieve remote monitoring and control of new energy equipment and energy storage systems. Simultaneously, the remote control unit transmits the operating status and feedback information of the equipment to the data acquisition module and the visualization interface unit. The data analysis unit includes a data acquisition subunit, a feature extraction and selection subunit, a model training subunit, a model evaluation subunit, a model selection subunit, and a model deployment and application subunit. The data acquisition subunit is responsible for acquiring historical data from the data storage and management module and real-time data from the data acquisition module. The feature extraction and selection subunit is responsible for extracting and selecting features suitable for analysis and prediction from the processed data. The model training subunit is responsible for training a prediction model using the newly collected data. The model evaluation subunit is responsible for evaluating the newly trained model using evaluation metrics to verify its effectiveness under new conditions. The model selection subunit is responsible for selecting the optimal model as the updated model based on the model evaluation results. The model deployment and application subunit is responsible for deploying the updated model to the actual application environment and applying it to new data analysis tasks to predict new energy production. The intelligent scheduling and optimization unit calculates the optimal energy configuration scheme, specifically through the following steps: Data collection and preprocessing: Obtain forecast data on new energy production and real-time data on grid load from the data processing and analysis module; Multi-objective optimization analysis: Optimize energy allocation schemes using multi-objective evolutionary algorithms; Scheduling strategy formulation: Based on the results of optimization analysis, specific scheduling strategies are formulated, including the allocation ratio of new energy sources, storage strategies, and backup energy scheduling plans.

2. The new energy intelligent allocation and prediction system based on big data optimization according to claim 1, characterized in that, The data acquisition module includes a multi-source heterogeneous data acquisition unit, a data format parsing unit, a data caching and queuing unit, and a data verification unit. The multi-source heterogeneous data acquisition unit is responsible for collecting data from multiple sources. The data format parsing unit parses the received data and converts it into a unified data format within the system. During the data acquisition process, a data caching and queuing mechanism is introduced. The data caching and queuing unit is responsible for temporarily storing the parsed data in the cache or placing it in the message queue for subsequent processing. The data verification unit performs preliminary verification on the acquired data. The multi-source heterogeneous data acquisition unit transmits the acquired raw data to the data format parsing unit for format parsing. The parsed data is transmitted to the data caching and queuing unit for temporary storage or queuing for processing. Data in the cache or queue is retrieved sequentially and enters the data verification unit for preliminary verification.

3. The new energy intelligent allocation and prediction system based on big data optimization according to claim 1, characterized in that, The data storage and management module includes a data storage unit, a data integration unit, a data access optimization unit, and a knowledge base and case study management unit. The data storage unit is responsible for data storage. The data integration unit is responsible for extracting data from multiple data sources, cleaning, transforming, and integrating it to build a unified data warehouse. The data warehouse includes a knowledge base and a case study library. The data access optimization unit is responsible for optimizing the data storage structure and query algorithms. The knowledge base and case study management unit builds and manages the knowledge base and case study library in the new energy field, supporting the accumulation, classification, indexing, and updating of knowledge. The data integration unit extracts data from multiple data sources and writes the cleaned and transformed data into the data storage unit for persistent storage. The data access optimization unit designs and optimizes query algorithms and data indexes based on the data structure and storage characteristics of the data storage unit. The data integration unit transmits the integrated data to the data warehouse management unit.

4. The new energy intelligent allocation and prediction system based on big data optimization according to claim 1, characterized in that, In the model training subunit, LSTM is used for prediction. The specific steps are as follows: Data preparation: Time series feature data is obtained from the feature extraction and selection sub-unit. The time series feature data is divided into time windows of fixed length. Features are extracted from each time window. The feature dataset is divided into training set, validation set, and test set with a ratio of 70%, 15%, and 15%, respectively. Model building: Building a prediction model using LSTM: Forgotten Gate: in, It is the weight matrix of the forget gate. It is a bias term. It is the sigmoid function. Indicates will and Concatenate them into a single vector; Input gate: The input gate is responsible for updating the cell state. The input gate consists of two parts: a sigmoid layer, which determines which information will be updated; and a tanh layer, which creates a new candidate value vector. The new candidate value vector is added to the cell state. Finally, the two pieces of information are multiplied to update the cell state. sigmoid layer: ; tanh layer: ; Cell status update: ; Output gate: The output gate determines which part of the cell state-based information is used for output, based on the current input. The state of the hidden layer at the previous moment and the latest cell status The output at the current time step is determined by the combined action of the sigmoid and tanh functions. ; sigmoid layer: ; Output hidden state: ; Training process: Forward propagation: For the input at each time step, calculate the hidden state and output according to the RNN; Calculate the loss: Use a loss function to measure the difference between the model's predictions and the actual labels; Backpropagation: Calculates the gradient of the loss with respect to the model parameters using the time backpropagation algorithm; Parameter update: Use Adam to update model parameters based on gradients; Iterative training: Repeat the process of forward propagation, loss calculation, backpropagation, and parameter update until the preset number of training rounds or error standard is reached; Testing and Reasoning: The processed new data is then input into the trained LSTM model to predict new energy output or grid load.

5. The new energy intelligent allocation and prediction system based on big data optimization according to claim 1, characterized in that, In multi-objective optimization analysis, a multi-objective evolutionary algorithm is used for optimization analysis. The specific steps are as follows: Problem definition and goal setting: Clearly define the goals of the optimization problem, namely, specific quantitative indicators of economic efficiency, environmental protection, and power grid stability; Encoding and initializing the population: Encode the energy allocation scheme into chromosomes in the genetic algorithm and randomly generate an initial population; Fitness assessment: Based on the set objective function, the fitness value of each individual in the population is calculated. The fitness value reflects the performance of the energy allocation scheme represented by that individual in achieving the optimization objective. Selection operation: Based on fitness values, select a subset of superior individuals from the current population as parents to generate the next generation of the population. The selection operation follows the principle of "survival of the fittest". Crossover and mutation: Crossover and mutation operations are performed on the selected parent individuals to produce new offspring individuals; New population generation and iteration: The offspring individuals generated by crossover and mutation are merged with the parent individuals to form a new population. Then, the fitness evaluation, selection, crossover and mutation operations are repeated until the stopping condition is met. Results Output and Evaluation: Output the best individual in the final population as the result of the optimization analysis, and evaluate the feasibility and effectiveness of these results in practical applications.

Images