Numerical weather prediction mass data processing method adaptive to new energy prediction

Abstract

The application relates to a numerical weather forecast mass data processing method suitable for new energy prediction. The method comprises the following steps: acquiring weather forecast data to be processed and a coordinate position corresponding to the weather forecast data to be processed; determining target weather forecast data from the weather forecast data to be processed according to the coordinate position; performing sorting processing on the target weather forecast data according to a prediction time corresponding to the target weather forecast data, so as to obtain sorted weather forecast data; and taking the sorted weather forecast data as a processing result of the weather forecast data to be processed. The method can reduce the processing time of weather forecast data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method for processing massive data in numerical weather forecasting adapted to new energy prediction. Background Technology

[0002] With the development of new energy technologies, large-scale grid connection of new energy sources poses challenges to the safe and stable operation of the power system. Power prediction is one of the key measures to solve the problem of grid connection of new energy sources. In the process of power prediction, making full use of NWP (Numerical Weather Prediction) data can improve the accuracy of power prediction.

[0003] NWP data is typically updated every 15 minutes, resulting in a large volume of data that is difficult to process online at high speed. Current technologies usually process historical NWP data in batches offline over a period of time, which takes a long time and results in poor data timeliness. This makes it impossible to fully utilize the potential patterns of meteorological characteristics to improve the accuracy of new energy power prediction.

[0004] Therefore, current renewable energy power prediction technologies suffer from the problem of long NWP data processing times. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for processing massive data of numerical weather forecasting that is adapted to new energy forecasting and can reduce processing time, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for processing massive amounts of numerical weather forecast data adapted to new energy prediction. The method includes:

[0007] Obtain the weather forecast data to be processed, and the coordinates corresponding to the weather forecast data to be processed;

[0008] Based on the coordinates, the target weather forecast data is determined from the weather forecast data to be processed;

[0009] Based on the forecast time corresponding to the target weather forecast data, the target weather forecast data is sorted to obtain sorted weather forecast data.

[0010] The sorted weather forecast data is used as the processing result of the weather forecast data to be processed.

[0011] In one embodiment, the coordinate position includes longitude and latitude; determining the target weather forecast data from the weather forecast data to be processed based on the coordinate position includes:

[0012] If the longitude position is within a preset longitude range and the latitude position is within a preset latitude range, the weather forecast data to be processed will be used as the target weather forecast data.

[0013] In one embodiment, determining the target weather forecast data from the weather forecast data to be processed based on the coordinate position further includes:

[0014] Determine the upper and lower limits of longitude corresponding to the longitude range, and determine the upper and lower limits of latitude corresponding to the latitude range;

[0015] Based on the upper longitude limit, the lower longitude limit, the upper latitude limit, and the lower latitude limit, interpolation lookup processing is performed on the weather forecast data to be processed to obtain the target weather forecast data.

[0016] In one embodiment, before sorting the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain sorted weather forecast data, the method further includes:

[0017] Obtain the filename of the target weather forecast data;

[0018] The forecast time for the target weather forecast data is obtained based on the data file name.

[0019] In one embodiment, the step of sorting the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain sorted weather forecast data includes:

[0020] The forecast time is bound to the target weather forecast data to obtain the bound weather forecast data;

[0021] Based on the forecast time, the bound weather forecast data is sorted to obtain the sorted weather forecast data.

[0022] In one embodiment, the bound weather forecast data includes at least one column, the at least one column including a forecast time column; the step of sorting the bound weather forecast data according to the forecast time to obtain the sorted weather forecast data includes:

[0023] Use the forecast time column in the bound weather forecast data as the index column;

[0024] The bound weather forecast data is sorted according to the index column to obtain the sorted weather forecast data.

[0025] In one embodiment, the step of sorting the bound weather forecast data according to the forecast time to obtain the sorted weather forecast data further includes:

[0026] If the coordinates and forecast times of at least two bound weather forecast data are the same, the target bound data is determined from the at least two bound weather forecast data.

[0027] The target-bound data is sorted to obtain the sorted weather forecast data.

[0028] In one embodiment, before sorting the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain sorted weather forecast data, the method further includes:

[0029] Obtain the correspondence between candidate stations and data grid points of the weather forecast data to be processed;

[0030] Based on the correspondence, the target grid point corresponding to the target station in the candidate stations is determined;

[0031] The weather forecast data to be processed corresponding to the target grid point is used as the target weather forecast data.

[0032] In one embodiment, before acquiring the weather forecast data to be processed and the coordinates corresponding to the weather forecast data to be processed, the method further includes:

[0033] Obtain weather forecast data samples and corresponding new energy output samples;

[0034] Based on the Pearson product-moment correlation coefficient between the new energy output sample and the weather forecast data sample, the target features are determined from the sample features of the weather forecast data sample;

[0035] Based on the target characteristics, the original weather forecast data is reduced to obtain the weather forecast data to be processed.

[0036] Secondly, this application also provides a numerical weather forecasting massive data processing device adapted to new energy prediction. The device includes:

[0037] The acquisition module is used to acquire the weather forecast data to be processed, as well as the coordinates of the weather forecast data to be processed.

[0038] The selection module is used to determine the target weather forecast data from the weather forecast data to be processed based on the coordinate position.

[0039] The sorting module is used to sort the target weather forecast data according to the forecast time corresponding to the target weather forecast data, so as to obtain sorted weather forecast data.

[0040] The processing module is used to take the sorted weather forecast data as the processing result of the weather forecast data to be processed.

[0041] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0042] Obtain the weather forecast data to be processed, and the coordinates corresponding to the weather forecast data to be processed;

[0043] Based on the coordinates, the target weather forecast data is determined from the weather forecast data to be processed;

[0044] Based on the forecast time corresponding to the target weather forecast data, the target weather forecast data is sorted to obtain sorted weather forecast data.

[0045] The sorted weather forecast data is used as the processing result of the weather forecast data to be processed.

[0046] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0047] Obtain the weather forecast data to be processed, and the coordinates corresponding to the weather forecast data to be processed;

[0048] Based on the coordinates, the target weather forecast data is determined from the weather forecast data to be processed;

[0049] Based on the forecast time corresponding to the target weather forecast data, the target weather forecast data is sorted to obtain sorted weather forecast data.

[0050] The sorted weather forecast data is used as the processing result of the weather forecast data to be processed.

[0051] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0052] Obtain the weather forecast data to be processed, and the coordinates corresponding to the weather forecast data to be processed;

[0053] Based on the coordinates, the target weather forecast data is determined from the weather forecast data to be processed;

[0054] Based on the forecast time corresponding to the target weather forecast data, the target weather forecast data is sorted to obtain sorted weather forecast data.

[0055] The sorted weather forecast data is used as the processing result of the weather forecast data to be processed.

[0056] The aforementioned numerical weather forecasting massive data processing method, device, computer equipment, storage medium, and computer program product adapted to new energy forecasting acquires the weather forecast data to be processed and the corresponding coordinates of the weather forecast data. Based on the coordinates, it determines the target weather forecast data from the weather forecast data to be processed. Based on the forecast time corresponding to the target weather forecast data, it sorts the target weather forecast data to obtain sorted weather forecast data, and uses the sorted weather forecast data as the processing result of the weather forecast data to be processed. It can also select the target weather forecast data from the massive weather forecast data to be processed based on the coordinates of the power plant, sort only the target weather forecast data according to the forecast time, and use the sorted weather forecast data for power plant power prediction. This reduces the amount of data to be processed in the power prediction process, reduces the data processing time, improves the data processing speed, and facilitates real-time online processing of weather forecast data. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating a method for processing massive amounts of numerical weather forecast data adapted to new energy prediction in one embodiment.

[0058] Figure 2 This is a flowchart illustrating a massive data processing method for numerical weather forecasting adapted to new energy prediction in another embodiment.

[0059] Figure 3 This is a flowchart illustrating a massive data processing method for numerical weather forecasting adapted to new energy prediction in another embodiment.

[0060] Figure 4 This is a structural block diagram of a numerical weather forecasting massive data processing device adapted for new energy prediction in one embodiment.

[0061] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] In one embodiment, such as Figure 1 As shown, a method for processing massive numerical weather forecast data adapted to new energy prediction is provided. This embodiment illustrates the method's application to a terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0064] Step S110: Obtain the weather forecast data to be processed, and the coordinates of the weather forecast data to be processed.

[0065] Among them, the weather forecast data to be processed can be numerical weather forecast data.

[0066] The coordinate position can be either longitude or latitude.

[0067] In practice, the weather forecast data to be processed can be input into the terminal. The weather forecast data to be processed contains coordinate positions. The terminal reads the coordinate positions from the weather forecast data to be processed and uses them as the coordinate positions corresponding to the weather forecast data to be processed.

[0068] In practical applications, numerical weather forecast data can be used as weather forecast data to be processed. The numerical weather forecast data is stored in nc (numerical control program file) format. The nc file contains longitude and latitude positions, which are used as coordinate positions.

[0069] For example, an acquired nc file is used as the weather forecast data to be processed. The nc file contains multidimensional variables, including but not limited to at least one of longitude (lon), latitude (lat), wind speed (WS), wind direction (WD), temperature (TEM), air density (Density), air pressure (PRS), total radiation (SR), direct radiation (SWDDIF), diffuse radiation (SWDDIR), high cloud cover (TCC), and total cloud cover (HCC). The lon and lat values ​​in the nc file are obtained as the coordinate positions corresponding to the weather forecast data to be processed.

[0070] Step S120: Determine the target weather forecast data from the weather forecast data to be processed based on the coordinate location.

[0071] The target weather forecast data can be selected weather forecast data used for new energy power prediction.

[0072] In practice, the longitude position of the weather forecast data to be processed can be compared with a preset longitude range, and the latitude position can be compared with a preset latitude range. Based on the comparison results, the target weather forecast data can be determined from the weather forecast data to be processed.

[0073] In practical applications, for weather forecast data to be processed, if the longitude and latitude are both within a pre-set longitude range, it is determined to be target weather forecast data; otherwise, if the longitude or latitude is not within a pre-set longitude range, it is determined not to be target weather forecast data.

[0074] For example, if the longitude range of the wind farm where new energy power prediction needs to be performed is 109°~117° East and the latitude range is 20°~25° North, and if the lon in an obtained nc file is within 109°~117° East and the lat is within 20°~25° North, then the multidimensional variables in the nc file are used as the target weather forecast data.

[0075] Step S130: Sort the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain sorted weather forecast data.

[0076] The forecast time can be the time period targeted by the numerical weather prediction.

[0077] In practice, at least one set of target weather forecast data can be determined, the forecast time corresponding to each set of target weather forecast data can be obtained, and the at least one set of target weather forecast data can be sorted according to the forecast time to obtain sorted weather forecast data.

[0078] For example, suppose the target weather forecast data forms an M×N matrix, where M rows correspond to M nc files (or M sets of target weather forecast data), and N columns correspond to N multidimensional variables in each nc file (or N multidimensional variables in each set of target weather forecast data). The filenames of the M nc files are nwp_202006180830, nwp_202006180800, nwp_202006180845, ... . Extracting and formatting the filenames yields the forecast times 2020 / 6 / 18 08:30, 2020 / 6 / 18 08:00, and 2020 / 6 / 18 respectively. At 08:45, ..., the target weather forecast data of group M are sorted according to the order of the forecast time to obtain the sorted weather forecast data. The sorted weather forecast data corresponds to the nc files nwp_202006180800, nwp_202006180830, nwp_202006180845, ...

[0079] Step S140: The sorted weather forecast data is used as the processing result of the weather forecast data to be processed.

[0080] In practice, the sorted weather forecast data can be used as the processing result of the weather forecast data to be processed, and then used for the prediction of new energy power of wind farms.

[0081] The aforementioned method for processing massive amounts of numerical weather forecast data adapted to new energy prediction acquires the weather forecast data to be processed and the corresponding coordinates. Based on the coordinates, it identifies target weather forecast data from the data to be processed. Then, based on the forecast time corresponding to the target weather forecast data, it sorts the target weather forecast data to obtain sorted weather forecast data. This sorted weather forecast data is used as the processing result for the data to be processed. Furthermore, it can select target weather forecast data from the massive amount of data to be processed based on the coordinates of the power plant. Based on the forecast time, it sorts only the target weather forecast data and uses the sorted weather forecast data for power plant power prediction. This reduces the amount of data to be processed during power prediction, lowers the data processing time, and improves the data processing speed, facilitating real-time online processing of weather forecast data.

[0082] In one embodiment, the coordinate location includes longitude and latitude; step S120 may specifically include: when the longitude location is within a preset longitude range and the latitude location is within a preset latitude range, the weather forecast data to be processed is used as the target weather forecast data.

[0083] In practice, longitude and latitude ranges can be preset. The longitude position corresponding to the weather forecast data to be processed is compared with the longitude range, and the latitude position corresponding to the weather forecast data to be processed is compared with the latitude range. If both the longitude and latitude positions are within the longitude range, the weather forecast data to be processed is determined to be the target weather forecast data. Otherwise, if either the longitude or latitude position is not within the longitude range, the weather forecast data to be processed is determined not to be the target weather forecast data.

[0084] For example, if the longitude range of the wind farm where new energy power prediction needs to be performed is 109°~117°E and the latitude range is 20°~25°N, then if the lon in the obtained nc file is within 109°~117°E and the lat is within 20°~25°N, then the multidimensional variables in the nc file are used as the target weather forecast data. Otherwise, if the lon in the obtained nc file is not within 109°~117°E or the lat is not within 20°~25°N, then the multidimensional variables in the nc file are not the target weather forecast data.

[0085] In this embodiment, when the longitude and latitude of the wind farm are within a preset range, the weather forecast data to be processed is used as the target weather forecast data. The weather forecast data to be processed can be filtered according to the longitude and latitude range of the wind farm, thereby reducing the amount of weather forecast data to be processed and reducing the data processing time.

[0086] In one embodiment, step S120 may further include: determining the upper and lower longitude limits corresponding to the longitude range, and determining the upper and lower latitude limits corresponding to the latitude range; and performing interpolation lookup processing on the weather forecast data to be processed based on the upper and lower longitude limits, the upper and lower latitude limits, to obtain the target weather forecast data.

[0087] In practice, the upper and lower limits of longitude can be obtained based on the upper and lower limits of the longitude range, and the upper and lower limits of latitude can be obtained based on the upper and lower limits of the latitude range. The longitude position between the upper and lower longitude limits and the latitude position between the upper and lower latitude limits are interpolated in the weather forecast data to be processed. The weather forecast data to be processed that has both the longitude position between the upper and lower longitude limits and the latitude position between the upper and lower latitude limits is determined as the target weather forecast data.

[0088] For example, suppose the weather forecast data to be processed forms a Q×N matrix, where Q rows correspond to the Q nc files (or Q groups of weather forecast data) to be processed, and N columns correspond to the N multidimensional variables of each nc file (or the N multidimensional variables of each group of weather forecast data). The N columns include a longitude column and a latitude column. First, interpolation is performed on the longitude column. For rows where the longitude position falls between the upper and lower longitude limits, interpolation is then performed on the latitude column. The data corresponding to the rows where the latitude position falls between the upper and lower latitude limits is used as the target weather forecast data. Alternatively, interpolation can be performed on the latitude column first, and then on the longitude column. The data corresponding to the rows where the longitude position falls between the upper and lower latitude limits is used as the target weather forecast data.

[0089] In this embodiment, the upper and lower limits of longitude corresponding to the longitude range are determined, as well as the upper and lower limits of latitude corresponding to the latitude range. Based on the upper and lower limits of longitude, latitude, and longitude, interpolation lookup processing is performed on the weather forecast data to be processed to obtain the target weather forecast data. The weather forecast data to be processed can be filtered according to the longitude and latitude range where the wind farm is located, thereby reducing the amount of weather forecast data to be processed and reducing data processing time.

[0090] In one embodiment, prior to step S130, the method may further include: obtaining the data file name of the target weather forecast data; and obtaining the forecast time of the target weather forecast data based on the data file name.

[0091] The data file name can be the file name of the nc file corresponding to the target weather forecast data.

[0092] In practice, the filename of the target weather forecast data can be obtained, and the forecast time of the target weather forecast data can be extracted from the filename.

[0093] For example, suppose the target weather forecast data forms an M×N matrix, where M rows correspond to M nc files (or M sets of target weather forecast data), with filenames such as nwp_202006180830, nwp_202006180800, nwp_202006180845, ... . Extracting the filenames yields 202006180830, 202006180800, 202006180845, ... After formatting, the forecast times are 2020 / 6 / 18 08:30, 2020 / 6 / 18 08:00, 2020 / 6 / 18 08:45, ...

[0094] In this embodiment, by obtaining the data file name of the target weather forecast data and then obtaining the forecast time of the target weather forecast data based on the data file name, the forecast time of the target weather forecast data can be obtained quickly, thereby improving the processing speed of weather forecast data.

[0095] In one embodiment, step S130 may specifically include: binding the forecast time with the target weather forecast data to obtain the bound weather forecast data; and sorting the bound weather forecast data according to the forecast time to obtain the sorted weather forecast data.

[0096] In practice, target weather forecast data can be bound to its corresponding forecast time to obtain bound weather forecast data. The bound weather forecast data can then be sorted according to the order of the forecast times to obtain sorted weather forecast data.

[0097] For example, suppose the target weather forecast data forms an M×N matrix, where M rows correspond to M nc files (or M sets of target weather forecast data), with corresponding forecast times of 2020 / 6 / 18 08:30, 2020 / 6 / 18 08:00, 2020 / 6 / 18 08:45, ... , the forecast time 2020 / 6 / 18 08:30 is bound to the first row of target weather forecast data, the forecast time 2020 / 6 / 18 08:00 is bound to the second row of target weather forecast data, and the forecast time 2020 / 6 / 18 08:45 is bound to the third row of target weather forecast data. The 08:30 time is bound to the target weather forecast data in the 3rd row, and so on, to obtain an M×(N+1) matrix of bound weather forecast data. Assuming that the first column of this matrix is ​​the forecast time, the first column can be used as the index column. The bound weather forecast data can be sorted according to the forecast time to obtain the sorted weather forecast data.

[0098] In this embodiment, the forecast time is bound to the target weather forecast data to obtain the bound weather forecast data. The bound weather forecast data is then sorted according to the forecast time to obtain the sorted weather forecast data. The target weather forecast data can be sorted according to the forecast time, which facilitates the processing of the weather forecast data according to the order of the forecast time and improves the processing efficiency of the weather forecast data.

[0099] In one embodiment, the bound weather forecast data contains at least one column, and the at least one column contains a forecast time column; the step of sorting the bound weather forecast data according to the forecast time to obtain sorted weather forecast data may specifically include: using the forecast time column in the bound weather forecast data as an index column; and sorting the bound weather forecast data according to the index column to obtain sorted weather forecast data.

[0100] In practice, the forecast time column in the bound weather forecast data can be used as the index column. The bound weather forecast data can be sorted according to the order of the forecast times in the index column to obtain the sorted weather forecast data.

[0101] For example, suppose the target weather forecast data forms an M×N matrix, where M rows correspond to M nc files (or M sets of target weather forecast data), with corresponding forecast times of 2020 / 6 / 18 08:30, 2020 / 6 / 18 08:00, 2020 / 6 / 18 08:45, ... , the forecast time 2020 / 6 / 18 08:30 is bound to the first row of target weather forecast data, the forecast time 2020 / 6 / 18 08:00 is bound to the second row of target weather forecast data, and the forecast time 2020 / 6 / 18 08:45 is bound to the third row of target weather forecast data. The 08:30 data is linked to the target weather forecast data in the 3rd row, and so on, to obtain an M×(N+1) matrix of linked weather forecast data. Assuming that the first column of this matrix is ​​the forecast time column, it is used as the index column. According to the order of the forecast time, the M rows of linked weather forecast data are sorted to obtain the sorted weather forecast data.

[0102] In this embodiment, the forecast time column in the bound weather forecast data is used as the index column. Based on the index column, the bound weather forecast data is sorted to obtain sorted weather forecast data. The target weather forecast data can be sorted according to the forecast time, which facilitates the processing of weather forecast data according to the order of forecast time and improves the processing efficiency of weather forecast data.

[0103] In one embodiment, the step of sorting the bound weather forecast data according to the forecast time to obtain sorted weather forecast data may further include: determining the target bound data from the at least two bound weather forecast data when the coordinate positions and forecast times corresponding to at least two bound weather forecast data are the same; and sorting the target bound data to obtain sorted weather forecast data.

[0104] In practice, for two or more bound weather forecast data, if their corresponding coordinate positions and forecast times are the same, a target bound data can be determined from the two or more bound weather forecast data, and the other bound weather forecast data can be deleted, so that only the target bound data participates in the sorting process of step S130 to obtain the sorted weather forecast data.

[0105] In practical applications, the rules for determining the target binding data can be, but are not limited to, specified or randomly determined.

[0106] For example, the bound weather forecast data forms an M×(N+1) matrix. If the forecast time, longitude, and latitude of the first and second rows of this matrix are the same, then the second row obtained later is determined as the target bound data, and the first row is deleted. Then, the target bound data can be sorted, that is, the second to M rows of the bound weather forecast data are sorted to obtain a (M-1)×(N+1) matrix of sorted weather forecast data.

[0107] In this embodiment, the target bound data is determined from the at least two bound weather forecast data when the coordinate positions and forecast times are the same. The target bound data is sorted to obtain sorted weather forecast data. When there are multiple weather forecast data at the same forecast time or the same coordinate position, only one data is retained, which improves the effectiveness of the weather forecast data.

[0108] In one embodiment, prior to step S130, the method may further include: obtaining the correspondence between candidate stations and data grid points of weather forecast data to be processed; determining the target grid point corresponding to the target station among the candidate stations based on the correspondence; and using the weather forecast data to be processed corresponding to the target grid point as the target weather forecast data.

[0109] Candidate sites can be any sites that require renewable energy forecasting. Target sites can be any sites that currently need to process numerical weather prediction data.

[0110] Here, data grid points can be the spatial grid corresponding to the weather forecast data to be processed. Target grid points can be the grid points representing the location of the target station.

[0111] In practice, the terminal can pre-acquire at least one data grid point corresponding to each candidate station. Since each data grid point corresponds to the longitude and latitude position in the weather forecast data to be processed, the target grid point corresponding to the target station in the candidate stations can be determined according to the correspondence between the candidate stations and the data grid points. Then, the target weather forecast data can be determined from the weather forecast data to be processed according to the longitude and latitude position of the target grid point.

[0112] In this embodiment, the correspondence between candidate stations and data grid points of weather forecast data to be processed is obtained; based on the correspondence, the target grid point corresponding to the target station in the candidate stations is determined; the weather forecast data to be processed corresponding to the target grid point is used as the target weather forecast data. The stations and the grid points of the weather forecast data can be pre-assigned. When it is necessary to obtain the weather forecast data of a specified station, the search can be performed directly according to the correspondence, which increases the speed of obtaining the target weather forecast data and thus improves the data processing speed.

[0113] In one embodiment, prior to step S110, the method may further include: acquiring weather forecast data samples and corresponding new energy output samples; determining target features from the sample features of the weather forecast data samples based on the Pearson product-moment correlation coefficient between the new energy output samples and the weather forecast data samples; and reducing the original weather forecast data based on the target features to obtain weather forecast data to be processed.

[0114] The weather forecast data sample can consist of multiple numerical weather forecast data sets. The sample characteristics of the weather forecast data sample can be multidimensional variables from the numerical weather forecast data. The target characteristics can be several variables from the multidimensional variables that have a significant impact on new energy output and which need to be retained.

[0115] Among them, the renewable energy output sample can be multiple sets of renewable energy outputs actually collected, and each set of renewable energy outputs corresponds to one numerical weather forecast data.

[0116] The raw weather forecast data can be the original numerical weather forecast data.

[0117] In practice, the terminal can obtain weather forecast data samples through prediction, and can also simulate the output of new energy sources based on the weather forecast data samples to obtain the new energy output samples corresponding to the weather forecast data samples. The Pearson product-moment correlation coefficient between the new energy output samples and the weather forecast data samples is calculated. Based on the Pearson product-moment correlation coefficient, target features are selected from the sample features of the weather forecast data samples. For the obtained raw weather forecast data, the target features in the raw weather forecast data are retained, and other features are deleted to obtain the weather forecast data to be processed.

[0118] In practical applications, multiple .nc files can be generated as weather forecast data samples. A set of renewable energy outputs is simulated based on each .nc file, serving as renewable energy output samples. The Pearson product-moment correlation coefficient between the multidimensional variables in the .nc file and the renewable energy output samples is calculated. The Pearson product-moment correlation coefficient measures the linear correlation between two variables, with a value between -1 and 1. When the Pearson product-moment correlation coefficient exceeds a preset threshold, the corresponding variable in the .nc file is identified as the target feature. For subsequent .nc files, only the target feature can be retained. For example, if the Pearson product-moment correlation coefficients corresponding to lon, lat, and WS in the multidimensional variables of the .nc file exceed the preset threshold, then for subsequent .nc files, variables other than lon, lat, and WS can be deleted, thus obtaining the weather forecast data to be processed.

[0119] In this embodiment, weather forecast data samples and corresponding new energy output samples are obtained; target features are determined from the sample features of weather forecast data samples based on the Pearson product-moment correlation coefficient between the new energy output samples and the weather forecast data samples; and the original weather forecast data is reduced based on the target features to obtain the weather forecast data to be processed, which can reduce the data dimension and improve the speed of processing massive numerical weather forecast data.

[0120] To facilitate a deeper understanding of the embodiments of this application by those skilled in the art, a specific example will be used for illustration below.

[0121] The output of new energy sources is affected by meteorological factors, exhibiting strong randomness and volatility. Large-scale grid connection of new energy sources poses challenges to the safe and stable operation of the power system. Power forecasting is one of the key measures to address the grid connection issues of new energy sources. Among power forecasting methods, fully utilizing numerical weather prediction data is crucial to improving the accuracy of new energy power forecasting. Currently, most numerical weather prediction data is transmitted and stored in .nc file format, resulting in an extremely large data volume: in terms of forecast frequency, several weather forecasts are issued daily (e.g., 4 times, i.e., an average of once every 6 hours); in terms of forecast lead time, each forecast predicts weather data for the next several days (e.g., the next 7 days); in terms of forecast temporal resolution, forecast data is provided every few minutes (usually 15 minutes); and in terms of forecast spatial resolution, the forecast area is measured in m km. n km (usually 1 km) 1km or 3km The data is generated using a 3km grid as a unit, forming the weather forecast data within that grid. Regarding forecast meteorological features, there are typically several (usually dozens) features such as wind speed / direction at different altitudes. Each forecast is updated every 15 minutes (consistent with the forecast time resolution), forming a separate .nc file containing weather forecast data for all meteorological features at all grid points within the forecast area at the corresponding forecast time. For renewable energy power forecasting scenarios, it is necessary to rapidly acquire numerical weather forecast data for renewable energy power plants at specified coordinates. This includes quickly acquiring all historical numerical weather forecast data from a past period (e.g., one year) during model training to rapidly conduct and complete model training; and rapidly acquiring real-time numerical weather forecast data for a future period (e.g., the next 7 days) during the power forecasting phase to complete power forecasting and report to the power dispatching agency.

[0122] Current common numerical weather prediction processing methods involve processing weather forecast .nc files for the target time period one by one. When processing each .nc file, it is necessary to traverse all grid points in the forecast area until the grid point containing the latitude and longitude of the target station is found, and then traverse all meteorological features under that grid point. This results in the acquisition of many meteorological features that may be useless or have almost no impact on new energy power prediction, wasting storage space. Moreover, the entire process is time-consuming and has poor data timeliness. The real-time nature of numerical weather prediction for power prediction has not been well utilized and explored. Therefore, there is currently a lack of a high-speed online processing method for massive numerical weather prediction data that is suitable for new energy prediction.

[0123] This application first uses the random forest algorithm to screen the combination of meteorological characteristic factors that have the greatest impact on the change of new energy power. Based on this, statistical correlation analysis is used to study the correlation between meteorological characteristics, further filtering the meteorological characteristics. Finally, a recursive feature elimination method is used to determine the final set of features selected for prediction. Using the above method to select meteorological characteristics in power prediction can improve the accuracy of new energy power prediction.

[0124] Specifically, during preprocessing, the grid points of the specified latitude and longitude stations in the forecast area of ​​the numerical weather forecast data to be cleaned can be quickly located and recorded in shared memory. This allows for subsequent use of the data by skipping the location search step and directly obtaining the grid point data, avoiding the excessive time consumption caused by iterating through the forecast area to locate each grid point one by one. Since the number of grid points within the forecast area is enormous, for grid points of the same longitude / latitude, each grid point is arranged in an ordered manner with approximately equal spacing (for example, for a grid point at 100° longitude and 20° north latitude, under the same longitude, if the spatial resolution is 3km...). If the distance is 3km, then the latitudes of its two lower grid points are approximately 20.03° and 20.06° respectively. When using interpolation to find the location, the time complexity is O(log log n). log log n), where log n represents the logarithm of n to the base 2, is far less than the current traversal search time complexity of O(n log n). n).

[0125] In addition, during preprocessing, the meteorological features that have the greatest impact on the change of renewable energy power at the target power station can be screened based on the Pearson product-moment correlation coefficient and recorded in shared memory for later use, thus avoiding traversing all features.

[0126] Numerical weather forecast (NWFR) meteorological grid data is used as the source data. The time resolution of the NWFR meteorological grid data is 15 minutes, with a delay of minutes. The NWFR data can be a multidimensional data folded file in .nc format, which cannot be read and parsed using ordinary document readers or editors. In order to perform renewable energy power prediction, the NWFR data needs to be processed into a two-dimensional dataset. Each column of the two-dimensional dataset corresponds to a NWP feature, and each row of data is a NWP weather data for the location of a power plant. The time interval between two adjacent rows is 15 minutes.

[0127] Numerical weather forecasting refers to a method that, based on actual atmospheric conditions and under certain initial and boundary conditions, uses large computers to perform numerical calculations to solve the fluid dynamics and thermodynamic equations describing the evolution of weather, and to predict the atmospheric motion state and weather phenomena over a certain period of time in the future.

[0128] Figure 2 A flowchart illustrating a method for processing massive amounts of numerical weather forecast data adapted to new energy prediction is provided. Based on... Figure 2 Numerical weather forecasting data processing methods adapted to new energy prediction can specifically include:

[0129] Step S201: Create a new target file in the terminal to store the NWP data extraction results.

[0130] Step S202: Read all .nc files in the current file storage path in sequence. For each .nc file, execute the following steps S203-S206 in sequence.

[0131] Step S203: Extract the time from the .nc file name and format the extracted time as a timestamp, for example: 2020 / 8 / 16 08:00:00.

[0132] Step S204: Locate and determine the latitude and longitude of the wind farm station for new energy power prediction in the .nc file.

[0133] Specifically, the latitude and longitude of the target can be extracted from the .nc file using parallel computing. Parallel computing, as opposed to serial computing, is an algorithm that executes multiple instructions at once, aiming to improve computational speed and solve large and complex computational problems by scaling up the problem-solving process. Parallel computing can reduce numerical weather forecasting processing time by several times.

[0134] An interpolation search algorithm can also be used to locate the corresponding weather forecast data for a specific new energy power plant based on its latitude and longitude. The interpolation search method fully utilizes the order relationship between elements and employs a divide-and-conquer strategy, enabling the data search task to be completed in log n time complexity even in the worst case. The algorithm uses four variables—upper / lower longitude limits, upper / lower latitude limits—to approximate the latitude and longitude of the target power plant. Then, it iterates through all grid latitude and longitude values ​​in the weather forecast data and compares them with the previous values. If the current grid latitude and longitude value falls within the range of the previous values, the weather forecast data for that grid is retrieved.

[0135] Step S205: Extract the variables corresponding to the determined latitude and longitude, treat each extracted variable as a column to obtain multidimensional variables, expand the multidimensional variables, and each dimension of the multidimensional variables corresponds to a column.

[0136] Specifically, high-dimensional meteorological indicators can be read, multi-dimensional indicators can be expanded into one-dimensional indicators, and meteorological indicators of each dimension can be extracted as a column of output data.

[0137] Step S206: Write the timestamp and multidimensional variables of the current .nc file into the target file, and save the target file.

[0138] Step S207: Load the target file into memory. Specifically, the user can customize the columns of the target file to be loaded.

[0139] Step S208: For the target file loaded into memory, sort the rows of the target file according to the timestamp (prediction time) as the index column.

[0140] Step S209: Export the sorted target files as .csv files to obtain the NWP data extraction results.

[0141] A CSV file is a comma-separated values ​​file format used to store data in plain text format.

[0142] Specifically, a .csv file can be exported after data processing. The .csv file is sorted according to the forecast time. When there are multiple records corresponding to the same forecast time, only the last record is kept.

[0143] The results of massive data processing for numerical weather prediction are shown in Table 1. Based on Table 1, a CSV file can be created in the terminal first, then an nc file named nwp_202006180800 can be read. The time in the filename can be extracted and formatted to obtain the forecast time (forecast_time) corresponding to the nc file as 2020 / 6 / 18 08:00. The multidimensional variables in this nc file can be read, including but not limited to longitude (lon), latitude (lat), wind speed (WS), wind direction (WD), temperature (TEM), air density (Density), air pressure (PRS), total radiation (SR), direct radiation (SWDDIF), diffuse radiation (SWDDIR), high cloud cover (TCC), low cloud cover (LCC), mesoscale convective complex (MCC), total cloud cover (HCC), temperature, and humidity. The longitude and latitude can then be compared with a preset longitude range. Compared to latitude range, if both longitude and latitude are within a preset range, the nc file stores NWP data for the target power plant's location. The forecast time (forecast_time) and multidimensional variables are stored in a single row of a CSV file (e.g., row 2 in Table 1), with each column corresponding to a variable. Otherwise, if either longitude or latitude is outside the preset range, the nc file does not store NWP data for the target power plant's location, and is discarded. Next, the above processing is performed sequentially on nc files named nwp_202006180815, nwp_202006180830, ... The resulting CSV file contains a forecast time column (e.g., column 1 in Table 1). Using the forecast time column as an index column, all rows in the CSV file are sorted according to the order of the forecast times in the index column. The sorted CSV file is then exported, yielding the extracted NWP data.

[0144] Table 1

[0145]

[0146] The aforementioned method for processing massive numerical weather forecast data adapted to new energy forecasting fills the gap in the current lack of online, high-speed batch processing methods for large-scale numerical weather forecast data. Using this method, processing a single numerical weather forecast .nc source file can be completed in just 1 second, and processing one day's numerical weather forecast data from a single new energy power station can be completed in approximately 100 seconds. This significantly improves the usability of numerical weather forecast data and plays a crucial role in subsequent new energy power forecasting and improving forecast accuracy.

[0147] In one embodiment, such as Figure 3As shown, a method for processing massive numerical weather forecast data adapted to new energy prediction is provided. Taking the application of this method to a terminal as an example, the method includes the following steps:

[0148] Step S401: Obtain the weather forecast data to be processed, and the coordinates of the weather forecast data to be processed;

[0149] Step S402: Determine the target weather forecast data from the weather forecast data to be processed based on the coordinate location;

[0150] Step S403: Obtain the filename of the target weather forecast data;

[0151] Step S404: Obtain the forecast time of the target weather forecast data based on the data file name;

[0152] Step S405: Bind the forecast time with the target weather forecast data to obtain the bound weather forecast data;

[0153] Step S406: Use the forecast time column in the bound weather forecast data as the index column;

[0154] Step S407: Sort the bound weather forecast data according to the index column to obtain sorted weather forecast data;

[0155] Step S408: The sorted weather forecast data is used as the processing result of the weather forecast data to be processed.

[0156] The aforementioned method for processing massive amounts of numerical weather forecast data adapted to new energy prediction involves acquiring the weather forecast data to be processed and its corresponding coordinates. Based on these coordinates, the method identifies the target weather forecast data from the remaining data, obtains its filename, and determines its forecast time. The forecast time is then bound to the target data, and the forecast time column is used as an index column to sort the data. This sorted data is then used as the processing result for the remaining data. Furthermore, the method can select the target weather forecast data from the massive dataset based on the power plant's coordinates, sort it only based on the forecast time, and use the sorted data for power plant power prediction. This reduces the amount of data to be processed, lowers processing time, and increases processing speed, facilitating real-time online processing of weather forecast data.

[0157] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0158] Based on the same inventive concept, this application also provides a numerical weather forecasting massive data processing device for implementing the numerical weather forecasting massive data processing method for adapting to new energy forecasting described above. The solution provided by this device is similar to the implementation scheme described in the above method. Therefore, the specific limitations in one or more embodiments of the numerical weather forecasting massive data processing device for adapting to new energy forecasting provided below can be found in the limitations of the numerical weather forecasting massive data processing method for adapting to new energy forecasting described above, and will not be repeated here.

[0159] In one embodiment, such as Figure 4 As shown, a massive data processing device for numerical weather forecasting adapted to new energy prediction is provided, comprising: an acquisition module 510, a selection module 520, a sorting module 530, and a processing module 540, wherein:

[0160] The acquisition module 510 is used to acquire the weather forecast data to be processed, and the coordinate position corresponding to the weather forecast data to be processed;

[0161] The selection module 520 is used to determine the target weather forecast data from the weather forecast data to be processed based on the coordinate position;

[0162] The sorting module 530 is used to sort the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain sorted weather forecast data.

[0163] The processing module 540 is used to take the sorted weather forecast data as the processing result of the weather forecast data to be processed.

[0164] In one embodiment, the selection module 520 is further configured to, when the longitude position is within a preset longitude range and the latitude position is within a preset latitude range, use the weather forecast data to be processed as the target weather forecast data.

[0165] In one embodiment, the selection module 520 is further configured to perform a binary search on the longitude position of the weather forecast data to be processed, and if the longitude position is within the longitude range, use the weather forecast data to be processed as the searched weather forecast data; and to perform a binary search on the latitude position of the searched weather forecast data, and if the latitude position is within the latitude range, use the searched weather forecast data as the target weather forecast data.

[0166] In one embodiment, the aforementioned numerical weather forecasting massive data processing device adapted to new energy prediction further includes:

[0167] The filename acquisition module is used to acquire the data filename of the target weather forecast data;

[0168] The forecast time determination module is used to obtain the forecast time of the target weather forecast data based on the data file name.

[0169] In one embodiment, the sorting module 530 is further configured to bind the forecast time with the target weather forecast data to obtain bound weather forecast data; and to sort the bound weather forecast data according to the forecast time to obtain sorted weather forecast data.

[0170] In one embodiment, the sorting module 530 is further configured to use the forecast time column in the bound weather forecast data as an index column; and to sort the bound weather forecast data according to the index column to obtain the sorted weather forecast data.

[0171] In one embodiment, the sorting module 530 is further configured to determine target bound data from the at least two bound weather forecast data when the coordinate positions and forecast times corresponding to the at least two bound weather forecast data are the same; and to sort the target bound data to obtain the sorted weather forecast data.

[0172] The modules in the aforementioned numerical weather forecasting massive data processing device adapted for new energy prediction can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0173] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a massive data processing method for numerical weather forecasting adapted to new energy prediction. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0174] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0176] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0177] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0178] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0179] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing massive data of numerical weather prediction (NWP) adapted to new energy prediction, characterized in that, The method includes: Obtain the weather forecast data to be processed, and the coordinates corresponding to the weather forecast data to be processed; Based on the coordinates, the target weather forecast data is determined from the weather forecast data to be processed; Based on the forecast time corresponding to the target weather forecast data, the target weather forecast data is sorted to obtain sorted weather forecast data. The sorted weather forecast data is used as the processing result of the weather forecast data to be processed.

2. The method of claim 1, wherein, The coordinate positions include longitude and latitude; determining the target weather forecast data from the weather forecast data to be processed based on the coordinate positions includes: If the longitude position is within a preset longitude range and the latitude position is within a preset latitude range, the weather forecast data to be processed will be used as the target weather forecast data.

3. The method of claim 2, wherein, The step of determining the target weather forecast data from the weather forecast data to be processed based on the coordinate position further includes: Determine the upper and lower limits of longitude corresponding to the longitude range, and determine the upper and lower limits of latitude corresponding to the latitude range; Based on the upper longitude limit, the lower longitude limit, the upper latitude limit, and the lower latitude limit, interpolation lookup processing is performed on the weather forecast data to be processed to obtain the target weather forecast data.

4. The method of claim 1, wherein, Before sorting the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain the sorted weather forecast data, the method further includes: Obtain the filename of the target weather forecast data; The forecast time for the target weather forecast data is obtained based on the data file name.

5. The method of claim 4, wherein, The step of sorting the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain sorted weather forecast data includes: The forecast time is bound to the target weather forecast data to obtain the bound weather forecast data; Based on the forecast time, the bound weather forecast data is sorted to obtain the sorted weather forecast data.

6. The method of claim 5, wherein, The bound weather forecast data includes at least one column, which includes a forecast time column; the sorting process of the bound weather forecast data according to the forecast time to obtain the sorted weather forecast data includes: Use the forecast time column in the bound weather forecast data as the index column; The bound weather forecast data is sorted according to the index column to obtain the sorted weather forecast data.

7. The method of claim 5, wherein, The step of sorting the bound weather forecast data according to the forecast time to obtain the sorted weather forecast data further includes: If the coordinates and forecast times of at least two bound weather forecast data are the same, the target bound data is determined from the at least two bound weather forecast data. The target-bound data is sorted to obtain the sorted weather forecast data.

8. The method of claim 1, wherein, Before sorting the target weather forecast data according to the forecast time corresponding to the target weather forecast data to obtain the sorted weather forecast data, the method further includes: Obtain the correspondence between candidate stations and data grid points of the weather forecast data to be processed; Based on the correspondence, the target grid point corresponding to the target station in the candidate stations is determined; The weather forecast data to be processed corresponding to the target grid point is used as the target weather forecast data.

9. The method of claim 1, wherein, Before acquiring the weather forecast data to be processed and the coordinates corresponding to the weather forecast data, the process also includes: Obtain weather forecast data samples and corresponding new energy output samples; Based on the Pearson product-moment correlation coefficient between the new energy output sample and the weather forecast data sample, the target features are determined from the sample features of the weather forecast data sample; Based on the target characteristics, the original weather forecast data is reduced to obtain the weather forecast data to be processed. 10.A device for processing massive data of numerical weather prediction (NWP) adapted to new energy prediction, characterized in that, The device includes: The acquisition module is used to acquire the weather forecast data to be processed, as well as the coordinates of the weather forecast data to be processed. The selection module is used to determine the target weather forecast data from the weather forecast data to be processed based on the coordinate position. The sorting module is used to sort the target weather forecast data according to the forecast time corresponding to the target weather forecast data, so as to obtain sorted weather forecast data. The processing module is used to take the sorted weather forecast data as the processing result of the weather forecast data to be processed.

Images