Chain equation multiple imputation method and device for comprehensive energy meteorological missing data

Abstract

The application discloses a kind of comprehensive energy meteorological missing data chain equation multiple interpolation method and device, it is related to data processing technical field, the method includes: obtaining the original multivariate meteorological data matrix of comprehensive energy system, data matrix includes at least two meteorological variables in different sampling time observation value and missing value;Meteorological variable is standardized, and the missing value of each meteorological variable is initially interpolated, to generate initial complete data matrix;M independent interpolation processes are executed, to generate M complete data matrix, M is the integer greater than 1, after inverse standardization processing is carried out to M complete data matrix generated by M interpolation processes, the same sampling time, same meteorological variable in the M complete data matrix after inverse standardization processing is fused and calculated, to obtain the final meteorological data completion result.The conditional dependence between multivariate meteorological data is fully utilized, and the accuracy and stability of missing data interpolation are improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a chain equation multiple interpolation method, apparatus, computer storage medium, and electronic equipment for comprehensive energy and meteorological missing data. Background Technology

[0002] Short-term load forecasting for integrated energy systems typically relies on various meteorological data as key input features. These meteorological data mainly include variables such as temperature, dew point temperature, relative humidity, and global horizontal irradiance. Accurate load forecasting is crucial for the optimal scheduling, economic operation, and stable control of integrated energy systems, and the completeness and accuracy of meteorological data directly determine the training quality and forecasting accuracy of load forecasting models.

[0003] However, in actual operation, the meteorological data acquisition process of integrated energy systems is susceptible to data gaps due to various factors. Specifically, sensor malfunctions, communication anomalies, data transmission delays, and planned or unplanned equipment maintenance can all lead to random point-like or continuous time-limited missing data in meteorological variables. To address these missing data issues, existing technologies typically involve directly deleting sample records containing missing values ​​or using simple univariate interpolation methods to fill in the missing data. Common univariate interpolation methods include mean interpolation, linear interpolation, and seasonal linear interpolation. Their basic idea is to estimate and reconstruct the missing terms of a single variable using its nearest observations over time.

[0004] However, the aforementioned existing technologies have the following technical shortcomings in practical applications: First, univariate imputation methods struggle to utilize the correlations between multiple variables. Methods such as mean interpolation, linear interpolation, and seasonal linear interpolation primarily rely on the temporal proximity of individual variables for reconstruction, failing to reflect the objectively existing conditional dependencies and coupling characteristics between multiple meteorological variables such as temperature, irradiance, and humidity. In integrated energy scenarios, meteorological variables often exhibit significant multivariate coupling characteristics, making univariate imputation methods prone to distorting the imputation results and disrupting the intrinsic correlation structure between variables. Second, reconstruction accuracy is insufficient in scenarios with continuous missing data. For problems involving continuous missing time periods due to prolonged equipment offlineness or maintenance, conventional linear interpolation methods lack sufficient intermediate observation data within the missing interval, making it difficult to accurately recover the original trends and fluctuations of meteorological variables. Furthermore, single-shot machine learning imputation methods typically output only a single deterministic estimate, failing to reflect the uncertainties and fluctuations present during the imputation process. Third, existing methods generally lack the ability to quantify imputation uncertainty. Most missing data completion methods only provide a single deterministic imputation result, failing to provide further statistical information such as within-group variance, between-group variance, and overall uncertainty of the imputation result. As a result, downstream load forecasting models cannot assess the credibility of the input data, which is not conducive to improving the reliability of the forecast results.

[0005] Therefore, there is an urgent need to provide a comprehensive energy and meteorological missing data imputation method that can fully utilize multivariate coupling relationships, achieve high reconstruction accuracy in continuous missing scenarios, and quantify imputation uncertainty. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a chain equation multiple interpolation method, apparatus, computer storage medium, and electronic device for comprehensive energy and meteorological missing data. This method can fully utilize the conditional dependencies between multivariate meteorological data to improve the accuracy and stability of missing data interpolation in both random and continuous missing scenarios.

[0007] In a first aspect, embodiments of the present invention provide a chain-equation multiple imputation method for missing meteorological data in integrated energy systems. The method includes: acquiring an original multivariate meteorological data matrix of an integrated energy system, the data matrix containing observed values ​​and missing values ​​of at least two meteorological variables at different sampling times; standardizing the meteorological variables and initially imputing the missing values ​​of each meteorological variable to generate an initial complete data matrix; executing M independent imputation processes to generate M complete data matrices, where M is an integer greater than 1. The independent imputation process includes: performing multiple rounds of chain-equation iterative updates based on the initial complete data matrix, wherein each round of chain-equation iterative updates includes: sequentially determining one meteorological variable with missing values ​​in the original multivariate meteorological data matrix as the current target variable, and modifying the values ​​of other meteorological variables in the current basic complete data matrix (excluding the current target variable)... Using the input features, a regression model is trained based on all observed values ​​of the current target variable. The trained regression model is then used to predict and update the missing values ​​of the current target variable, replacing the values ​​of the current target variable in the current complete data matrix. After all meteorological variables with missing values ​​have been updated once, the updated complete data matrix is ​​used as the basis for the next round of chain-iterative updates. In response to the current round of chain-iterative updates meeting the preset convergence condition, the chain-iterative updates in the interpolation process are terminated, and the complete data matrix obtained in this round is determined as a complete data matrix output by this round of interpolation. After inverse standardization is performed on the M complete data matrices generated by the M rounds of interpolation, the values ​​of the same meteorological variable at the same sampling time in the M complete data matrices after inverse standardization are fused and calculated to obtain the final meteorological data completion result.

[0008] In one possible implementation, initial imputation is performed for the missing values ​​of each meteorological variable, including:

[0009] For any meteorological variable, calculate the mean of all observed values ​​of the meteorological variable, and use the mean as the initial imputation value for the missing values ​​of the meteorological variable.

[0010] In one possible implementation, the preset convergence conditions include:

[0011] Calculate the relative change between the complete data matrix obtained after this round of chain iteration update and the complete data matrix obtained after the previous round of chain iteration update;

[0012] If the relative change is less than the preset convergence threshold, or the number of rounds of chain-iterative updates reaches the preset maximum number of iterations, then the preset convergence condition is satisfied.

[0013] In one possible implementation, the preset convergence threshold is 10. -3 The maximum number of iterations is preset to 50.

[0014] In one possible implementation, the regression model is any one of the following: random forest regression model, gradient boosting regression tree model, or support vector regression model.

[0015] In one possible implementation, the meteorological variables are standardized, including:

[0016] For any meteorological variable, calculate the mean and standard deviation of all observed values ​​of the meteorological variable, and use the mean and standard deviation to perform Z-score standardization on all values ​​of the meteorological variable.

[0017] In one possible implementation, the values ​​of the same meteorological variable at the same sampling time in the M complete data matrices after inverse normalization are fused and calculated to obtain the final meteorological data completion result, including:

[0018] Calculate the arithmetic mean of M values ​​of the same meteorological variable at the same sampling time, and use the arithmetic mean as the final completion result for that sampling time and that meteorological variable.

[0019] Secondly, embodiments of the present invention provide a chain equation multiple interpolation device for comprehensive energy and meteorological missing data. The device includes: an acquisition module, a generation module, an execution module, and a result module, wherein:

[0020] The acquisition module is used to acquire the raw multivariate meteorological data matrix of the integrated energy system. The data matrix contains the observed values ​​and missing values ​​of at least two meteorological variables at different sampling times.

[0021] The generation module is used to standardize meteorological variables and perform initial imputation on missing values ​​of each meteorological variable to generate an initial complete data matrix.

[0022] The execution module performs M independent interpolation processes to generate M complete data matrices, where M is an integer greater than 1. The independent interpolation processes include:

[0023] Based on the initial complete data matrix, multiple rounds of chain-like iterative updates are performed. Each round of chain-like iterative updates includes: sequentially identifying a meteorological variable with missing values ​​in the original multivariate meteorological data matrix as the current target variable, and using the values ​​of other meteorological variables in the current basic complete data matrix (excluding the current target variable) as input features; training a regression model based on all observed values ​​of the current target variable; using the trained regression model to predict and update the missing values ​​of the current target variable, replacing the values ​​of the current target variable in the current basic complete data matrix; after all meteorological variables with missing values ​​have been updated once, the updated complete data matrix is ​​used as the basis for the next round of chain-like iterative updates; in response to the chain-like iterative updates meeting the preset convergence conditions, the chain-like iterative updates in the interpolation process are terminated, and the complete data matrix obtained in this round of updates is determined as a complete data matrix output by this round of interpolation.

[0024] The module is used to perform inverse normalization on the M complete data matrices generated by the M rounds of interpolation, and then perform fusion calculation on the values ​​of the same sampling time and the same meteorological variable in the M complete data matrices after inverse normalization to obtain the final meteorological data completion result.

[0025] Thirdly, embodiments of the present invention provide a computer storage medium storing multiple instructions adapted for loading by a processor and executing the steps of the above-described method.

[0026] Fourthly, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being adapted to be loaded by the processor and to execute the steps of the method described above.

[0027] The beneficial effects of the technical solutions provided by some embodiments of the present invention include at least the following: Based on obtaining an original data matrix containing at least two meteorological variables and performing standardization and initial imputation, multiple independent imputation processes are executed. Each imputation process employs a chain-like iterative update method, sequentially using each meteorological variable with missing values ​​as the target variable and the current values ​​of the remaining variables as input features to train a regression model and update the missing values, until a preset convergence condition is met, at which point a complete data matrix is ​​output. Finally, the multiple complete data matrices obtained from the multiple imputations are subjected to inverse standardization and fusion calculation. This method, through a chain-like conditional modeling mechanism, enables each meteorological variable to fully exchange and utilize the conditional dependency information between them during the iteration process, thereby effectively restoring the multivariate coupling relationship at the location of missing values ​​and avoiding the distortion of the imputation results caused by neglecting the correlation between variables in traditional univariate imputation methods. Meanwhile, by performing multiple independent interpolations and fusing the resulting multiple complete data matrices, on the one hand, the statistical differences between the multiple interpolation results provide a quantifiable characterization basis for the interpolation uncertainty; on the other hand, the fusion processing suppresses the fluctuations caused by random factors in a single interpolation process, thereby improving the stability and reliability of the final completion result under complex conditions such as continuous missing data. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 An exemplary system architecture diagram of the chain equation multiple interpolation method for comprehensive energy and meteorological missing data provided in the embodiments of the present invention;

[0030] Figure 2 A flowchart illustrating the chain equation multiple interpolation method for comprehensive energy meteorological missing data provided in this embodiment of the invention;

[0031] Figure 3 A logic block diagram of the chain equation multiple interpolation method for comprehensive energy meteorological missing data provided in the embodiments of the present invention;

[0032] Figure 4 A comparison chart of MAPE (%) for different imputation methods under different missing rates provided in the embodiments of the present invention;

[0033] Figure 5 This is a comparison chart of the prediction effects of different imputation methods at a 10% missing rate, provided by an embodiment of the present invention.

[0034] Figure 6 A structural block diagram of a chain equation multiple interpolation device for comprehensive energy and meteorological missing data provided in an embodiment of the present invention;

[0035] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0036] To make the features and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0038] In the description of this invention, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances. Furthermore, in the description of this invention, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0039] As mentioned earlier, short-term load forecasting for integrated energy systems typically relies on multivariate meteorological data such as temperature, dew point temperature, relative humidity, and global horizontal irradiance as key inputs. However, in actual data acquisition, meteorological data often exhibits random point-like or continuous time-limited gaps due to factors such as sensor malfunctions, communication anomalies, data transmission delays, and equipment maintenance. Existing technologies, such as directly deleting missing samples or using univariate completion methods like mean interpolation or linear interpolation, not only struggle to utilize the objectively existing conditional dependencies and coupling characteristics between multiple meteorological variables, but also suffer from insufficient reconstruction accuracy in scenarios with continuous gaps due to a lack of intermediate observational data. Furthermore, they generally lack the ability to quantify the uncertainty of interpolation results, thus limiting the training quality and forecasting reliability of downstream load forecasting models.

[0040] In view of this, the present invention provides a chain-equation multiple imputation method and apparatus for comprehensive energy and meteorological missing data. It aims to fully explore the correlations between multivariate meteorological data through chain-conditional modeling, and improve the accuracy and stability of missing data imputation in both random and continuous missing data scenarios by combining multiple independent imputation with a result fusion mechanism.

[0041] Please see Figure 1 , Figure 1 This is an exemplary system architecture diagram of the chain equation multiple interpolation method for comprehensive energy and meteorological missing data provided in an embodiment of the present invention.

[0042] like Figure 1 As shown, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links, etc.

[0043] Terminal 101 can interact with server 103 via network 102 to receive messages from or send messages to server 103. Alternatively, terminal 101 can interact with server 103 via network 102 to receive messages or data sent to server 103 by other users. Terminal 101 can be hardware or software. When terminal 101 is hardware, it can be various electronic devices, including but not limited to smartwatches, smartphones, tablets, laptops, and desktop computers. When terminal 101 is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0044] In this embodiment of the invention, terminal 101 can acquire the original multivariate meteorological data matrix of the integrated energy system. The data matrix contains the observed values ​​and missing values ​​of at least two meteorological variables at different sampling times. The meteorological variables are standardized, and the missing values ​​of each meteorological variable are initially imputed to generate an initial complete data matrix. M independent imputation processes are executed to generate M complete data matrices, where M is an integer greater than 1. After inverse standardization is performed on the M complete data matrices generated by the M rounds of imputation, the values ​​of the same meteorological variables at the same sampling time in the M complete data matrices after inverse standardization are fused and calculated to obtain the final meteorological data completion result.

[0045] Server 103 can be a business server providing various services. It should be noted that server 103 can be either hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0046] Alternatively, the system architecture may not include server 103. In other words, server 103 may be an optional device in the embodiments of this specification. That is, the method provided in the embodiments of this specification can be applied to a system structure that only includes terminal 101. The embodiments of this invention do not limit this.

[0047] It should be understood that Figure 1 The number of terminals, networks, and servers shown is only illustrative; the number can be any number of terminals, networks, and servers depending on the implementation requirements.

[0048] Please see Figure 2 , Figure 2 This is a flowchart illustrating the chain equation multiple interpolation method for comprehensive energy and meteorological missing data provided in an embodiment of the present invention. Figure 2 As shown, the chain equation multiple interpolation method for integrating missing energy and meteorological data can include at least the following:

[0049] S201. Obtain the original multivariate meteorological data matrix of the integrated energy system. The data matrix contains the observed values ​​and missing values ​​of at least two meteorological variables at different sampling times.

[0050] Specifically, to provide complete and accurate input features for the subsequent short-term load forecasting model of the integrated energy system, it is first necessary to obtain a raw dataset containing both complete and missing information. Raw records of multiple meteorological variables over a continuous time period are extracted from the integrated energy system's data acquisition and monitoring control system or historical database. These meteorological variables must include at least two or more of the following: temperature, dew point temperature, relative humidity, and global horizontal irradiance. This is because the load characteristics of the integrated energy system are closely related to these multiple meteorological factors, and a single meteorological variable cannot fully reflect the load variation patterns. Specifically, the acquired raw data is organized into an n-row × p-column raw multivariate meteorological data matrix. Where n is the total number of samples at the sampling time, and p is the total number of meteorological variables. Let X be the value of the i-th sample in the j-th variable. This matrix includes both validly collected observations and missing values ​​due to sensor malfunctions, communication interruptions, etc. These missing values ​​may exist in the data records as null values, NaN markers, or specific outlier values. To facilitate the computer program's identification and processing of the data status, this embodiment further constructs a missing value mask matrix with the same dimensions as the original multivariate meteorological data matrix X. , among which, when hour, For the observed value, when hour, This represents missing values. The missing values ​​mask matrix clearly distinguishes the observation status of each data point and provides an index for locating missing values ​​and extracting observation samples in subsequent steps. For example, for the j-th meteorological variable, an index set of its observation samples can be constructed based on the values ​​in the j-th column of the missing values ​​mask matrix M. and the set of indexes of missing samples ,in, This provides a clear data partitioning foundation for subsequent training of regression models based on observational data. By obtaining the original multivariate meteorological data matrix and constructing a missing value mask matrix as described above, all available observational information is fully preserved, while the spatiotemporal distribution of missing values ​​is clearly identified. This provides a clear and accurate data structure foundation for subsequent reconstruction of missing values ​​by fully utilizing the correlation between multivariates, avoiding the information loss problem caused by directly deleting missing samples.

[0051] S202. Standardize the meteorological variables and perform initial imputation on the missing values ​​of each meteorological variable to generate an initial complete data matrix.

[0052] Specifically, due to significant differences in dimensions and numerical scales among different meteorological variables—for example, the range of temperature values ​​differs completely from the percentage range of relative humidity—directly modeling the raw values ​​may result in larger-scale variables dominating the regression model, thus weakening the effective information of other variables and affecting the accurate capture of multivariate correlations. Therefore, each meteorological variable in the original multivariate meteorological data matrix X is first standardized to eliminate the adverse effects of dimensional differences on subsequent model training. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 A logic block diagram of the chain equation multiple interpolation method for comprehensive energy and meteorological missing data provided in this embodiment of the invention is shown below. Figure 3 As shown, after completing the steps of inputting raw meteorological data and marking missing data, standardization processing is performed. Specifically, for any j-th meteorological variable, its mean is first calculated using all observed values ​​of that variable. and standard deviation Then, for each value of the variable, Z-score normalization is performed. The transformation formula is as follows:

[0053]

[0054] For example, the mean of observed temperature variables =25℃, standard deviation =5℃; Average relative humidity observed value =60%, standard deviation =10%. After this transformation, the distribution of observed data for each meteorological variable is adjusted to a standard normal distribution with a mean of 0 and a standard deviation of 1, thus ensuring that all variables have equal weight in subsequent regression modeling. In an alternative implementation, the standardization process can also be replaced by normalization, that is, using the minimum and maximum values ​​of the observed values ​​of the variable to linearly map the data to the [0,1] interval, which can also achieve the purpose of eliminating dimensional differences. However, this embodiment prefers to use Z-score standardization because it is more robust to outliers and more suitable for situations where there are occasional extreme values ​​in meteorological data.

[0055] For further information, please refer to [link / reference]. Figure 3 After standardization, the positions of the original missing values ​​in the obtained standardized data matrix Z are still empty. To enable the subsequent chain-reaction iterative update process, these missing positions need to be assigned an initial value, i.e., initial imputation. For any meteorological variable, the mean of all observed values ​​of that variable is calculated, and this mean is used as the initial imputation value for the missing values. Since the aforementioned standardization process has adjusted the mean of the observed values ​​of each variable to 0, the initial imputation operation for missing values ​​in the standardized space can directly use the corresponding values. The value is assigned to 0. Through the above initial imputation, the originally missing normalized data matrix Z is filled into an initial complete data matrix with no missing values. This matrix will serve as the starting point for subsequent chained iterative updates. For example, for variables with missing values, a mean initialization strategy is used to fill in the missing terms. The initial imputation result for the j-th variable at round t=0 is... Within the standardized space After initializing all variables, an initial complete matrix is ​​obtained. It should be noted that the selection of initial imputation values ​​does not require high precision. Their core function is to provide a reasonable starting point for the iterative algorithm. Subsequent chain-recursive processes will gradually correct these initial imputation values ​​through multivariate regression modeling, bringing them to a state more consistent with the actual data distribution. By standardizing the meteorological variables, the interference of different units on multivariate regression modeling is eliminated, ensuring the fairness and accuracy of subsequent regression model training. At the same time, by performing mean-based initial imputation on missing values, the initial complete data matrix required to start the iterative process is quickly generated.

[0056] S203. Perform M independent interpolation processes to generate M complete data matrices, where M is an integer greater than 1.

[0057] Specifically, to overcome the shortcomings of poor stability and inability to quantify interpolation uncertainty in single-round interpolation, this step adopts a multiple interpolation strategy. This involves independently repeating the complete interpolation process M times, thereby generating M independent but reasonably reflective complete data matrices that reflect the true distribution characteristics of the data. In one possible implementation, the value of M can be configured according to actual accuracy requirements and computational resources. Typically, M is chosen as an integer greater than 1, such as 5, 10, or 20. Setting M to an integer greater than 1 aims to introduce reasonable randomness through multiple independent interpolations, ensuring that the differences between interpolation results from different rounds reflect the inherent uncertainty in missing value prediction.

[0058] Each independent interpolation process includes the following sub-steps: based on the initial complete data matrix generated in the previous stage... The process involves multiple rounds of chain-like iterative updates. In each round of these updates, the following operations are performed: The meteorological variable with missing values ​​in the original multivariate meteorological data matrix is ​​sequentially identified as the current target variable. The order in which the target variable is identified can be based on the variable number or on the missing value ratio from low to high. This embodiment does not impose a specific limitation on this, as long as all variables with missing values ​​are updated once in each iteration. Based on the above preparation, a regression model is trained using all observed values ​​of the current target variable j (i.e., the samples corresponding to the index set Oⱼ). The task of this regression model is to learn the mapping relationship between the input feature Z₋ⱼ and the target variable zⱼ. For example, based on the observation location... Using the data above, construct regression relationships ,in For the j-th variable observation, These are the values ​​of the remaining variables at the observation locations in the previous round. For the random forest regression model, This is the residual term; when training the model, the mean square error of the observed samples is minimized, i.e. , The parameters for this round of the random forest model are specified (preferably, the number of decision trees in the random forest model is 100, and the maximum depth is 10). Optionally, the regression model can be any one of the following: random forest regression model, gradient boosting regression tree model, or support vector regression model. The random forest regression model, by constructing multiple decision trees and ensembling their predictions, can effectively capture the nonlinear coupling relationships between multiple variables, while also exhibiting strong robustness to outliers and noise. Therefore, this embodiment preferably uses the random forest regression model. When training the random forest regression model, the number of decision trees can be set to 100, and the maximum depth to 10, to achieve a good balance between model expressiveness and computational cost. In alternative implementations, if the gradient boosting regression tree model is used, an appropriate learning rate and number of iterations need to be set; if the support vector regression model is used, an appropriate kernel function and penalty parameter need to be selected. All of the above models can effectively fit the nonlinear dependencies of multiple variables.

[0059] Furthermore, after the model training is completed, the trained regression model is used to predict the missing values ​​of the current target variable j (i.e., the samples corresponding to the index set Rⱼ), and obtain... The input features corresponding to the missing samples are input into the regression model, and the predicted values ​​are used to update the corresponding missing positions in the current basic complete data matrix, replacing the original initial values ​​or the values ​​from the previous iteration. This completes the update of the current target variable j, specifically updating all values ​​of the j-th variable. After all meteorological variables with missing values ​​in this round have been updated in the above manner, a new complete matrix will be formed. The updated complete data matrix serves as the basis for the next round of chain-reaction iteration. This process is the core mechanism of chain equation multiple interpolation: each variable is successively used as the target variable and jointly predicted by other variables. The updated values ​​of the variables are immediately used in the modeling process of subsequent variables, thus enabling the relevant information between multiple variables to be fully transferred and utilized during the iteration process.

[0060] Further, please refer to Figure 3 To determine whether the chain-iterative update process has stabilized, a convergence evaluation of the update results is needed. Preset convergence criteria include: calculating the complete data matrix obtained after this round of chain-iterative updates. The complete data matrix obtained after the previous round of chain-like iterative updates The relative change between them; if the relative change is less than a preset convergence threshold, or the number of iterations in the chain-like iteration update reaches a preset maximum number of iterations, then the preset convergence condition is satisfied. In this embodiment, the preset convergence threshold is 10. -3The maximum number of iterations is preset to 50. For example, first calculate the relative change of the complete matrix between two adjacent iterations. ,in It is the Frobenius norm. =10 -8 (To prevent the denominator from being zero); preset convergence threshold =10 -3 Maximum number of iterations =50; if or If the current round of interpolation fails, a complete dataset is obtained; otherwise, the process returns to the step of performing multiple rounds of chained iterative updates and continues iterating.

[0061] By performing M independent interpolation processes, the shortcomings of single interpolation, such as high randomness and inability to reflect the range of uncertainty, are overcome. Through chain-conditional modeling and iterative updates, the coupling information between multiple meteorological variables is fully explored and utilized, making the estimation of missing values ​​closer to the inherent laws of real data. By setting clear convergence criteria and maximum number of iterations, it is ensured that the iterative process can converge stably to a reasonable result, while avoiding the waste of computational resources caused by infinite loops.

[0062] S204. After performing inverse normalization on the M complete data matrices generated by the M-round interpolation process, the values ​​of the same sampling time and the same meteorological variable in the M complete data matrices after inverse normalization are fused and calculated to obtain the final meteorological data completion result.

[0063] Specifically, each complete data matrix first needs to undergo inverse normalization to restore its original dimensions. Inverse normalization is the reverse process of normalization, and the inverse normalization formula is: ( The mean of the m datasets is taken to obtain the final completed data matrix. .

[0064] Furthermore, to extract the most representative and stable final completion result from the M independent interpolation results, it is necessary to perform fusion calculations on the data at corresponding positions in the M complete data matrices. Specifically, for any specified sampling time i and any specified meteorological variable j, the arithmetic mean of the M values ​​corresponding to the same sampling time and the same meteorological variable in the M complete data matrices is calculated, and the arithmetic mean is used as the final completion result for sampling time i and meteorological variable j. In alternative embodiments, the fusion calculation can also use other statistical measures such as weighted average and median, but the arithmetic mean is the preferred option in this embodiment due to its simple calculation and good statistical smoothing effect. Through the above fusion calculation, a complete meteorological data matrix is ​​finally output, in which all the original missing positions have been reasonably filled, which can be used as a reliable input for the short-term load forecasting model of the integrated energy system. By performing inverse standardization and fusion calculation on the multiple interpolation results, the data is restored to its original physical dimensions, enabling it to be directly applied to downstream prediction tasks. On the other hand, the average fusion effectively suppresses the random fluctuations and overfitting risks that may be introduced by single interpolation, significantly improving the stability, accuracy, and reliability of the final interpolation results. This solves the problem of insufficient reliability of single interpolation results and the inability to assess interpolation uncertainty in existing technologies.

[0065] This invention provides a chain-equation multiple imputation method for comprehensive energy meteorological missing data. Based on obtaining an original data matrix containing at least two meteorological variables and performing standardization and initial imputation, multiple independent imputation processes are executed. Each imputation process employs a chain-iterative update approach, sequentially using each meteorological variable with missing values ​​as the target variable and the current values ​​of the remaining variables as input features to train a regression model and update the missing values, until a preset convergence condition is met, outputting a complete data matrix. Finally, the multiple complete data matrices obtained from the multiple imputations are subjected to inverse standardization and fused calculation. This method, through a chain-conditional modeling mechanism, allows each meteorological variable to fully exchange and utilize the conditional dependency information between them during the iteration process, thereby effectively restoring the multivariate coupling relationship at the location of missing values. This avoids the distortion problem of imputation results caused by traditional univariate imputation methods that ignore the correlation between variables. Meanwhile, by performing multiple independent interpolations and fusing the resulting multiple complete data matrices, on the one hand, the statistical differences between the multiple interpolation results provide a quantifiable characterization basis for the interpolation uncertainty; on the other hand, the fusion processing suppresses the fluctuations caused by random factors in a single interpolation process, thereby improving the stability and reliability of the final completion result under complex conditions such as continuous missing data.

[0066] Please see Figure 4 , Figure 4This is a comparison chart of MAPE (%) for different imputation methods under different missing rates provided in the embodiments of the present invention, as shown in the figure. Figure 4 As shown in the figure, this line graph visually compares the imputation error levels of the proposed method, the mean imputation method, and the linear imputation method under three different missing value conditions: 5%, 10%, and 20%. In the simulated scenario where random point-like missing values ​​and continuous time-span missing values ​​coexist, as the missing value gradually increases from 5% to 20%, the MAPE value of the proposed method is consistently lower than that of the mean imputation and linear imputation methods, and the error growth is more gradual. This result is attributed to the design of the chain-iterative update mechanism in the proposed method: by sequentially using each meteorological variable containing missing values ​​as the target variable and the current values ​​of the remaining variables as input features to train the regression model, the coupling relationship between multiple variables can be fully explored and utilized during the iteration process, thus maintaining the ability to accurately reconstruct missing values ​​even at high missing value rates. Therefore, the proposed method has good adaptability and robustness to different missing value scenarios, effectively overcoming the defect of existing univariate imputation methods where the accuracy of completion decreases significantly with increasing missing value ratios due to neglecting the correlation between variables.

[0067] Please see Figure 5 , Figure 5 This is a comparison chart of the prediction effects of different imputation methods at a 10% missing rate, provided by an embodiment of the present invention. Figure 5 As shown in the figure, this bar chart displays the three evaluation indicators of the load forecasting model—the absolute value of the mean deviation (|MBE|), the coefficient of determination (R²), and the consistency correlation coefficient (CCC)—when meteorological data, after being supplemented using the method of this invention, the mean interpolation method, and the linear interpolation method respectively, are used as input. Figure 5 As can be seen, the prediction results using the method of this invention to complete the data have the lowest |MBE| index, indicating the smallest prediction bias; and the highest R² and CCC indices, indicating the best goodness of fit and consistency between the predicted and actual values. This advantage stems from the synergistic effect of the multiple independent imputation and result fusion steps in the method of this invention: multiple independent imputations provide diverse and reasonable estimates for missing values, while the fusion calculation effectively suppresses the random fluctuations that may be introduced by a single imputation, thus providing more stable and reliable input data for downstream prediction models. Therefore, the method of this invention not only improves the accuracy of missing data imputation itself, but also significantly improves the overall performance of load forecasting tasks that rely on this data, demonstrating its engineering application value as an upstream data processing link.

[0068] Please see Figure 6 , Figure 6 A structural block diagram of a chain equation multiple interpolation device for comprehensive energy and meteorological missing data provided in an embodiment of the present invention. Figure 6As shown: The chain equation multiple interpolation device 600 for integrating missing energy and meteorological data includes: an acquisition module 610, a generation module 620, an execution module 630, and a result module 640, wherein:

[0069] The acquisition module 610 is used to acquire the original multivariate meteorological data matrix of the integrated energy system. The data matrix contains the observed values ​​and missing values ​​of at least two meteorological variables at different sampling times.

[0070] The generation module 620 is used to standardize the meteorological variables and perform initial imputation on the missing values ​​of each meteorological variable to generate an initial complete data matrix.

[0071] Execution module 630 is used to perform M independent interpolation processes to generate M complete data matrices, where M is an integer greater than 1. The independent interpolation processes include:

[0072] Based on the initial complete data matrix, multiple rounds of chain-like iterative updates are performed. Each round of chain-like iterative updates includes: sequentially identifying a meteorological variable with missing values ​​in the original multivariate meteorological data matrix as the current target variable, and using the values ​​of other meteorological variables in the current basic complete data matrix (excluding the current target variable) as input features; training a regression model based on all observed values ​​of the current target variable; using the trained regression model to predict and update the missing values ​​of the current target variable, replacing the values ​​of the current target variable in the current basic complete data matrix; after all meteorological variables with missing values ​​have been updated once, the updated complete data matrix is ​​used as the basis for the next round of chain-like iterative updates; in response to the chain-like iterative updates meeting the preset convergence conditions, the chain-like iterative updates in the interpolation process are terminated, and the complete data matrix obtained in this round of updates is determined as a complete data matrix output by this round of interpolation.

[0073] Module 640 is obtained, which is used to perform inverse normalization on the M complete data matrices generated by the M rounds of interpolation, and then perform fusion calculation on the values ​​of the same sampling time and the same meteorological variable in the M complete data matrices after inverse normalization to obtain the final meteorological data completion result.

[0074] It should be noted that the chain equation multiple interpolation device for missing comprehensive energy and meteorological data provided in the above embodiments is only illustrated by the division of the above functional modules when executing the chain equation multiple interpolation method for missing comprehensive energy and meteorological data. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the chain equation multiple interpolation device for missing comprehensive energy and meteorological data provided in the above embodiments and the chain equation multiple interpolation method embodiments for missing comprehensive energy and meteorological data belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0075] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0076] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 7 As shown, the electronic device 700 may include: at least one processor 701, at least one network interface 704, a user interface 703, a memory 705, and at least one communication bus 702.

[0077] The communication bus 702 is used to enable communication between these components.

[0078] The user interface 703 may include a display screen and a camera. Optional user interfaces 703 may include standard wired interfaces and wireless interfaces.

[0079] The network interface 704 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0080] The processor 701 may include one or more processing cores. The processor 701 connects to various parts within the electronic device 700 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 705, and by calling data stored in the memory 705. Optionally, the processor 701 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 701 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 701 and may be implemented as a separate chip.

[0081] The memory 705 may include random access memory (RAM) or read-only memory. Optionally, the memory 705 may include a non-transitory computer-readable storage medium. The memory 705 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 705 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 705 may also be at least one storage device located remotely from the aforementioned processor 701. Figure 7 As shown, the memory 705, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a chain equation multiple interpolation application for missing comprehensive energy and meteorological data.

[0082] exist Figure 7In the illustrated electronic device 700, the user interface 703 is mainly used to provide an input interface for the user and acquire user input data; while the processor 701 can be used to call the chain equation multiple interpolation application for the integrated energy meteorological missing data stored in the memory 705, and specifically perform the following operations: acquire the original multivariate meteorological data matrix of the integrated energy system, the data matrix containing the observed values ​​and missing values ​​of at least two meteorological variables at different sampling times; standardize the meteorological variables and perform initial interpolation on the missing values ​​of each meteorological variable to generate an initial complete data matrix; execute M independent interpolation processes to generate M complete data matrices, where M is an integer greater than 1, wherein the independent interpolation process includes: based on the initial complete data matrix, performing multiple rounds of chain iterative updates, wherein each round of chain iterative update includes The process includes: First, identifying a meteorological variable with missing values ​​in the original multivariate meteorological data matrix as the current target variable. Then, using the values ​​of all other meteorological variables in the current complete data matrix as input features, a regression model is trained based on all observed values ​​of the current target variable. Next, the trained regression model is used to predict and update the missing values ​​of the current target variable, replacing their values ​​in the current complete data matrix. After all meteorological variables with missing values ​​have been updated once, the updated complete data matrix is ​​used as the basis for the next round of chain-iterative updates. Finally, in response to the current round of chain-iterative updates meeting a preset convergence condition, the chain-iterative updates in the interpolation process are terminated, and the complete data matrix obtained in this round of updates is determined as the output complete data matrix of this round of interpolation.

[0083] After performing inverse normalization on the M complete data matrices generated by the M-round interpolation process, the values ​​of the same sampling time and the same meteorological variable in the M complete data matrices after inverse normalization are fused and calculated to obtain the final meteorological data completion result.

[0084] In some possible embodiments, processor 701 performs initial imputation for the missing values ​​of each meteorological variable, specifically by performing:

[0085] For any meteorological variable, calculate the mean of all observed values ​​of the meteorological variable, and use the mean as the initial imputation value for the missing values ​​of the meteorological variable.

[0086] In some possible embodiments, the preset convergence conditions include:

[0087] Calculate the relative change between the complete data matrix obtained after this round of chain iteration update and the complete data matrix obtained after the previous round of chain iteration update;

[0088] If the relative change is less than the preset convergence threshold, or the number of rounds of chain-iterative updates reaches the preset maximum number of iterations, then the preset convergence condition is satisfied.

[0089] In some possible embodiments, the preset convergence threshold is 10. -3 The maximum number of iterations is preset to 50.

[0090] In some possible implementations, the regression model is any one of a random forest regression model, a gradient boosting regression tree model, or a support vector regression model.

[0091] In some possible embodiments, processor 701 performs standardization processing on meteorological variables, specifically for performing:

[0092] For any meteorological variable, calculate the mean and standard deviation of all observed values ​​of the meteorological variable, and use the mean and standard deviation to perform Z-score standardization on all values ​​of the meteorological variable.

[0093] In some possible embodiments, the processor 701 performs a fusion calculation on the values ​​of the same meteorological variable at the same sampling time in the M complete data matrices after inverse normalization, to obtain the final meteorological data completion result, specifically for the following purposes:

[0094] Calculate the arithmetic mean of M values ​​of the same meteorological variable at the same sampling time, and use the arithmetic mean as the final completion result for that sampling time and that meteorological variable.

[0095] This invention also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the above-described instructions. Figure 2 One or more steps in the illustrated embodiment. If the constituent modules of the chain equation multiple interpolation device for the aforementioned integrated energy and meteorological missing data are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0096] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid state disks (SSDs)).

[0097] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation schemes can be combined arbitrarily.

[0098] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A chain equation multiple interpolation method for comprehensive energy and meteorological missing data, characterized in that, The method includes: Obtain the original multivariate meteorological data matrix of the integrated energy system, wherein the data matrix contains the observed values ​​and missing values ​​of at least two meteorological variables at different sampling times; The meteorological variables are standardized, and the missing values ​​of each meteorological variable are initially imputed to generate an initial complete data matrix. Perform M independent interpolation processes to generate M complete data matrices, where M is an integer greater than 1. The independent interpolation processes include: Based on the initial complete data matrix, multiple rounds of chain-like iterative updates are performed. Each round of chain-like iterative updates includes: sequentially identifying a meteorological variable with missing values ​​in the original multivariate meteorological data matrix as the current target variable, and using the values ​​of other meteorological variables in the current basic complete data matrix (excluding the current target variable) as input features; training a regression model based on all observations of the current target variable; using the trained regression model to predict and update the missing values ​​of the current target variable to replace the values ​​of the current target variable in the current basic complete data matrix; after all meteorological variables with missing values ​​have been updated once, using the updated complete data matrix as the basis for the next round of chain-like iterative updates; in response to the current round of chain-like iterative updates satisfying a preset convergence condition, terminating the chain-like iterative updates in the interpolation process, and determining the complete data matrix obtained in this round of updates as a complete data matrix output by this round of interpolation process; After performing inverse normalization on the M complete data matrices generated by the M rounds of interpolation, the values ​​of the same sampling time and the same meteorological variable in the M complete data matrices after inverse normalization are fused and calculated to obtain the final meteorological data completion result.

2. The chain equation multiple interpolation method for missing comprehensive energy and meteorological data according to claim 1, characterized in that, The initial imputation of missing values ​​for each of the meteorological variables includes: For any of the meteorological variables, calculate the mean of all observed values ​​of the meteorological variable, and use the mean as the initial imputation value for the missing values ​​of the meteorological variable.

3. The chain equation multiple interpolation method for comprehensive energy and meteorological missing data according to claim 1, characterized in that, The preset convergence conditions include: Calculate the relative change between the complete data matrix obtained after this round of chain iteration update and the complete data matrix obtained after the previous round of chain iteration update; If the relative change is less than a preset convergence threshold, or the number of iterations in the chain-like update reaches a preset maximum number of iterations, then the preset convergence condition is satisfied.

4. The chain equation multiple interpolation method for missing comprehensive energy meteorological data according to claim 3, characterized in that, The preset convergence threshold is 10. -3 The maximum number of iterations is preset to 50.

5. The chain equation multiple interpolation method for comprehensive energy and meteorological missing data according to claim 1, characterized in that, The regression model can be any one of the following: random forest regression model, gradient boosting regression tree model, or support vector regression model.

6. The chain equation multiple interpolation method for comprehensive energy and meteorological missing data according to claim 1, characterized in that, The standardization process for the meteorological variables includes: For any of the meteorological variables, calculate the mean and standard deviation of all observed values ​​of the meteorological variable, and use the mean and standard deviation to perform Z-score standardization on all values ​​of the meteorological variable.

7. The chain equation multiple interpolation method for comprehensive energy and meteorological missing data according to claim 1, characterized in that, The values ​​of the same sampling time and the same meteorological variable in the M complete data matrices after inverse normalization are fused and calculated to obtain the final meteorological data completion result, including: Calculate the arithmetic mean of M values ​​of the same meteorological variable at the same sampling time, and use the arithmetic mean as the final completion result for that sampling time and that meteorological variable.

8. A chain equation multiple interpolation device for comprehensive energy and meteorological missing data, characterized in that, The device includes: The acquisition module is used to acquire the original multivariate meteorological data matrix of the integrated energy system. The data matrix contains the observed values ​​and missing values ​​of at least two meteorological variables at different sampling times. The generation module is used to standardize the meteorological variables and perform initial imputation on the missing values ​​of each meteorological variable to generate an initial complete data matrix. The execution module is used to perform M independent interpolation processes to generate M complete data matrices, where M is an integer greater than 1. The independent interpolation processes include: Based on the initial complete data matrix, multiple rounds of chain-like iterative updates are performed. Each round of chain-like iterative updates includes: sequentially identifying a meteorological variable with missing values ​​in the original multivariate meteorological data matrix as the current target variable, and using the values ​​of other meteorological variables in the current basic complete data matrix (excluding the current target variable) as input features; training a regression model based on all observations of the current target variable; using the trained regression model to predict and update the missing values ​​of the current target variable to replace the values ​​of the current target variable in the current basic complete data matrix; after all meteorological variables with missing values ​​have been updated once, using the updated complete data matrix as the basis for the next round of chain-like iterative updates; in response to the current round of chain-like iterative updates satisfying a preset convergence condition, terminating the chain-like iterative updates in the interpolation process, and determining the complete data matrix obtained in this round of updates as a complete data matrix output by this round of interpolation process; The module is used to perform inverse normalization on the M complete data matrices generated by the M rounds of interpolation, and then perform fusion calculation on the values ​​of the same sampling time and the same meteorological variable in the M complete data matrices after inverse normalization to obtain the final meteorological data completion result.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the steps of the method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method as described in any one of claims 1 to 7.

Images