Data compression method and device based on dynamic parameter adjustment

Abstract

The invention relates to a data compression method and device based on dynamic parameter adjustment. The method comprises the following steps: constructing an initial data fitting model based on historical data of power grid operation; continuously collecting real-time data flow of the power grid, extracting real-time data points in the real-time data flow, and substituting the real-time data points into the initial data fitting model to obtain predicted values of the real-time data points; calculating an error between the predicted value and the corresponding real-time data point, and dynamically adjusting parameters of the initial data fitting model according to the error by adopting a multi-stage optimization strategy; performing fusion processing on the multiple groups of adjusted model parameters, and constructing a final data fitting model; and extracting subsequent real-time data points in the real-time data stream, substituting the subsequent real-time data points into the final data fitting model, and determining a data compression result of the real-time data stream based on the final data fitting model under the condition that the final data fitting model meets a preset termination condition. By adopting the method, the data precision can be improved.

Description

Technical Field

[0001] This application relates to the field of data compression technology, and in particular to a data compression method, apparatus, computer equipment, readable storage medium, and program product based on dynamic parameter adjustment. Background Technology

[0002] Currently, most data compression methods commonly used in power grid digital platforms are based on linear compression algorithms, such as the rotating door compression algorithm. While these algorithms can achieve data compression to a certain extent, they suffer from the following problems when processing large-scale, complex, and nonlinear power grid data:

[0003] Traditional linear compression algorithms are only suitable for data with uniform signal distribution and simple trends. Real-time power grid data often exhibits nonlinear characteristics (such as sudden load changes), which linear models cannot adequately represent, resulting in low compression ratios and wasted storage resources. When processing high-frequency real-time data streams, linear compression algorithms require significant computation and transmission, easily introducing processing delays and impacting real-time power grid monitoring and emergency response. Power grid data is affected by equipment status, environment, and other factors, exhibiting strong nonlinearity. Traditional linear compression algorithms struggle to adapt to such variations, resulting in large compression errors and potentially leading to the loss of critical information. Summary of the Invention

[0004] Therefore, it is necessary to provide a data compression method, apparatus, computer equipment, computer-readable storage medium, and computer program product based on dynamic parameter adjustment that can improve data accuracy in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a data compression method based on dynamic parameter adjustment, including:

[0006] An initial data fitting model is constructed based on historical power grid operation data.

[0007] Continuously collect real-time data streams of the power grid, extract real-time data points from the real-time data streams, and substitute them into the initial data fitting model to obtain the predicted values ​​of the real-time data points;

[0008] The error between the predicted value and the corresponding real-time data point is calculated, and a multi-stage optimization strategy is adopted to dynamically adjust the parameters of the initial data fitting model according to the error, so as to obtain multiple sets of adjusted model parameters.

[0009] The multiple sets of adjusted model parameters are fused to obtain target model parameters, and a final data fitting model is constructed based on the target model parameters.

[0010] Subsequent real-time data points are extracted from the real-time data stream and substituted into the final data fitting model. When the parameter adjustment state of the final data fitting model and the error between the output predicted value and the corresponding real-time data point both meet the preset termination conditions, the data compression result of the real-time data stream is determined based on the final data fitting model.

[0011] In one embodiment, the multi-stage optimization strategy dynamically adjusts the parameters of the initial data fitting model based on the error, resulting in multiple sets of adjusted model parameters, including:

[0012] Based on the error, the direction and magnitude of parameter adjustment are determined, and with the goal of minimizing the difference between the current error and the historical error, a parameter optimization algorithm is used to calculate the parameter adjustment increment.

[0013] Based on the parameters, the parameters of the model fitted to the current data are iteratively updated by adjusting the increment.

[0014] In one embodiment, the process of fusing the multiple sets of adjusted model parameters to obtain the target model parameters includes:

[0015] Based on the matching degree between each group of adjusted model parameters and the corresponding real-time data points, a corresponding weight is assigned to each group of adjusted model parameters, and the weight is positively correlated with the matching degree.

[0016] Based on the weights, a weighted average is calculated for the adjusted model parameters of each group to obtain the target model parameters.

[0017] In one embodiment, the preset termination condition includes the error between the predicted value output by the final data fitting model and the corresponding real-time data point being less than a preset error threshold, and the change in the magnitude of two adjacent parameter adjustments being less than a set standard.

[0018] In one embodiment, before fusing the multiple sets of adjusted model parameters to obtain the target model parameters, the following steps are included:

[0019] If the error is less than a preset error threshold, abnormal parameter groups that do not match the real-time data stream change pattern are removed from the multiple sets of adjusted model parameters.

[0020] In one embodiment, constructing the initial data fitting model based on historical power grid operation data includes:

[0021] Based on the historical data and prior knowledge of the domain, the initial range of values ​​for the model parameters is determined;

[0022] The historical data are fitted using a data fitting algorithm, and the initial model parameters are obtained by solving within the initial value range.

[0023] Based on the initial model parameters, an initial data fitting model is constructed.

[0024] Secondly, this application also provides a data compression device based on dynamic parameter adjustment, comprising:

[0025] The model building module is used to build an initial data fitting model based on historical power grid operation data.

[0026] The data prediction module is used to continuously collect real-time data streams from the power grid, extract real-time data points from the real-time data streams, and substitute them into the initial data fitting model to obtain the predicted values ​​of the real-time data points.

[0027] The model optimization module is used to calculate the error between the predicted value and the corresponding real-time data point, and adopt a multi-stage optimization strategy to dynamically adjust the parameters of the initial data fitting model according to the error, so as to obtain multiple sets of adjusted model parameters.

[0028] The model building module is also used to fuse the multiple sets of adjusted model parameters to obtain target model parameters, and to build a final data fitting model based on the target model parameters.

[0029] The data compression module is used to extract subsequent real-time data points from the real-time data stream and substitute them into the final data fitting model. When the parameter adjustment state of the final data fitting model and the error between the output predicted value and the corresponding real-time data point both meet the preset termination conditions, the data compression result of the real-time data stream is determined based on the final data fitting model.

[0030] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the above claims.

[0031] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0032] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above claims.

[0033] The aforementioned data compression methods, devices, computer equipment, computer-readable storage media, and computer program products based on dynamic parameter adjustment achieve adaptive adaptation to the dynamic characteristics of power grid data by constructing a basic model based on historical power grid data and continuously optimizing the model in conjunction with real-time data streams. This improves data compression efficiency and effectively reduces storage resource consumption while ensuring data feature integrity and fitting accuracy, meeting the core requirements of power grid digital platforms for rapid transmission and efficient storage of large-scale real-time data. By employing multi-stage optimization strategies and parameter fusion processing, the model's adaptability to complex power grid data is enhanced, effectively resolving conflicts and inconsistencies among multi-source data, improving data fusion accuracy, and thus ensuring the real-time performance and reliability of power grid monitoring and control. Simultaneously, the combination of dynamic parameter adjustment and preset termination conditions reduces invalid computational consumption during data processing, further improving overall data processing efficiency and avoiding processing delays common in traditional methods. This makes it suitable for power grid application scenarios with extremely high real-time requirements. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating a data compression method based on dynamic parameter adjustment in one embodiment;

[0036] Figure 2 This is a flowchart illustrating the steps of using a multi-stage optimization strategy in one embodiment to dynamically adjust the parameters of the initial data fitting model based on the error, thereby obtaining multiple sets of adjusted model parameters.

[0037] Figure 3 This is a flowchart illustrating the steps of fusing multiple sets of adjusted model parameters to obtain target model parameters in one embodiment.

[0038] Figure 4 This is a flowchart illustrating a data compression method based on dynamic parameter adjustment in another embodiment;

[0039] Figure 5 This is a flowchart illustrating step S110 in one embodiment;

[0040] Figure 6 This is a structural block diagram of a data compression device based on dynamic parameter adjustment in one embodiment;

[0041] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0044] In one embodiment, such as Figure 1 As shown, a data compression method based on dynamic parameter adjustment is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0045] S110, based on historical power grid operation data, constructs an initial data fitting model.

[0046] Historical data on power grid operation can refer to the collection of power-related data collected and stored by various devices and sensors during the past operation of the power grid, including but not limited to equipment operating parameters, load change data, power supply and demand data, and environmental impact-related data.

[0047] For example, the terminal can collect historical data of power grid operation, combine prior knowledge in the power grid field to determine the reasonable range of model parameters, use a data fitting algorithm adapted to the time distribution characteristics of power grid data to perform statistical analysis and pattern extraction on the historical data, solve for the initial model parameters that can reflect the core change patterns of the historical data within the preset parameter value range, and then construct an initial data fitting model based on the initial model parameters.

[0048] S120 continuously collects real-time data streams from the power grid, extracts real-time data points from the real-time data streams, and substitutes them into the initial data fitting model to obtain the predicted values ​​of the real-time data points.

[0049] In this context, the real-time power grid data stream refers to a continuous sequence of power data generated and transmitted in real time by various sensors, monitoring terminals, and communication networks during power grid operation. A real-time data point refers to an independent data unit extracted from the real-time power grid data stream according to preset rules, containing a unique time identifier and a corresponding actual power data value.

[0050] For example, the terminal can continuously acquire real-time power grid data streams through the multi-source data transmission links (including wired communication links, wireless communication links, etc.) of the power grid monitoring system; according to the data acquisition frequency and data granularity requirements, it can extract real-time data points one by one from the real-time data stream, determine the time identifier of each data point and the corresponding actual value of power data; input the time identifier of the real-time data point into the constructed initial data fitting model to obtain the predicted value corresponding to the real-time data point.

[0051] S130 calculates the error between the predicted value and the corresponding real-time data point, and adopts a multi-stage optimization strategy to dynamically adjust the parameters of the initial data fitting model according to the error, thereby obtaining multiple sets of adjusted model parameters.

[0052] For example, the terminal can use preset deviation calculation rules (including absolute error calculation, relative error calculation, etc.) to solve the error between the predicted value and the actual value corresponding to each real-time data point; initiate a multi-stage optimization strategy, including: the first stage aims to establish a benchmark range for parameter adjustment with the goal of conforming to the distribution characteristics of historical data; the subsequent stages continuously receive new real-time data points and dynamically update the error calculation results, and adaptively adjust the direction and intensity of parameter optimization according to the error change trend, data fluctuation amplitude, and other information, and adapt parameters to the characteristics of nonlinear changes and multi-factor coupling effects of power grid data; through multiple rounds of phased iterative adjustments, obtain multiple sets of adjusted model parameters adapted to different data change scenarios (including stable change scenarios, violent fluctuation scenarios, etc.).

[0053] S140: The multiple sets of adjusted model parameters are fused to obtain the target model parameters, and the final data fitting model is constructed based on the target model parameters.

[0054] For example, the terminal can first screen the validity of multiple sets of adjusted model parameters, eliminating abnormal parameter sets that do not conform to the overall change pattern of the real-time power grid data stream or whose fitting error exceeds a reasonable range; then, based on the fitting accuracy of each set of valid parameters in the corresponding data scenario (i.e., the degree of matching with real-time data points), assign corresponding weights to each set of parameters (the higher the fitting accuracy, the larger the weight ratio); according to the preset parameter integration rules, perform fusion processing on all valid parameters to obtain the target model parameters; substitute the target model parameters into the basic structure of the data fitting model to construct the final data fitting model that can adapt to the dynamic change characteristics of the real-time power grid data.

[0055] S150: Extract subsequent real-time data points from the real-time data stream and substitute them into the final data fitting model. If the parameter adjustment state of the final data fitting model and the error between the output predicted value and the corresponding real-time data point both meet the preset termination conditions, determine the data compression result of the real-time data stream based on the final data fitting model.

[0056] For example, the preset termination conditions may include stability criteria for the model parameter adjustment state and error control criteria between the predicted and actual values.

[0057] For example, the terminal can continuously extract subsequent real-time data points from the real-time data stream of the power grid, and substitute them one by one into the final data fitting model. It can then calculate the error between the predicted value output by the model and the actual value of the subsequent real-time data points in real time, while simultaneously monitoring the parameter adjustment status of the final data fitting model. When the fluctuation range of the model parameter adjustment is detected to be lower than the preset stability standard (i.e., the parameters tend to stabilize and converge), and the error between the predicted value and the actual value is controlled within the preset accuracy threshold range, the preset termination condition is determined to be met. At this point, based on the final data fitting model, the entire real-time data stream of the power grid is systematically processed. Core data features that reflect key operating states of the power grid and changes in equipment operating conditions are retained, while duplicate, invalid, and redundant data information is removed. This allows for the final determination of the data compression result of the real-time data stream of the power grid.

[0058] The aforementioned data compression method based on dynamic parameter adjustment achieves adaptive adaptation to the dynamic characteristics of power grid data by constructing a basic model based on historical power grid data and continuously optimizing the model in conjunction with real-time data streams. This improves data compression efficiency and effectively reduces storage resource consumption while ensuring data feature integrity and fitting accuracy, meeting the core requirements of the power grid digital platform for rapid transmission and efficient storage of large-scale real-time data. By employing multi-stage optimization strategies and parameter fusion processing, the model's adaptability to complex power grid data is enhanced, effectively resolving conflicts and inconsistencies among multi-source data and improving data fusion accuracy, thereby ensuring the real-time performance and reliability of power grid monitoring and control. Simultaneously, the combination of dynamic parameter adjustment and preset termination conditions reduces invalid computational consumption during data processing, further improving overall data processing efficiency and avoiding processing delays common in traditional methods. This makes it suitable for power grid application scenarios with extremely high real-time requirements.

[0059] In one exemplary embodiment, such as Figure 2 As shown, the steps described above, which employ a multi-stage optimization strategy to dynamically adjust the parameters of the initial data fitting model based on the error, and obtain multiple sets of adjusted model parameters, include:

[0060] S210: Based on the error, determine the direction and magnitude of parameter adjustment, and with the goal of minimizing the difference between the current error and the historical error, use a parameter optimization algorithm to calculate the parameter adjustment increment.

[0061] S220, iteratively updates the parameters of the model fitted to the current data based on parameter adjustment increments.

[0062] For example, the terminal can determine the current error (the error is the deviation between the actual value of the real-time data point and the predicted value output by the initial data fitting model); with the goal of minimizing the difference between the current error and the historical error, a parameter optimization algorithm is used, combined with a preset learning rate, to analyze the correlation between the error and the model parameters, and calculate the adjustment increment corresponding to each parameter; the parameters of the current data fitting model are superimposed with the corresponding adjustment increments to obtain the updated model parameters, completing a single round of parameter iterative update; in each stage of the multi-stage optimization strategy, the parameter adjustment increment calculation and iterative update steps are repeated to obtain multiple sets of adjusted model parameters.

[0063] In this embodiment, by determining and iteratively updating the direction and magnitude of parameter adjustment, the model parameters are adapted to the changing characteristics of power grid data, thereby improving the efficiency and accuracy of parameter adjustment.

[0064] In one exemplary embodiment, such as Figure 3 As shown, the steps described above for fusing multiple sets of adjusted model parameters to obtain the target model parameters include:

[0065] S310: Based on the matching degree between the adjusted model parameters of each group and the corresponding real-time data points, assign corresponding weights to the adjusted model parameters of each group. The weights are positively correlated with the matching degree.

[0066] S320, based on the weights, calculates the weighted average of the adjusted model parameters of each group to obtain the target model parameters.

[0067] For example, the terminal can apply each set of adjusted model parameters to the real-time data points of the corresponding stage for fitting verification, calculate the deviation between the fitting result and the actual value of the real-time data points, and the reciprocal of the deviation represents the matching degree between the set of parameters and the real-time data points; assign weights to each set of adjusted model parameters according to the matching degree; and calculate the weighted average of the model parameters of all sets based on the weights to obtain the target model parameters.

[0068] In this embodiment, by using matching degree-oriented weight allocation and weighted average calculation, and integrating the parameter features of each stage, the adaptation requirements of different data change scenarios are balanced, thereby improving the adaptability of the target model parameters and thus improving the compression accuracy and stability of the final data fitting model.

[0069] In an exemplary embodiment, the aforementioned preset termination conditions include the error between the predicted value output by the final data fitting model and the corresponding real-time data point being less than a preset error threshold, and the change in the magnitude of two adjacent parameter adjustments being less than a set standard.

[0070] For example, the terminal can determine a preset error threshold based on the requirements of the power grid digital platform for data compression accuracy and the error distribution characteristics of historical data; determine a set standard based on the fluctuation characteristics of power grid data and the requirements for data processing efficiency, wherein the set standard is that the change amplitude of each parameter in the model is lower than a preset percentage in two adjacent parameter adjustments; substitute subsequent real-time data points into the final data fitting model, calculate the error between the predicted value and the actual value, and record the change amplitude of each parameter in two adjacent parameter adjustments; when the error is less than the preset error threshold and the change amplitude of each parameter is lower than the set standard, it is determined that the preset termination condition is met.

[0071] In this embodiment, the data compression process is terminated when the accuracy is met through the coordinated determination of dual termination conditions, thus avoiding insufficient compression accuracy or waste of computing resources and balancing data compression accuracy and efficiency.

[0072] In one exemplary embodiment, such as Figure 4 As shown, the above-mentioned data compression method based on dynamic parameter adjustment may further include:

[0073] S160, if the error is less than the preset error threshold, remove abnormal parameter groups from multiple sets of adjusted model parameters that do not match the real-time data stream change pattern.

[0074] For example, the terminal can determine a preset error threshold; for multiple sets of adjusted model parameters, substitute each set of parameters into the corresponding real-time data points for fitting calculation to obtain the fitting error of each set of parameters; filter parameter sets whose fitting errors are less than the preset error threshold; analyze the degree of fit between the fitting curves corresponding to the filtered parameter sets and the changing patterns of the real-time data stream; and determine parameter sets whose fitting curves cannot reflect the core changing characteristics of the real-time data stream as abnormal parameter sets and remove them.

[0075] In this embodiment, by filtering errors and verifying the adaptability of variation patterns, abnormal parameter groups are eliminated, reducing the interference of irrelevant parameters on the calculation of target model parameters, ensuring the effectiveness and adaptability of the parameter groups on which the fusion processing is based, and improving the accuracy and reliability of the data compression results.

[0076] In one exemplary embodiment, such as Figure 5 As shown, step S110 above includes:

[0077] A1. Based on historical data and prior knowledge of the domain, determine the initial range of values ​​for the model parameters.

[0078] A2 uses a data fitting algorithm to fit and calculate historical data, and solves for the initial model parameters within the initial value range.

[0079] A3. Based on the initial model parameters, construct the initial data fitting model.

[0080] For example, the terminal can collect historical data on power grid operation, combine it with prior knowledge in the power grid field, determine the initial range of parameter values ​​for the initial data fitting model; use a data fitting algorithm to substitute the time variables and corresponding power data values ​​in the historical data into the model, perform fitting calculations within the initial range, and solve for the initial model parameters; substitute the initial model parameters into a preset model structure to construct the initial data fitting model.

[0081] In this embodiment, the initial range of parameter values ​​is determined by historical data and prior knowledge of the domain. The initial model parameters are then solved using a data fitting algorithm, ensuring that the initial data fitting model closely matches the distribution characteristics of historical power grid data. This provides a reliable foundation for real-time data processing and dynamic parameter adjustment, improving the initial adaptability of the compression algorithm and the efficiency of subsequent optimization. The initial data fitting model is then constructed.

[0082] In one possible implementation, the terminal can construct an initial data fitting model based on historical power grid operation data, and combine this with prior knowledge of the power grid domain to determine the initial range of model parameters. , , Assume the data follows a quadratic function. (Where a, b, and c are the hyperparameters of the quadratic function, and t is the time variable), the least squares method is used to fit the historical data, and preliminary parameter estimates are obtained within the above initial value range. , , Based on the initial model parameters , , Construct an initial data fitting model.

[0083] Then, real-time power grid data streams are continuously collected through the multi-source data transmission link of the power grid monitoring system; real-time data points containing unique time identifiers and corresponding actual power data values ​​are extracted from the real-time data streams according to preset rules. Determine the time identifier for each data point. and the corresponding actual power data values Time stamp Substitute the initial data into the model The predicted value corresponding to the real-time data point is obtained through model calculation. .

[0084] Then use the formula (in, To calculate the error between the predicted value and the actual value of the corresponding real-time data point (for error tolerance), a multi-stage optimization strategy is then used to dynamically adjust the parameters of the initial data fitting model based on this error, resulting in multiple sets of adjusted model parameters. The multi-stage optimization strategy specifically includes three stages: the first stage is the preliminary fitting stage, which determines the initial model parameters based on the fitting results of historical data. The first stage establishes a baseline range for parameter adjustment; the second stage is the dynamic adjustment and error optimization stage, which continuously receives new real-time data points. The error calculation results are dynamically updated, and if the error exceeds a set threshold... Then, the process moves to the third stage; this is the fine-tuning stage, where the direction and intensity of parameter optimization are adaptively adjusted based on information such as the error trend and data fluctuation amplitude. During parameter adjustment, the goal is to minimize the difference between the current error E and historical errors. Gradient descent is used as the parameter optimization algorithm, combined with a preset learning rate. Through the formula: , , Calculate the adjustment increment for each parameter. , , Then through the formula , , The current model parameters are superimposed with the corresponding adjustment increments to obtain the updated model parameters, completing a single round of parameter iteration update. Through multiple rounds of phased iterative adjustments, multiple sets of adjusted model parameters adapted to different data change scenarios are obtained. , , (i is the stage number).

[0085] Before fusing multiple sets of adjusted model parameters, a preset error threshold can be determined first. For each set of adjusted model parameters , , Substitute them into the corresponding real-time data points respectively. Through formula Calculate the fitting error for each set of parameters and select those with fitting errors less than a preset error threshold. The parameter sets are then analyzed to determine the degree of fit between the fitted curves and the real-time data stream's changing patterns. Parameter sets whose fitted curves fail to reflect the core changing characteristics of the real-time data stream are identified as abnormal and removed. Finally, weights are assigned to each parameter set based on the matching degree between the effective parameters and the corresponding real-time data points. The matching degree is positively correlated with the weight; that is, the smaller the fitting bias, the higher the matching degree, and the higher the weight. The larger the proportion; based on this weight Through parameter fusion formula , , (Where n is the total number of parameter adjustment stages) The target model parameters are obtained by performing a weighted average calculation on all effective parameters. , , The target model parameters are then substituted into the quadratic function model structure. , and construct the final data fitting model.

[0086] The terminal can continuously extract subsequent real-time data points from the real-time data stream of the power grid. Substitute them one by one into the final data fitting model The system calculates the error between the predicted value output by the real-time calculation model and the actual value of subsequent real-time data points, and simultaneously monitors the parameter adjustment status of the final data fitting model. Preset termination conditions include: the error between the predicted value output by the final data fitting model and the corresponding real-time data point is less than a preset error threshold ϵ; and the change in the magnitude of two adjacent parameter adjustments is less than a set standard (this set standard can be determined based on the fluctuation characteristics of power grid data and data processing efficiency requirements; specifically, it can be that the change in the magnitude of each parameter a, b, and c in two adjacent parameter adjustments is less than a preset percentage). When both of the above conditions are met, the terminal determines that the preset termination conditions are met, and systematically processes the entire power grid real-time data stream based on the final data fitting model. It retains core data characteristics that reflect key power grid operating states and equipment operating condition changes, removes duplicate, invalid, and redundant data information, and finally determines the data compression result of the power grid real-time data stream.

[0087] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0088] Based on the same inventive concept, this application also provides a data compression apparatus based on dynamic parameter adjustment for implementing the data compression method based on dynamic parameter adjustment described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more embodiments of the data compression apparatus based on dynamic parameter adjustment provided below can be found in the limitations of the data compression method based on dynamic parameter adjustment described above, and will not be repeated here.

[0089] In one exemplary embodiment, such as Figure 6 As shown, a data compression device 600 based on dynamic parameter adjustment is provided, including: a model building module 601, a data prediction module 602, a model optimization module 603, and a data compression module 604, wherein:

[0090] Model building module 601 is used to build an initial data fitting model based on historical power grid operation data;

[0091] The data prediction module 602 is used to continuously collect real-time data streams from the power grid, extract real-time data points from the real-time data streams, and substitute them into the initial data fitting model to obtain the predicted values ​​of the real-time data points.

[0092] The model optimization module 603 is used to calculate the error between the predicted value and the corresponding real-time data point, and adopts a multi-stage optimization strategy to dynamically adjust the parameters of the initial data fitting model according to the error, so as to obtain multiple sets of adjusted model parameters.

[0093] The aforementioned model building module 601 is also used to fuse multiple sets of adjusted model parameters to obtain target model parameters, and to build a final data fitting model based on the target model parameters.

[0094] The data compression module 604 is used to extract subsequent real-time data points from the real-time data stream and substitute them into the final data fitting model. Under the condition that the parameter adjustment state of the final data fitting model and the error between the output predicted value and the corresponding real-time data point both meet the preset termination conditions, the data compression result of the real-time data stream is determined based on the final data fitting model.

[0095] In one embodiment, the model optimization module 603 is further configured to:

[0096] Based on the error, the direction and magnitude of parameter adjustment are determined. With the goal of minimizing the difference between the current error and the historical error, a parameter optimization algorithm is used to calculate the parameter adjustment increment.

[0097] The parameters of the model fitted to the current data are iteratively updated based on the parameter adjustment increment.

[0098] In one embodiment, the model building module 601 is further configured to:

[0099] Based on the matching degree between the adjusted model parameters of each group and the corresponding real-time data points, corresponding weights are assigned to the adjusted model parameters of each group. The weights are positively correlated with the matching degree.

[0100] Based on the weights, a weighted average is calculated for the adjusted model parameters of each group to obtain the target model parameters.

[0101] In one embodiment, the aforementioned preset termination conditions include the error between the predicted value output by the final data fitting model and the corresponding real-time data point being less than a preset error threshold, and the change in the magnitude of two adjacent parameter adjustments being less than a set standard.

[0102] In one embodiment, the data compression device 600 based on dynamic parameter adjustment further includes a parameter processing module, used for:

[0103] If the error is less than the preset error threshold, remove abnormal parameter groups from multiple sets of adjusted model parameters that do not match the real-time data stream change pattern.

[0104] In one embodiment, the model building module 601 is further configured to:

[0105] Based on historical data and prior domain knowledge, the initial range of values ​​for model parameters is determined;

[0106] A data fitting algorithm is used to fit and calculate the historical data, and the initial model parameters are obtained within the initial value range.

[0107] Based on the initial model parameters, an initial data fitting model is constructed.

[0108] The modules in the aforementioned data compression device based on dynamic parameter adjustment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0109] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a data compression method based on dynamic parameter adjustments. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0110] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0111] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0112] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0113] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0114] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0116] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data compression method based on dynamic parameter adjustment, characterized in that, The method includes: An initial data fitting model is constructed based on historical power grid operation data. Continuously collect real-time data streams of the power grid, extract real-time data points from the real-time data streams, and substitute them into the initial data fitting model to obtain the predicted values ​​of the real-time data points; The error between the predicted value and the corresponding real-time data point is calculated, and a multi-stage optimization strategy is adopted to dynamically adjust the parameters of the initial data fitting model according to the error, so as to obtain multiple sets of adjusted model parameters. The multiple sets of adjusted model parameters are fused to obtain target model parameters, and a final data fitting model is constructed based on the target model parameters. Subsequent real-time data points are extracted from the real-time data stream and substituted into the final data fitting model. When the parameter adjustment state of the final data fitting model and the error between the output predicted value and the corresponding real-time data point both meet the preset termination conditions, the data compression result of the real-time data stream is determined based on the final data fitting model.

2. The method according to claim 1, characterized in that, The multi-stage optimization strategy dynamically adjusts the parameters of the initial data fitting model based on the error, resulting in multiple sets of adjusted model parameters, including: Based on the error, the direction and magnitude of parameter adjustment are determined, and with the goal of minimizing the difference between the current error and the historical error, a parameter optimization algorithm is used to calculate the parameter adjustment increment. Based on the parameters, the parameters of the model fitted to the current data are iteratively updated by adjusting the increment.

3. The method according to claim 1, characterized in that, The process of fusing the multiple sets of adjusted model parameters to obtain the target model parameters includes: Based on the matching degree between each group of adjusted model parameters and the corresponding real-time data points, a corresponding weight is assigned to each group of adjusted model parameters, and the weight is positively correlated with the matching degree. Based on the weights, a weighted average is calculated for the adjusted model parameters of each group to obtain the target model parameters.

4. The method according to claim 1, characterized in that, The preset termination conditions include the error between the predicted value output by the final data fitting model and the corresponding real-time data point being less than a preset error threshold, and the change in the magnitude of two adjacent parameter adjustments being less than a set standard.

5. The method according to claim 1, characterized in that, Before fusing the multiple sets of adjusted model parameters to obtain the target model parameters, the following steps are included: If the error is less than a preset error threshold, abnormal parameter groups that do not match the real-time data stream change pattern are removed from the multiple sets of adjusted model parameters.

6. The method according to claim 1, characterized in that, The initial data fitting model constructed based on historical power grid operation data includes: Based on the historical data and prior knowledge of the domain, the initial range of values ​​for the model parameters is determined; The historical data is fitted using a data fitting algorithm, and the initial model parameters are obtained by solving within the initial value range. Based on the initial model parameters, an initial data fitting model is constructed.

7. A data compression device based on dynamic parameter adjustment, characterized in that, The device includes: The model building module is used to build an initial data fitting model based on historical power grid operation data. The data prediction module is used to continuously collect real-time data streams from the power grid, extract real-time data points from the real-time data streams, and substitute them into the initial data fitting model to obtain the predicted values ​​of the real-time data points. The model optimization module is used to calculate the error between the predicted value and the corresponding real-time data point, and adopt a multi-stage optimization strategy to dynamically adjust the parameters of the initial data fitting model according to the error, so as to obtain multiple sets of adjusted model parameters. The model building module is also used to fuse the multiple sets of adjusted model parameters to obtain target model parameters, and to build a final data fitting model based on the target model parameters. The data compression module is used to extract subsequent real-time data points from the real-time data stream and substitute them into the final data fitting model. When the parameter adjustment state of the final data fitting model and the error between the output predicted value and the corresponding real-time data point both meet the preset termination conditions, the data compression result of the real-time data stream is determined based on the final data fitting model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images