A data resource optimization system based on digital twinning

Abstract

The application discloses a kind of data resource optimization systems based on digital twinning, the system includes: data resource access mark module, twin mapping module, processing action preplay module, bias contribution quantification module, value decay modeling module, retention strategy generation module, strategy execution module, feedback correction module and result output module;The application relates to the technical field of data resource management.This kind of data resource optimization systems based on digital twinning, through data resource twin unit, the mapping relationship between data resource and digital twin object, model parameter, business state, prediction task, simulation node and update time window is established, so that the source, action position, influence range and recovery path of each data resource in digital twinning environment can be traced back, improve the explainability and reviewability of data resource optimization process.

Description

Technical Field

[0001] This invention relates to the field of data resource management, and more specifically, to a data resource optimization system based on digital twins. Background Technology

[0002] As enterprise business systems, IoT data acquisition systems, and digital twin platforms continue to operate, structured data, log data, interface data, sensor data, file data, image data, and event data accumulate. The retention, migration, and archiving of these data resources directly impact the state restoration, predictive analysis, and anomaly identification capabilities of the digital twin model. In existing technologies, CN114860833B discloses a data platform and data processing method applied to digital twin water conservancy projects, focusing on data source management, data integration, data governance, and data application; CN118466852B discloses a classification and hierarchical storage method for reservoir digital twin data, primarily based on data dictionary mapping and hot / cold heat indexing for classification and hierarchical storage; and CN116684303B discloses a data center operation and maintenance method and system based on digital twins, focusing on the simulation and management of data center operation and maintenance status. While the aforementioned solutions can optimize data access, governance, or storage in digital twin scenarios, they generally fail to pre-assess the impact of different processing actions on the twin's state, prediction results, risk level, business process status, and anomaly identification results before data deletion, downsampling, summarization, or archiving. Furthermore, they do not couple the deviation contribution with the data value decay curve to form a closed-loop callback mechanism. Therefore, a data resource optimization system capable of dynamically determining the data retention format based on digital twin simulation results is needed. Summary of the Invention

[0003] The purpose of this invention is to provide a data resource optimization system based on digital twins, which solves the problem that existing data resource optimization methods mainly determine data cleaning, migration and archiving strategies based on data generation time, access frequency or storage capacity, making it difficult to identify the actual impact of data resources on the state restoration, prediction tasks and anomaly identification results of digital twin models.

[0004] This invention achieves the above objectives through the following technical solution: a data resource optimization system based on digital twins, the system comprising:

[0005] The module includes a data resource access and labeling module, a twin mapping module, a processing action pre-simulation module, a deviation contribution quantification module, a value decay modeling module, a retention strategy generation module, a strategy execution module, a feedback correction module, and a result output module.

[0006] The data resource access tagging module is used to access structured data, log data, interface data, sensor data, file data, image data, and event data in the business system, and generate corresponding data resource tags.

[0007] The twin mapping module is used to construct data resource twin units based on data resource tags, and to establish mapping relationships between data resource twin units and digital twin objects, model parameters, business status, prediction tasks, simulation nodes, and update time windows.

[0008] The processing action pre-simulation module is used to generate simulated data versions of the data resources to be optimized under different processing actions in the digital twin environment;

[0009] The deviation contribution quantification module is used to input each simulation data version into the same digital twin model and compare it with the baseline output before processing to obtain the twin deviation contribution.

[0010] The value decay modeling module is used to generate data resource value decay curves.

[0011] The retention strategy generation module is used to couple the twin bias contribution with the data resource value decay curve to generate a hierarchical retention strategy.

[0012] The policy execution module is used to configure the retention type, storage area, access permissions and update cycle according to the hierarchical retention policy;

[0013] The feedback correction module is used to perform callback corrections on the hierarchical retention strategy based on model running deviations.

[0014] The result output module is used to output the data resource optimization results.

[0015] Furthermore, the data resource tags include:

[0016] Data source, generation time, business scenario, field type, call frequency, data quality level, update time, associated objects, anomaly markers, permission level, and historical call results;

[0017] The historical call results include the predicted output, simulation output, state restoration output, and anomaly identification output generated when the data resource is used in the digital twin model call;

[0018] The data quality level is determined by the proportion of missing data, the proportion of duplicate data, the consistency of format, the amount of time delay, and the proportion of outliers, and is updated synchronously with the data resources.

[0019] Furthermore, the twin mapping module includes:

[0020] Tag parsing unit, object matching unit, and mapping index unit;

[0021] The tag parsing unit is used to parse the business scenario, associated objects, generation time, and update time.

[0022] The object matching unit is used to match data resources to the state parameters, structural parameters, prediction task input items, simulation node input items, or business process nodes of the digital twin object.

[0023] The mapping index unit is used to form an index table between data resource identifiers, twin object identifiers, model parameter identifiers, task identifiers, node identifiers, and time window identifiers, and to update the index table when data resources are updated, business scenarios are switched, or model parameter versions change.

[0024] Furthermore, the processing action pre-rehearsal module includes:

[0025] Action configuration unit, simulated copy generation unit, and twin invocation unit;

[0026] The processing actions configured by the action configuration unit include original retention, field pruning, time downsampling, feature summarization, event indexing, compressed storage, migrated storage, archived storage, and candidate deletion;

[0027] The simulation copy generation unit generates a corresponding simulation data version within the isolated pre-rehearsal area;

[0028] The twin calling unit inputs the simulated data version into the digital twin model without rewriting the original data of the business system.

[0029] Furthermore, the deviation contribution quantification module includes:

[0030] The unit includes a benchmark output acquisition unit, a pre-simulation output acquisition unit, a deviation vector generation unit, and a contribution level determination unit.

[0031] The baseline output acquisition unit is used to acquire the state of the twin object before processing, the prediction result, the risk level, the resource consumption simulation result, the business process status, and the anomaly identification result;

[0032] The deviation vector generation unit is used to generate a deviation vector that includes state deviation, prediction deviation, risk deviation, resource deviation, process deviation, and identification deviation.

[0033] The contribution level determination unit determines the twin deviation contribution level based on the deviation vector and a preset threshold range.

[0034] When any deviation component falls into the first threshold range, it is determined to be in the first level; when all deviation components fall into the second threshold range, it is determined to be in the second level.

[0035] The first threshold interval is the interval that exceeds the reserved verification threshold, and the second threshold interval is the interval that does not exceed the downgrade verification threshold.

[0036] Furthermore, the value decay modeling module includes:

[0037] Type identification unit, attenuation rule matching unit, and curve update unit;

[0038] The type identification unit identifies real-time monitoring data, basic archive data, abnormal event data, periodic statistical data, and model training samples based on data resource tags;

[0039] The decay rule matching unit matches fast decay rules, slow decay rules, event retention rules, periodic update rules, and contribution retention rules, respectively.

[0040] The curve update unit updates the value decay curve based on the generation time, call frequency, changes in business scenarios, changes in data quality, changes in repetition, and changes in model contribution, and records the curve version.

[0041] Furthermore, the retention policy generation module includes:

[0042] Coupled matrix construction unit, form preservation determination unit, and strategy verification unit;

[0043] The coupling matrix construction unit constructs a data resource retention matrix with twin bias contribution level as the first dimension and the decay interval where the value decay curve is located as the second dimension.

[0044] The retention form determination unit determines the data resources as original data, feature summary, event index, statistical fragment, archived copy or candidate deleted data based on the data resource retention matrix, and generates corresponding recovery conditions.

[0045] The policy verification unit is used to verify whether the retention format is consistent with the permission level, business scenario, and update time window.

[0046] Furthermore, when there are abnormal markers in the data resource tags, or when the data resource is identified as a boundary sample or model correction sample, the retention form determination unit limits the minimum retention form of the data resource to event index and key fragments.

[0047] When the frequency of data resource access is lower than the preset access threshold, the retention form determination unit does not directly determine it as candidate data for deletion, but retains the event occurrence time, event type, associated object, key fields, key time period and recovery path.

[0048] Furthermore, the policy execution module includes:

[0049] Storage area configuration unit, permission configuration unit, and periodic configuration unit;

[0050] The storage area configuration unit configures data resources with twin bias contribution level of level 1 and value decay curve in slow decay interval to high response storage area, configures data resources with twin bias contribution level of level 1 and value decay curve in fast decay interval as event index or feature summary, configures data resources with twin bias contribution level of level 2 and value decay curve in slow decay interval as statistical fragment or low frequency archive copy, and configures data resources with twin bias contribution level of level 2 and value decay curve in fast decay interval as compressed, merged or candidate deletion status;

[0051] The permission configuration unit configures call permissions according to permission levels and business scenarios;

[0052] The period configuration unit configures the update period based on the decay interval.

[0053] Furthermore, the feedback correction module includes:

[0054] Operation monitoring unit, deviation triggering unit, resource location unit, and level callback unit;

[0055] The operation monitoring unit is used to continuously monitor the prediction deviation, state restoration deviation, and anomaly identification accuracy of the digital twin model;

[0056] The deviation triggering unit triggers resource location when the prediction deviation or state restoration deviation exceeds the corresponding preset threshold, or when the anomaly recognition accuracy is lower than the corresponding preset threshold.

[0057] The resource location unit locates the data resources that affect the prediction task, simulation node, model parameters and time window based on the mapping relationship between the data resource twin unit and the prediction task, simulation node, model parameters and time window.

[0058] The level callback unit will affect the upward adjustment of the retention level of data resources and adjust the corresponding retention form from candidate deletion, archived copy, statistical fragment or feature summary to event index, key fragment or raw data;

[0059] The result output module synchronously outputs the adjusted retention format, storage area, calling permissions, update cycle, recovery conditions, feedback correction rules and execution time, and writes the result of this callback into the historical call results.

[0060] The beneficial effects of this invention are as follows:

[0061] 1. Before performing field pruning, time downsampling, feature summarization, archiving, or candidate deletion, this invention first rehearses the processing actions in a digital twin environment and compares the deviation between the rehearsal output and the baseline output. This enables the identification of the actual contribution of data resources to the state of the digital twin object, prediction results, risk level, business process status, and anomaly identification results, thereby reducing the risk of critical data being mistakenly deleted or excessively downgraded.

[0062] 2. This invention establishes a mapping relationship between data resources and digital twin objects, model parameters, business status, prediction tasks, simulation nodes, and update time windows through data resource twin units, so that the source, position, scope of influence, and recovery path of each data resource in the digital twin environment can be traced, thereby improving the interpretability and verifiability of the data resource optimization process.

[0063] 3. This invention couples the twin bias contribution with the data resource value decay curve, and can generate differentiated hierarchical retention strategies based on the different value change patterns of real-time monitoring data, basic archive data, abnormal event data, periodic statistical data and model training samples, avoiding the use of uniform access frequency rules to process different types of data resources.

[0064] 4. This invention sets a minimum retention format for abnormal event samples, boundary samples, and model correction samples. Even if such data resources are called less frequently during the statistical period, at least the event index and key fragments are retained, thereby preserving the data foundation required for anomaly review, boundary state correction, and model retraining.

[0065] 5. This invention continuously monitors the prediction bias, state restoration bias, and anomaly identification accuracy of the digital twin model through a feedback correction module. When the model's performance declines, it locates the relevant data resources and upgrades their retention level, transforming data resource optimization from a one-time static configuration into a recoverable and reversible closed-loop optimization process. Attached Figure Description

[0066] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0067] Figure 1 This is a system block diagram of the present invention;

[0068] Figure 2 This is a flowchart illustrating the data resource optimization process of the present invention.

[0069] Figure 3 This is a flowchart of the data processing action simulation and twin bias contribution evaluation of the present invention;

[0070] Figure 4This is a flowchart of the retention strategy generation and twin feedback correction process of the present invention. Detailed Implementation

[0071] The present application will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are only used to further illustrate the present application and should not be construed as limiting the scope of protection of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.

[0072] Example 1:

[0073] Please see Figure 1-4 This invention provides a data resource optimization system based on digital twins. The system includes a data resource access marking module, a twin mapping module, a processing action pre-simulation module, a deviation contribution quantification module, a value decay modeling module, a retention strategy generation module, a strategy execution module, a feedback correction module, and a result output module. These modules can be deployed in the same data resource management server, or they can be deployed separately in a business system server, a digital twin platform server, and a storage management server, and interact through data interfaces.

[0074] In one embodiment, the data resource access tagging module is used to access structured data, log data, interface data, sensor data, file data, image data, and event data from the business system, and to generate a data resource tag for each data resource. The data resource tag includes data source, generation time, business scenario, field type, call frequency, data quality level, update time, associated object, anomaly flag, permission level, and historical call results. Historical call results include the predicted output, simulation output, state restoration output, and anomaly identification output generated when the data resource participates in the digital twin model call.

[0075] To ensure the reproducibility of data quality level determination, the data resource access marking module determines the data quality defect rate based on the proportion of missing data, duplicate data, format errors, time delays, and outliers. The calculation formula is as follows:

[0076]

[0077] in, Indicates the first Data quality defect rate of each data resource, dimensionless; The missing percentage is determined by the ratio of the number of missing fields to the number of fields that should be collected. The repetition rate is determined by the ratio of the number of duplicate records to the total number of records. The percentage of records with formatting errors is determined by the ratio of the number of records whose formatting does not conform to the field rules to the total number of records. The time lag ratio is determined by the ratio of the number of records exceeding the allowable acquisition delay to the total number of records. This represents the outlier ratio, determined by the ratio of the number of records exceeding the business allowable upper and lower limits to the total number of records. All the above ratios are... to The dimensionless values ​​between.

[0078] Determined by the maximum value This prevents a serious quality defect from being offset by other, less severe defects, ensuring that the data quality level reflects the most significant sources of data risk. Specifically, the number of missing fields, duplicate records, and malformed records are automatically counted by the data resource access marking module according to field rules; the allowable collection delay is determined by the data collection cycle, for example, a sampling cycle of [missing information]. The sensor data, whose allowable acquisition delay is set to to The upper and lower limits of the allowed business operations are determined by the first normal sample in the history of the business system. percentiles and the Percentiles, determined in conjunction with equipment operating boundaries or business rule boundaries. When When the data quality level is determined to be Level 1; when When the data quality level is determined to be Level 2; when At that time, the data quality level was determined to be Level 3. The same level classification rules were used within the same business scenario.

[0079] In one embodiment, the twin mapping module is used to construct data resource twin units based on data resource tags and establish mapping relationships between data resource twin units and digital twin objects, model parameters, business status, prediction tasks, simulation nodes, and update time windows. The twin mapping module includes a tag parsing unit, an object matching unit, and a mapping index unit. The tag parsing unit is used to parse business scenarios, associated objects, generation time, and update time; the object matching unit is used to match data resources to the status parameters, structural parameters, prediction task inputs, simulation node inputs, or business process nodes of the digital twin object; the mapping index unit is used to form an index table between data resource identifiers, twin object identifiers, model parameter identifiers, task identifiers, node identifiers, and time window identifiers.

[0080] The advantage of this design is that data resources are no longer managed solely based on files, table fields, or log records, but rather on their role and function within the digital twin environment. When a sensor data point corresponds to device temperature, pressure, or flow rate, the twin mapping module associates it with the corresponding digital twin object's operational status parameters and fault prediction tasks. When an interface log corresponds to a business process interruption event, the twin mapping module associates it with the business process node and anomaly identification node. When a historical sample is used for model calibration, the twin mapping module associates it with the model parameter version and training sample set. Through these mapping relationships, the source, role, scope of influence, and update time window of data resources within the digital twin environment can be traced.

[0081] In one embodiment, the processing action pre-simulation module is used to generate simulated data versions of the data resources to be optimized under different processing actions in a digital twin environment. The processing action pre-simulation module includes an action configuration unit, a simulated copy generation unit, and a twin invocation unit. The processing actions configured by the action configuration unit include original data retention, field pruning, time downsampling, feature summarization, event indexing, compressed storage, migrated storage, archived storage, and candidate deletion. The simulated copy generation unit generates the corresponding simulated data version within an isolated pre-simulation area. The twin invocation unit inputs the simulated data version into the digital twin model without rewriting the original data in the business system.

[0082] Specifically, field pruning is used to delete fields that have no direct input relationship with the current twin task; time downsampling is used to reduce the sampling density of high-frequency time series data; feature summarization is used to convert the original data into peak, mean, fluctuation range, duration, and trend features; event indexing is used to retain event occurrence time, event type, related objects, key fields, key time periods, and recovery paths; compressed storage is used to reduce storage space usage; migration storage is used to transfer data resources to low-frequency storage areas; archive storage is used to create low-frequency call copies; and candidate deletion is used to mark data resources as pending deletion and wait for feedback for correction and confirmation. The advantage of this design is that the system can observe the impact of different processing actions on the twin model output in advance without destroying the original business data.

[0083] In one embodiment, the deviation contribution quantification module is used to input each version of simulated data into the same digital twin model and compare it with the baseline output before processing to obtain the twin deviation contribution. Before formally implementing the hierarchical retention of data resources, the system first selects at least a continuous range of data... Historical stable operating data is used as the benchmark sample set. Historical stable operating data refers to data where the digital twin model has not undergone version switching, business processes have not been restructured, and data resources have not undergone large-scale migration or deletion. The benchmark sample set is used to determine the model's natural fluctuation range, reference value for call frequency, value decay time constant, and deviation trigger threshold.

[0084] The deviation contribution quantification module includes a baseline output acquisition unit, a pre-simulation output acquisition unit, a deviation vector generation unit, and a contribution level determination unit. The baseline output acquisition unit acquires the state of the digital twin object before processing, prediction results, risk level, resource usage simulation results, business process status, and anomaly identification results. The pre-simulation output acquisition unit acquires the corresponding outputs after each simulation data version is input into the digital twin model. The deviation vector generation unit generates state deviation, prediction deviation, risk deviation, resource deviation, process deviation, and identification deviation. The contribution level determination unit determines the twin deviation contribution level based on the above deviations and a preset threshold range.

[0085] For the The first processing action The formula for calculating the normalized bias component in the model output is as follows:

[0086]

[0087] in, Indicates the first The first processing action corresponds to the first The normalized bias component output by the model is dimensionless. Indicates the first The first processing action Class model output values; Indicates the first time when no processing action is performed. Class benchmark output value; Indicates the first The minimum stable reference value for the class output is used to avoid the reference output from approaching... This can cause the deviation to be amplified abnormally. The model output includes the twin object status, prediction results, risk level, resource consumption simulation results, business process status, and anomaly identification results.

[0088] Before calculating the above formula, the system first performs unified processing on model outputs with different dimensions. If the model output is temperature, pressure, flow rate, storage capacity, or time duration, then... and First, convert all values ​​to the same units before calculating; if the model output is precision, recall, or risk level, convert it first. to The standardized values ​​between the values ​​are used; if the model output is a business process state, it is converted into a dimensionless deviation value based on whether the state nodes are consistent, the deviation in state duration, and the deviation in state transition order. Through the above processing, twin model outputs from different sources, different units, and different forms of representation can be compared on the same scale.

[0089] In one embodiment, the deviation contribution quantification module does not use a simple weighted summation of multiple deviation components, but instead uses the maximum deviation method to determine the deviation contribution of a single processing action. The calculation formula is as follows:

[0090]

[0091] in, Indicates the first The deviation contribution corresponding to each processing action is dimensionless. This represents the set of model output types participating in the comparison. The reason for this setting is that during the data resource optimization process, if any key model output deviates significantly, it may affect the state restoration, prediction judgment, or anomaly identification results of the digital twin model; using the maximum deviation value can highlight key deviation items and prevent high-risk outputs from being diluted by other low-deviation outputs.

[0092] Furthermore, for the same data resource, the processing action pre-simulation module sequentially generates simulated data versions corresponding to field pruning, time downsampling, feature summarization, event indexing, compressed storage, migrated storage, archived storage, and candidate deletion, and calculates the deviation contribution for each. The formula for calculating the twin deviation contribution value of this data resource is as follows:

[0093]

[0094] in, Indicates the first The twin bias contribution value of each data resource, dimensionless; Indicates the first A set of processing actions involving data resources in the pre-rehearsal. If Then the data resource is identified as a high twin bias contributing data resource; if Then the data resource is identified as a low twin bias contributing data resource; if If so, the data resource is identified as an observation data resource, and further judgment is made based on the value decay curve.

[0095] Among them, the retention verification threshold and downgrade verification threshold The range of natural fluctuations in the model is determined by the baseline sample set. Specifically, during the historical stable operation phase without data processing, the natural deviation components of various model outputs within adjacent statistical periods are calculated according to the normalized deviation component calculation method, and the i-th value is taken. Percentile as a retention check threshold The initial value is taken as its first value. Percentile as a threshold for downgrade verification The initial value. When the business system has a clear tolerance limit for prediction error or anomaly identification error. The threshold must not exceed the dimensionless value corresponding to this tolerance limit. Through this method, the threshold is derived from the model's own fluctuations without exceeding the business's permissible boundaries.

[0096] In one embodiment, the value decay modeling module is used to generate a data resource value decay curve. The value decay modeling module includes a type identification unit, a decay rule matching unit, and a curve update unit. The type identification unit identifies real-time monitoring data, basic archive data, abnormal event data, periodic statistical data, and model training samples based on data resource tags; the decay rule matching unit matches rapid decay rules, slow decay rules, event-based rules, periodic update rules, and contribution-based rules, respectively; the curve update unit updates the value decay curve based on generation time, call frequency, changes in business scenarios, changes in data quality, changes in repetition, and changes in model contribution, and records the curve version.

[0097] To standardize the value judgment criteria for different data resources, the value depreciation value is limited to [value value]. to The dimensionless values ​​between them are calculated using the following formula:

[0098]

[0099] in, Indicates the first Each data resource at any time The value of data resources is dimensionless. Indicates the generation time of the data resource; This indicates the duration the data resource has existed, with the unit being days. This represents the time constant of value decay, expressed in days. This indicates that the call retains the coefficient, which is dimensionless; It represents the minimum retention value and is dimensionless.

[0100] Call the retention coefficient The value range is determined based on the current statistical period call frequency and historical reference call frequency. to When the call frequency in the current statistical period is greater than or equal to the historical reference call frequency, When the call frequency in the current statistical period is lower than the historical reference call frequency, Determined by the ratio of the two. Historical reference call frequency is determined by the call frequency of similar data resources in the benchmark sample set. Percentiles are determined. This prevents the value of data resources from being excessively reduced due to occasional low-frequency access.

[0101] Determined based on data resource type: Real-time monitoring data Pick From the date Daily; periodic statistical data Take one to three statistical periods; basic archival data Pick From the date Day; Model training samples Determined based on the most recent model training or calibration period; anomalous event data and boundary samples. Not lower than The above parameters are determined through historical operation records, business update cycles, and model training cycles; when the business scenario changes or the model version is switched, the value decay modeling module regenerates the curve version.

[0102] In one embodiment, the retention strategy generation module couples the twin bias contribution with the data resource value decay curve to generate a tiered retention strategy. The retention strategy generation module includes a coupling matrix construction unit, a retention form determination unit, and a strategy verification unit. The coupling matrix construction unit constructs a data resource retention matrix with the twin bias contribution level as the first dimension and the decay interval of the value decay curve as the second dimension. The retention form determination unit, based on the data resource retention matrix, identifies the data resource as original data, feature summary, event index, statistical fragment, archived copy, or candidate deletion data, and generates corresponding recovery conditions. The strategy verification unit verifies whether the retention form is consistent with the permission level, business scenario, and update time window.

[0103] Specifically, if ,and The retention policy generation module will retain the data resources as original data and configure them to a high-response storage area; if ,and If so, the data resources will be retained as event indexes or feature summaries; if ,and If so, the data resources will be retained as statistical fragments or low-frequency archived copies; ,and If so, the data resources will be configured to be compressed, merged, or candidate for deletion.

[0104] Among them, the value retention threshold The value decay curve of similar data resources is determined based on the median of the value decay curve in the benchmark sample set; for basic archival data, abnormal event data, and model training samples, Not lower than For real-time monitoring data and periodic statistical data, It can be adjusted according to the business update cycle, but should remain consistent within the same business scenario.

[0105] In one embodiment, when anomaly markers are present in the data resource tags, or when the data resource is identified as a boundary sample or model correction sample, the retention form determination unit limits the minimum retention form of the data resource to event indexes and key fragments. Even if the data resource's call frequency is lower than a preset call threshold, it is not directly determined as candidate data for deletion; instead, the event occurrence time, event type, associated objects, key fields, key time periods, and recovery path are retained. The advantage of this design is that it can prevent low-frequency but high-value data resources from being mistakenly deleted during routine cleanup.

[0106] In one embodiment, the policy execution module is used to configure retention types, storage areas, access permissions, and update cycles according to a tiered retention policy. The policy execution module includes a storage area configuration unit, a permission configuration unit, and a cycle configuration unit. The storage area configuration unit allocates data resources with high twin-swap bias contribution and slow value decay to a high-response storage area; allocates data resources with high twin-swap bias contribution and fast value decay as event indexes or feature summaries; allocates data resources with low twin-swap bias contribution and slow value decay as statistical fragments or low-frequency archived copies; and allocates data resources with low twin-swap bias contribution and fast value decay to compressed, merged, or candidate deletion states. The permission configuration unit configures access permissions based on permission levels and business scenarios. The cycle configuration unit configures the update cycle based on the decay interval.

[0107] In one embodiment, the feedback correction module is used to perform callback corrections on the hierarchical retention strategy based on model operation deviations. The feedback correction module includes an operation monitoring unit, a deviation triggering unit, a resource location unit, and a level callback unit. The operation monitoring unit continuously monitors the prediction deviation, state restoration deviation, and anomaly identification accuracy of the digital twin model. The deviation triggering unit triggers resource location when the prediction deviation or state restoration deviation exceeds a corresponding preset threshold, or when the anomaly identification accuracy is lower than a corresponding preset threshold. The resource location unit locates the data resources affecting the model based on the mapping relationship between the data resource twin unit and the prediction task, simulation node, model parameters, and time window. The level callback unit upgrades the retention level of the data resources affecting the model and adjusts the corresponding retention format from candidate deletion, archived copy, statistical fragment, or feature summary to event index, key fragment, or original data.

[0108] The feedback correction module monitors the operational status of the digital twin model after hierarchical retention and calculates the feedback trigger deviation. The calculation formula is as follows:

[0109]

[0110] in, Indicates time Feedback triggering bias, dimensionless; This indicates the prediction bias after implementing tiered retention; Represents the baseline value of the prediction deviation in the baseline sample set; This indicates the deviation in state restoration after performing hierarchical retention; This represents the baseline value for state restoration deviation; This indicates the anomaly identification accuracy after implementing hierarchical retention. Indicates the baseline anomaly identification accuracy; , and These are the corresponding reference values, used to prevent the denominators from being too close. Prediction bias and state restoration bias are uniformly converted to dimensionless relative bias before calculation; anomaly identification accuracy is uniformly converted to... to Decimals between [a certain number] and [a certain number].

[0111] when At this time, the feedback correction module triggers resource location. Feedback correction threshold. The first bias triggered by feedback from the benchmark sample set Percentile determination; when digital twin models are used for security risk identification, fault warning, or anomaly detection, Not higher than When used for general resource scheduling or low-risk business status analysis, The upper limit of historical model fluctuations can be determined. The resource location unit, based on the mapping relationship between the data resource twin unit and the prediction task, simulation nodes, model parameters, and time window, locates the factors leading to... Increase the relevant data resources, and the retention level of the corresponding data resources will be upgraded by the level callback unit.

[0112] In one embodiment, the result output module is used to output the data resource optimization results. The data resource optimization results include data resource identifier, data source, retention format, storage area, access permissions, update cycle, recovery conditions, feedback correction rules, and execution time. The recovery conditions are used to trigger the recovery call of archived copies, statistical fragments, event indexes, or key fragments when the digital twin model experiences increased prediction bias, increased state restoration bias, or decreased anomaly identification accuracy. The feedback correction rules are used to update the tiered retention strategy for subsequent similar data resources based on the changes in model output after the recovery call.

[0113] Through the above implementation methods, this invention can, without directly altering the original data of the business system, first rehearse different data processing actions in a digital twin environment, then determine the deviation contribution of data resources based on the differences in the output of the twin model before and after processing, and generate a tiered retention strategy by combining the value decay curve. This system can reduce the storage resource consumption of low-contribution, rapidly decaying data, while retaining data resources that play a crucial role in state restoration, prediction tasks, risk assessment, and anomaly identification, thereby achieving traceability, recoverability, and closed-loop correction in the data resource optimization process.

[0114] Example 2:

[0115] In this embodiment, a data resource optimization system based on digital twins is applied to an energy equipment operation management platform in an industrial park. This platform includes air compressors, cooling pumps, distribution cabinets, temperature and humidity sensors, and energy consumption metering equipment. The business system generates daily equipment sensor data, interface call logs, maintenance records, alarm events, energy consumption statistics tables, and equipment file data. The digital twin platform is used to reconstruct and predict equipment operating status, energy consumption trends, fault risks, and abnormal events.

[0116] The system accesses continuous data resources through the data resource access marking module. Historical operating data for the day, including the previous No model version switching, equipment structure adjustment, or data migration operations occurred on that day, therefore, this Daily data serves as the baseline sample set. The system uniformly converts temperature data to... Pressure data is uniformly converted to Traffic data is uniformly converted to Storage capacity is uniformly converted to The time interval is uniformly converted to Prediction bias, state restoration bias, and anomaly detection accuracy are all converted into... to The dimensionless values ​​between.

[0117] During the data resource access phase, the system generates data resource tags for the air compressor temperature sensing data. The tag content includes: the data source is the air compressor temperature acquisition terminal, and the generation time is every A single data collection session was conducted for equipment fault prediction. The field type was a continuous numeric field, the associated object was a digital twin of an air compressor, and the access level was operation and maintenance. Historical data retrieved included fault prediction output and status restoration output. The system calculated the data quality defect rate for this data resource using a data quality defect rate calculation method. The system also determined the missing percentage of this temperature sensor data. repetition ratio Formatting error rate Time delay ratio Outlier ratio ,but .because The system determines the data quality level of the temperature sensor data to be Level 1.

[0118] The missing value ratio, duplicate value ratio, time delay ratio, and outlier ratio mentioned above are all derived from the data access marking module. Automatic statistics of the daily baseline sample set; where time lag is determined based on the sampling period. Allow the acquisition delay to be set to The upper and lower limits of outliers are determined by the historical normal sample number. percentiles and the Percentiles are determined in conjunction with air compressor operating rules. This design ensures a clear source for data quality levels and avoids the inability to verify data across different implementation scenarios due to manual experience-based scoring.

[0119] The twin mapping module constructs a temperature data resource twin unit based on the data resource tag, and establishes a digital twin object of the data resource with the air compressor, thermal state parameters, fault prediction tasks, alarm simulation nodes, and so on. Update the mapping relationship between time windows. With this setting, when temperature data is downsampled, summarized, or candidate deleted, the system can directly locate the prediction task and simulation node affected by it, avoiding the inability to trace the data source during subsequent feedback corrections.

[0120] The processing action pre-simulation module generates simulated data versions of the temperature sensor data within the isolated pre-simulation area, including original retention, time downsampling, feature summarization, archive storage, and candidate deletion versions. Time downsampling includes... Sampling adjusted to Sampling; Feature summarization includes retaining each The data includes the highest, lowest, and average temperatures within the time window, as well as the temperature rise slope. Archive storage involves migrating the original data to a low-frequency archive storage area. Candidate deletion includes retaining only the data resource identifier and the underlying index path. All of these processing actions are performed in an isolated pre-launch area without rewriting the original temperature data in the business system.

[0121] The deviation contribution quantification module inputs each simulation data version into the same digital twin model of the air compressor and compares it with the baseline output before processing. For time downsampling processing, the system obtains the predicted deviation component as follows: The state restoration deviation component is The risk level deviation component is The anomaly identification deviation component is Thus, the deviation contribution corresponding to time downsampling processing is obtained. For candidate deletion processing, the system obtains the prediction bias component as follows: The state restoration deviation component is The risk level deviation component is The anomaly identification deviation component is Thus, the deviation contribution amount corresponding to the candidate deletion process is obtained. For the same temperature sensing data, the system takes the maximum value among the deviation contributions of each processing action as the twin bias contribution value, thus obtaining... .

[0122] In the benchmark sample set, the system calculates the first natural fluctuation of the model within adjacent statistical periods. Percentile determination retains the verification threshold According to the Percentiles determine the downgrade verification threshold .because The temperature sensing data was identified as a high twin bias contributing data resource.

[0123] The value decay modeling module further generates a value decay curve based on the data resource type. Since this temperature data is real-time monitoring data, the system uses the value decay time constant. Set as This value is derived from the short-term prediction window of the air compressor failure prediction model and the duration of historical temperature anomalies; minimum retention value. Set as ;Current call retain coefficient The frequency of calls in the current statistical period is compared with the frequency of calls to similar data resources in the benchmark sample set. Percentiles are determined. If the current time is [time elapsed since data generation]... Day, and Then, based on the value decay curve, the result is obtained. Approximately In this business scenario, what is the value retention threshold for similar real-time monitoring data? The value decay curve of the benchmark sample set was determined to be... .because ,and The policy generation module retains this temperature sensing data as raw data, and the policy execution module configures it to a high-response storage area. Access permissions are maintained at the operation and maintenance level, and the update cycle remains unchanged. .

[0124] Within the same industrial park's energy equipment operation management platform, the system also processes interface call logs. A particular interface log corresponds to a cooling pump communication interruption event; this data resource is... The frequency of calls within the daily statistical period is low, and it is only called once. This interface log is typically archived or deleted directly using the standard method of cleaning up logs based on call frequency. However, when generating data resource tags, the data resource access tagging module sets its exception tag to "communication interruption event" and associates it with the cooling pump digital twin object and the business process exception identification node.

[0125] The action simulation module performs field pruning, feature summarization, archiving, and candidate deletion simulation on the interface log. The deviation contribution quantification module calculates the interface log's performance under candidate deletion processing. Less than the downgrade verification threshold However, because this data resource has an anomaly marker and has been identified as an abnormal event sample, the retention format determination unit does not directly determine it as candidate data for deletion. Instead, it limits its minimum retention format to event index and key fragments. Specific retention content includes event occurrence time, event type, associated object, interface number, exception return code, communication interruption duration, recovery time, and original log recovery path.

[0126] After the strategy is executed, the feedback correction module continuously monitors the running status of the digital twin model. If the hierarchical retention is executed... On that day, the accuracy rate of cooling pump anomaly detection increased from the baseline value. Reduce to The prediction deviation is based on the baseline value. Increase to The state restoration deviation is determined by the reference value. Increase to and set , , The feedback triggering deviation is calculated from this. In this application scenario, the feedback correction threshold... The first bias triggered by feedback from the benchmark sample set Percentiles are determined as .because The feedback correction module triggers resource location.

[0127] Based on the mapping relationship between the interface log data resource twin unit and the cooling pump anomaly identification node, communication status parameters, business process nodes, and time windows, the resource location unit identifies the communication interruption interface log as one of the affected data resources. The level callback unit upgrades the retention level of the interface log and changes its retention format from event index to critical fragment, while restoring the logs before and after the anomaly. The original log content is used for the anomaly detection model to review and retrain. The result output module synchronously outputs the adjusted data resource identifier, retention format, storage area, calling permissions, update cycle, recovery conditions, feedback correction rules, and execution time, and writes the result of this callback into the historical call results.

[0128] As can be seen from this embodiment, the present invention can employ different optimization processing methods for different types of data resources. For real-time sensor data with high twin bias contribution but still of high value, the system retains its original data and configures it in a high-response storage area; for interface logs with low call frequency but marked with anomalies, the system will not delete them directly based solely on access frequency, but will at least retain the event index and key fragments, and automatically upgrade the retention level when the model's anomaly identification capability declines. Thus, the present invention can simultaneously balance storage resource conservation, digital twin model operational stability, and anomaly event traceability.

[0129] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data resource optimization system based on digital twins, characterized in that, The system includes: The module includes a data resource access and labeling module, a twin mapping module, a processing action pre-simulation module, a deviation contribution quantification module, a value decay modeling module, a retention strategy generation module, a strategy execution module, a feedback correction module, and a result output module. The data resource access tagging module is used to access structured data, log data, interface data, sensor data, file data, image data, and event data in the business system, and generate corresponding data resource tags. The twin mapping module is used to construct data resource twin units based on data resource tags, and to establish mapping relationships between data resource twin units and digital twin objects, model parameters, business status, prediction tasks, simulation nodes, and update time windows. The processing action pre-simulation module is used to generate simulated data versions of the data resources to be optimized under different processing actions in the digital twin environment; The deviation contribution quantification module is used to input each simulation data version into the same digital twin model and compare it with the baseline output before processing to obtain the twin deviation contribution. The value decay modeling module is used to generate data resource value decay curves. The retention strategy generation module is used to couple the twin bias contribution with the data resource value decay curve to generate a hierarchical retention strategy. The policy execution module is used to configure the retention type, storage area, access permissions and update cycle according to the hierarchical retention policy; The feedback correction module is used to perform callback corrections on the hierarchical retention strategy based on model running deviations. The result output module is used to output the data resource optimization results.

2. The data resource optimization system based on digital twins according to claim 1, characterized in that, The data resource tags include: Data source, generation time, business scenario, field type, call frequency, data quality level, update time, associated objects, anomaly markers, permission level, and historical call results; The historical call results include the predicted output, simulation output, state restoration output, and anomaly identification output generated when the data resource is used in the digital twin model call; The data quality level is determined by the proportion of missing data, the proportion of duplicate data, the consistency of format, the amount of time delay, and the proportion of outliers, and is updated synchronously with the data resources.

3. The data resource optimization system based on digital twins according to claim 2, characterized in that, The twin mapping module includes: Tag parsing unit, object matching unit, and mapping index unit; The tag parsing unit is used to parse the business scenario, associated objects, generation time, and update time. The object matching unit is used to match data resources to the state parameters, structural parameters, prediction task input items, simulation node input items, or business process nodes of the digital twin object. The mapping index unit is used to form an index table between data resource identifiers, twin object identifiers, model parameter identifiers, task identifiers, node identifiers, and time window identifiers, and to update the index table when data resources are updated, business scenarios are switched, or model parameter versions change.

4. The data resource optimization system based on digital twins according to claim 1, characterized in that, The processing action pre-simulation module includes: Action configuration unit, simulated copy generation unit, and twin invocation unit; The processing actions configured by the action configuration unit include original retention, field pruning, time downsampling, feature summarization, event indexing, compressed storage, migrated storage, archived storage, and candidate deletion; The simulation copy generation unit generates a corresponding simulation data version within the isolated pre-rehearsal area; The twin calling unit inputs the simulated data version into the digital twin model without rewriting the original data of the business system.

5. The data resource optimization system based on digital twins according to claim 4, characterized in that, The deviation contribution quantification module includes: The unit includes a benchmark output acquisition unit, a pre-simulation output acquisition unit, a deviation vector generation unit, and a contribution level determination unit. The baseline output acquisition unit is used to acquire the state of the twin object before processing, the prediction result, the risk level, the resource consumption simulation result, the business process status, and the anomaly identification result; The deviation vector generation unit is used to generate a deviation vector that includes state deviation, prediction deviation, risk deviation, resource deviation, process deviation, and identification deviation. The contribution level determination unit determines the twin deviation contribution level based on the deviation vector and a preset threshold range. When any deviation component falls into the first threshold range, it is determined to be in the first level; when all deviation components fall into the second threshold range, it is determined to be in the second level. The first threshold interval is the interval that exceeds the reserved verification threshold, and the second threshold interval is the interval that does not exceed the downgrade verification threshold.

6. The data resource optimization system based on digital twins according to claim 2, characterized in that, The value decay modeling module includes: Type identification unit, attenuation rule matching unit, and curve update unit; The type identification unit identifies real-time monitoring data, basic archive data, abnormal event data, periodic statistical data, and model training samples based on data resource tags; The decay rule matching unit matches fast decay rules, slow decay rules, event retention rules, periodic update rules, and contribution retention rules, respectively. The curve update unit updates the value decay curve based on the generation time, call frequency, changes in business scenarios, changes in data quality, changes in repetition, and changes in model contribution, and records the curve version.

7. The data resource optimization system based on digital twins according to claim 6, characterized in that, The retention strategy generation module includes: Coupled matrix construction unit, form preservation determination unit, and strategy verification unit; The coupling matrix construction unit constructs a data resource retention matrix with twin bias contribution level as the first dimension and the decay interval where the value decay curve is located as the second dimension. The retention form determination unit determines the data resources as original data, feature summary, event index, statistical fragment, archived copy or candidate deleted data based on the data resource retention matrix, and generates corresponding recovery conditions. The policy verification unit is used to verify whether the retention format is consistent with the permission level, business scenario, and update time window.

8. The data resource optimization system based on digital twins according to claim 7, characterized in that: When the data resource preservation form determination unit has an abnormal marker in the data resource label, or when the data resource is identified as a boundary sample or model correction sample, the minimum preservation form of the data resource is limited to event index and key fragment. When the frequency of data resource access is lower than the preset access threshold, the retention form determination unit does not directly determine it as candidate data for deletion, but retains the event occurrence time, event type, associated object, key fields, key time period and recovery path.

9. The data resource optimization system based on digital twins according to claim 7, characterized in that, The strategy execution module includes: Storage area configuration unit, permission configuration unit, and periodic configuration unit; The storage area configuration unit configures data resources with a twin bias contribution level of the first level and a value decay curve in the slow decay range to the high-response storage area, configures data resources with a twin bias contribution level of the first level and a value decay curve in the fast decay range as event indexes or feature summaries, configures data resources with a twin bias contribution level of the second level and a value decay curve in the slow decay range as statistical fragments or low-frequency archived copies, and configures data resources with a twin bias contribution level of the second level and a value decay curve in the fast decay range as compressed, merged, or candidate deletion states. The permission configuration unit configures call permissions according to permission levels and business scenarios; The period configuration unit configures the update period based on the decay interval.

10. The data resource optimization system based on digital twins according to claim 9, characterized in that, The feedback correction module includes: Operation monitoring unit, deviation triggering unit, resource location unit, and level callback unit; The operation monitoring unit is used to continuously monitor the prediction deviation, state restoration deviation, and anomaly identification accuracy of the digital twin model; The deviation triggering unit triggers resource location when the prediction deviation or state restoration deviation exceeds the corresponding preset threshold, or when the anomaly recognition accuracy is lower than the corresponding preset threshold. The resource location unit locates the data resources that affect the prediction task, simulation node, model parameters and time window based on the mapping relationship between the data resource twin unit and the prediction task, simulation node, model parameters and time window. The level callback unit will affect the upward adjustment of the retention level of data resources and adjust the corresponding retention form from candidate deletion, archived copy, statistical fragment or feature summary to event index, key fragment or raw data; The result output module synchronously outputs the adjusted retention format, storage area, calling permissions, update cycle, recovery conditions, feedback correction rules and execution time, and writes the result of this callback into the historical call results.

Images