Cache data management method and system based on distributed encrypted storage

Abstract

The present invention discloses a cache data management and control method and system based on distributed encrypted storage, which relates to the technical field related to data processing. The method includes: obtaining distributed encrypted storage data of a storage device; establishing a data importance evaluation result; obtaining data features of cache data; generating a sequential sorting result of cache data; establishing a reward network according to the access prediction result, performing access verification of cache data, and establishing a reward result; obtaining the device load of the cache device, and performing cache data management and control according to the device load, reward result, and sequential sorting result. The method solves the technical problems existing in the existing cache data management, such as insufficient privacy protection of cache data, cache management ignoring the importance of the data itself and future access trends, which leads to low data cache hit rate and access efficiency, and low cache data security, and realizes refined dynamic regulation of cache data, achieving the technical effect of improving the efficiency and security of data cache.

Description

Technical Field

[0001] The present application relates to the technical field related to data processing, and specifically to a cache data management method and system based on distributed encrypted storage. Background Art

[0002] In today's digital age, data has become the core resource for enterprise operations and decision-making. With the explosive growth of data volume, how to efficiently and securely manage and utilize this data has become a key issue that the industry needs to solve. When faced with massive data, traditional centralized storage architecture faces challenges such as poor scalability, high risk of single point failure, and insufficient data privacy protection. With the maturity of blockchain and distributed technology, distributed encrypted storage has gradually become a new favorite in the field of data storage. However, relying solely on distributed encrypted storage is still not enough to cope with complex and changeable actual application scenarios, especially in the management of cached data. Cache is a bridge between storage systems and applications, and its performance directly affects user experience and system response speed. Traditional cache management strategies often determine the cache priority of data based on simple access frequency or timestamps, ignoring the importance of the data itself and future access trends, resulting in low cache efficiency and waste of resources.

[0003] Therefore, at the current stage, there are technical problems in cache data management related technologies, such as insufficient cache data privacy protection and cache management ignoring the importance of the data itself and future access trends, which leads to low data cache hit rate and access efficiency and low cache data security. Summary of the invention

[0004] The present application provides a cache data management and control method and system based on distributed encrypted storage, adopts distributed encrypted storage, evaluates data importance, predicts future access trends, dynamically adjusts cache strategies and other technical means, solves the technical problems existing in existing cache data management, such as insufficient cache data privacy protection, cache management ignores the importance of the data itself and future access trends, and thus leads to low data cache hit rate and access efficiency and low cache data security, realizes refined dynamic regulation of cache data, and achieves the technical effect of improving the efficiency and security of data caching.

[0005] The present application provides a cache data management and control method based on distributed encrypted storage, the method comprising: establishing digital communication with a storage device to obtain distributed encrypted storage data of the storage device; calling a user's evaluation database, performing data importance evaluation on the distributed encrypted storage data through the evaluation database, establishing a data importance evaluation result, and mapping the data importance evaluation result to the cache data; obtaining data features of the cache data, the data features comprising data size features, access frequency features, and last access time features; generating a sequential sorting result of the cache data according to the data importance evaluation result and the data features; predicting access to the cache data through historical data to establish an access prediction result, establishing a reward network according to the access prediction result, performing access verification of the cache data through the reward network, and establishing a reward result; obtaining a device load of the cache device, and performing cache data management and control according to the device load, the reward result, and the sequential sorting result.

[0006] In a possible implementation, the reward network is established according to the access prediction result, and the following processing is performed: The reward network formula is as follows:

[0007] ;

[0008] in, Characterization time Down state Take Action The reward function is To monitor the time window length, represents the time step, Characterization in time step The discount factor, The weight coefficient that characterizes cache hits, Characterization in time step Next data item Cache hits, Characterization in time step Next data item The predicted visit frequency, The weight coefficient that characterizes cache misses, Characterization in time step Next data item Cache misses, The weight coefficient that characterizes the use of cache resources, Representation in state Take Action Cache resource usage, Characterization in time step The cache collection below.

[0009] In a possible implementation, cache data management is performed based on the device load, reward results, and sequential sorting results, and the following processing is performed: load evaluation is performed on the device load to establish a load rating, and the load evaluation includes real-time load evaluation and predicted load evaluation; a resource release mapping of the load rating is established, and release constraints are generated through the resource release mapping and load rating; cache data management is performed based on the release constraints, the reward results, and sequential sorting results.

[0010] In a possible implementation, cache data management is performed based on the release constraints, the reward results, and the sequential sorting results, and the following processing is also performed: the real-time load evaluation and the predicted load evaluation are input into the response decision model to generate a response decision node; fuzzy decision is performed through the response decision node, the release constraints, the reward results, and the sequential sorting results, and cache data management is performed based on the fuzzy decision results.

[0011] In a possible implementation, the access prediction of cached data is performed through historical data to establish an access prediction result, and the following processing is also performed: timing analysis is performed on the historical data to establish a timing identifier; an adaptive timing window is established through the timing identifier, and a long and short timing analysis of the historical data is performed based on the adaptive timing window, and an access prediction result is established according to the long and short timing analysis results.

[0012] In a possible implementation, the cache data management method based on distributed encrypted storage also performs the following processing: establishing a synchronization verification mechanism, and performing consistency verification on the distributed encrypted storage data and the cache data through the synchronization verification mechanism; determining whether the consistency verification result is within a preset deviation threshold; if the consistency verification result is within the preset deviation threshold, directly updating the cache data.

[0013] In a possible implementation, the judgment of whether the consistency verification result is within the preset deviation threshold also performs the following processing: if the consistency verification result is not within the preset deviation threshold, an automatic anomaly detection instruction is generated; data verification of the corresponding distributed encrypted storage data is performed through the automatic anomaly detection instruction to generate a first verification result; data anomalies of the corresponding distributed encrypted storage data are reported, and the data anomalies are fed back to the user, and a second verification result is generated based on the user's feedback data; data update management of the cached data is performed based on the first verification result and the second verification result.

[0014] The present application also provides a cache data management and control system based on distributed encrypted storage, including: a distributed encrypted storage data acquisition module, the distributed encrypted storage data acquisition module is used to establish digital communication with a storage device and acquire distributed encrypted storage data of the storage device; a data importance evaluation result establishment module, the data importance evaluation result establishment module is used to call a user's evaluation database, perform data importance evaluation on the distributed encrypted storage data through the evaluation database, establish a data importance evaluation result, and map the data importance evaluation result to the cache data; a cache data feature acquisition module, the cache data feature acquisition module is used to acquire data features of the cache data, the data The features include data size features, access frequency features, and last access time features; a sequential sorting result generation module, which is used to generate sequential sorting results of cached data based on the data importance evaluation results and data features; a reward result establishment module, which is used to predict access to cached data through historical data, establish access prediction results, establish a reward network based on the access prediction results, verify access to cached data through the reward network, and establish reward results; a cache data management and control module, which is used to obtain the device load of the cache device, and manage the cache data based on the device load, reward results, and sequential sorting results.

[0015] Through the cache data management method and system based on distributed encrypted storage proposed in this application, it is intended to obtain the distributed encrypted storage data of the storage device; establish the data importance evaluation result; obtain the data characteristics of the cache data; generate the sequential sorting result of the cache data; predict the access of the cache data through historical data, establish a reward network based on the access prediction result, verify the access of the cache data through the reward network, and establish the reward result; obtain the device load of the cache device, and manage the cache data based on the device load, reward result, and sequential sorting result. It solves the technical problems of insufficient privacy protection of cache data in existing cache data management, cache management ignoring the importance of the data itself and future access trends, which leads to low data cache hit rate and access efficiency and low cache data security, realizes refined dynamic regulation of cache data, and achieves the technical effect of improving the efficiency and security of data cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the accompanying drawings of the embodiment of the present disclosure will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0017] Figure 1 A schematic flow chart of a cache data management and control method based on distributed encrypted storage provided in an embodiment of the present application.

[0018] Figure 2 A schematic diagram of the structure of a cache data management and control system based on distributed encrypted storage provided in an embodiment of the present application.

[0019] Explanation of the accompanying drawings: distributed encrypted storage data acquisition module 10, data importance evaluation result establishment module 20, cache data feature acquisition module 30, sequence sorting result generation module 40, reward result establishment module 50, cache data management and control module 60. DETAILED DESCRIPTION

[0020] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0021] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0022] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0023] The present application embodiment provides a cache data management method based on distributed encrypted storage, such as Figure 1 As shown, the method includes:

[0024] Step S100, establish digital communication with the storage device, and obtain the distributed encrypted storage data of the storage device. Digital communication refers to the method of transmitting information using digital signals as carriers. Establishing digital communication with the storage device refers to the communication link established between the system (digital device) and the storage device, which is used to transmit control instructions and data to ensure accurate and fast transmission of cached data. Specifically, the system and the storage device are connected through network cables, optical fibers, USB interfaces, wireless networks, etc. Both parties need to follow certain communication protocols (such as TCP / IP, HTTP, FTP, etc.) to ensure the correct transmission and analysis of data. Obtain distributed encrypted storage data of a storage device, wherein distributed storage is a data storage technology that stores data in a dispersed manner on multiple independent storage nodes and forms a virtual storage system through network connections, thereby improving data reliability and accessibility and reducing the risk of single point failures; encrypted storage refers to encrypting data before writing it to a storage device to protect the confidentiality and integrity of the data. Only users with corresponding decryption keys can access and decrypt the data. Specifically, through identity authentication and authorization processes, access to encrypted data in a distributed storage system is obtained, and a data request is sent to the storage device through a digital communication link to specify the data set to be obtained. After receiving the request, the storage device will decrypt the specified encrypted data according to the access request authority and encryption policy, and transmit the decrypted data to the system through the digital communication link. After receiving the data, the integrity and authenticity of the data can be verified through hash verification, digital signature, etc., and then subsequent storage, processing or analysis can be performed.

[0025] Step S200, calling the user's evaluation database, evaluating the importance of the distributed encrypted storage data through the evaluation database, establishing the data importance evaluation result, and mapping the data importance evaluation result to the cached data. Calling the user's evaluation database specifically refers to calling the data related to the user evaluation in the evaluation database through the database query interface or API, wherein the evaluation database contains the user's evaluation information on the stored data, and these evaluations may be based on multiple dimensions, such as the business value, sensitivity, accuracy, etc. of the data; then, the distributed encrypted storage data is evaluated for data importance through the evaluation database combined with scenario-aware computing. Specifically, specific indicators for evaluating the importance of data are determined. These indicators can objectively reflect the value of data to users or businesses. For example, they can include the business impact of data, the risk level of data leakage, the uniqueness of data, etc. Scenario-aware computing refers to combining scenario-aware technology to dynamically adjust the evaluation indicators and weights of data importance according to the current environment, needs and situations or predicted future scenarios (such as changes in business needs, security threats, etc.). Scenario-aware computing can more accurately reflect the value changes of the data itself in different scenarios, using the data in the evaluation database and the results of scenario-aware computing. , evaluate the importance of data and establish the data importance evaluation result, that is, output the data importance result obtained in the evaluation process in an appropriate form, such as scoring, ranking or sorting; then formulate corresponding cache strategies based on the data importance evaluation result. For example, for data with higher importance, it can be stored in the cache first to ensure fast access and efficient use of the data. Specifically, map the data importance evaluation result with the cache data, that is, use the evaluation result as the basis of the cache strategy to guide the selection, sorting and updating of cache data. Among them, cache data refers to data temporarily stored in memory or fast storage devices in order to improve data access speed. In a distributed encrypted storage environment, cache data may include frequently accessed encrypted data blocks or decrypted data fragments; finally, based on the evaluation results and the execution effect of the cache strategy, continuously optimize the cache strategy and data mapping process. For example, the data importance can be re-evaluated regularly, the cache capacity and update frequency can be adjusted, etc., to improve the efficiency and accuracy of the cache.

[0026] Step S300, obtain data features of cached data, the data features include data size features, access frequency features, and last access time features. Obtaining data features of cached data refers to collecting and analyzing the properties or characteristics of data items stored in the cache in order to better understand and manage cache resources, help evaluate the value of cached data, and provide important basis for cache optimization, replacement strategy, performance tuning, etc. Data features include data size features, access frequency features, and last access time features. Specifically, data size features refer to the size of storage space occupied by each data item in the cache. In cache management, data size is directly related to the capacity utilization of the cache and the effectiveness of the replacement strategy. For example, if a cache item is very large, but it is not frequently accessed, it may occupy a large amount of cache space, thereby reducing the overall efficiency of the cache. Based on the data size features, the cache system can implement more sophisticated cache replacement strategies, such as giving priority to replacing data items that are large in size but have low access frequencies; access frequency features refer to the number of times or frequency that each data item in the cache is accessed. Access frequency features are one of the key indicators for evaluating the importance of data items. Generally, the higher the access frequency, the better the cache replacement strategy. The more important a data item is considered, the greater its impact on system performance. The cache system can use access frequency characteristics to optimize cache replacement strategies, such as implementing algorithms such as Least Recently Used (LRU) or Least Frequently Used (LFU) to ensure that data items with high access frequency can remain in the cache for a long time; the last access time feature refers to the timestamp of the last access of each data item in the cache. The last access time feature is very important for implementing time-based cache replacement strategies. For example, in the LRU algorithm, the cache system will sort the data items according to their last access time and give priority to replacing those data items that have not been accessed for a long time. In addition, by monitoring the last access time of data items, the cache system can also predict which data items may be accessed in the future, thereby loading them into the cache in advance to improve the hit rate; by obtaining and analyzing data size characteristics, access frequency characteristics, and last access time characteristics, the cache system can implement more sophisticated cache replacement strategies and optimize the use of cache resources, thereby improving cache efficiency and hit rate.

[0027] Step S400, generating a sequential sorting result of cached data according to the data importance evaluation result and data features. Combining the data importance evaluation (such as business value, sensitivity, etc.) and specific data features (including data size, access frequency, and last access time) to determine the storage order or priority of data items in the cache helps to optimize the use of the cache and ensure that the most important data can be stored in the cache first, thereby improving the cache hit rate. Specifically, different weights are assigned to the data importance evaluation results and data features, and a weighted calculation of the comprehensive scoring result is performed. Then, the order of the cached data is generated based on the comprehensive scoring result, and the weight can be dynamically adjusted according to business needs and historical data; a priority queue is used to manage cached data items, and the priority of each data item is determined according to the comprehensive evaluation results. Data items with high priority are placed at the front of the queue and are accessed and retained in the cache first. Through the implementation and optimization of reasonable sorting algorithms or strategies, the efficient operation of the cache system and the effective use of resources are ensured.

[0028] Step S500, predicting access to cached data through historical data, establishing access prediction results, establishing a reward network based on the access prediction results, verifying access to cached data through the reward network, and establishing reward results. Collect historical access records of cache data, including access time, access frequency, data size, data type and other information, clean the historical data, remove noise data and outliers, ensure data accuracy and consistency, extract features related to access prediction from the cleaned data, such as time pattern (such as access peak period, trough period), access frequency distribution, data size distribution, etc., and then build a suitable prediction model based on the characteristics of the data and access prediction requirements. For example, build an access prediction model based on a time series analysis model or a machine learning model (such as a neural network, decision tree, random forest, etc.), use the preprocessed historical data to train the selected model, that is, use historical data as model input and access prediction related features as model output for training, adjust the model parameters during the training process to optimize the prediction performance, and then use the trained access prediction model to predict future cache data access and establish access prediction results, such as access time, access frequency, access data type or specific data item, etc., infer future access patterns based on access data characteristics, and use them to guide the formulation and optimization of cache strategies, such as determining which data should be stored in the cache first and when to update the cache. A reward network is established based on the access prediction results. Specifically, the primary task of the reward network is to clarify the reward target, that is, to optimize the performance of the cache system, improve the cache hit rate and reduce the access delay. Then, based on the access prediction results, a reasonable reward mechanism is designed. Based on multiple factors, such as cache hit rate, access delay, system load, etc., a comprehensive reward score is obtained through weighted calculation. For cache strategies or data items with high hit rates, higher rewards can be given, and for accesses that cause cache misses or high delays, lower rewards can be given to ensure that the cache system can be motivated to adopt more effective cache strategies, such as caching data with high access frequency first, dynamically adjusting cache capacity, etc. The reward network may include multiple components, such as prediction. The prediction model, decision engine, reward calculator and feedback mechanism are used to distribute rewards according to the access prediction results and the actual performance of the cache system, so as to realize continuous monitoring and optimization of the cache system performance. That is, the access prediction results are integrated into the reward network as input, and the access verification of cache data is performed to establish the reward results. Specifically, the accuracy of the access prediction results is verified by actually running the cache system, and the actual access of the cache data is recorded and compared with the prediction results. The reward results are generated according to the results of the access verification and the defined reward mechanism. For example, corresponding rewards are given to the prediction results and cache strategies with excellent performance, and adjustments and optimizations are made to the prediction results and cache strategies with poor performance.

[0029] In a possible implementation, step S500 further includes rewarding the network formula as follows:

[0030] ;

[0031] in, Characterization time Down state Take Action The reward function is To monitor the time window length, represents the time step, Characterization in time step The discount factor, The weight coefficient that characterizes cache hits, Characterization in time step Next data item Cache hits, Characterization in time step Next data item The predicted visit frequency, The weight coefficient that characterizes cache misses, Characterization in time step Next data item Cache misses, The weight coefficient that characterizes the use of cache resources, Representation in state Take Action Cache resource usage, Characterization in time step The cache collection under. In the reward function, Indicates that in the time window The weighted sum of all factors is calculated, and the discount factor It balances current and future rewards. Specifically, it gradually reduces the impact of the future and gives higher weight to recent events. Characterize the discount factor, ,generally, Values ​​close to 1 indicate a high importance of future rewards; A lower value indicates a higher importance of the current reward.

[0032] In a possible implementation, step S500 further includes step S510, performing time series analysis on the historical data and establishing a time series identifier. Time series analysis refers to analyzing the historical data in chronological order, identifying the time dimension and related cache data in the data, converting the original historical data into a data set with clear time sequence and structural characteristics, and then establishing a unique time series identifier for each data point or data segment based on the time series analysis, which usually includes information such as timestamp, metric name, label, etc., to uniquely identify each data point or data segment in the data set.

[0033] It also includes step S520, establishing an adaptive timing window through the timing identifier, performing long and short timing analysis of historical data based on the adaptive timing window, and establishing access prediction results based on the long and short timing analysis results. Establishing an adaptive timing window through a timing identifier specifically refers to dynamically determining the size and position of the adaptive timing window based on information such as the timestamp and metric value in the timing identifier, combined with analysis requirements (such as prediction accuracy, response time, etc.), wherein an adaptive timing window is a time window that dynamically adjusts its size and position based on data characteristics and analysis requirements. Compared with a fixed-size time window, an adaptive timing window can more flexibly capture trends and changes in the data, thereby improving the accuracy and efficiency of the analysis; then, based on the adaptive timing window, long and short timing analysis of historical data is performed, that is, on the basis of the adaptive timing window, long and short timing analysis of historical data is performed, through Compare data features at different time scales, such as time series decomposition and trend forecasting. Long and short time series analysis refers to the analysis of historical data at different time scales. Long time series analysis focuses on trends and cyclical changes in data over a longer time frame, while short time series analysis focuses more on fluctuations and anomalies in data over a shorter time frame. According to the results of long and short time series analysis, use appropriate forecasting models (such as ARIMA models, exponential smoothing models, etc.) to forecast future data, that is, use the constructed forecasting model to forecast the number of visits at future time points. The forecast results include the number of visits trend and peak forecast in the future period, that is, the visit forecast results are obtained.

[0034] Step S600, obtain the device load of the cache device, and perform cache data management and control according to the device load, reward results, and sequential sorting results. The load data of the cache device is obtained in real time through the monitoring system of the cache device, wherein the device load refers to the operating pressure and load of the cache device at the current time point or within a period of time, which usually includes multiple aspects such as the amount of data processed, the number of requests, and the response time, such as key indicators such as CPU usage, memory occupancy, disk I / O speed, and network bandwidth. Then, the device load, reward results, and sequential sorting results are comprehensively considered to formulate a suitable cache data management and control strategy, such as giving priority to caching data with high access frequency, replacing data with low access frequency, adjusting the cache size to cope with changes in device load, etc. Cache data management and control are performed according to the cache data management and control strategy, and the performance changes of the cache system are monitored in real time. The necessary adjustments and optimizations to the management and control strategy are made according to the monitoring results, which helps to optimize the performance and resource utilization of the cache system, improve the cache hit rate, and reduce unnecessary access delays, so as to ensure the stability and efficiency of the cache system.

[0035] In a possible implementation, step S600 further includes step S610, performing load evaluation on the device load and establishing a load rating, wherein the load evaluation includes real-time load evaluation and predicted load evaluation. The purpose of load evaluation on the device load is to evaluate the load status of the device at different time points and its ability to predict future loads, and mainly includes real-time load evaluation and predicted load evaluation. Specifically, real-time load evaluation refers to the evaluation of the actual load status of the device at the current time point or time period, that is, real-time monitoring of the current load of the device, that is, the workload or pressure (data volume, number of processing times) borne by the device within a specific time, including CPU utilization, memory utilization, I / O operations, etc., performing weighted averaging on the real-time load data, calculating the current load level of the device, and understanding the current operating status of the device, including whether it is overloaded, underloaded or normal; predicting load Evaluation refers to the prediction and assessment of the load condition of the device in the future. Prediction models are built based on neural networks, decision trees, etc., and historical data is used for training to obtain load prediction models. According to historical data and current trends, the load condition of the cache device in the future is predicted to provide decision support for future cache data management and control, and plan resource allocation and caching strategies in advance. Load rating is to conduct a graded evaluation of the load condition of the cache device based on the results of real-time load evaluation and predicted load evaluation, and set multiple rating levels, such as low load, medium load, high load, etc. Through load rating, the load condition of the cache device can be more intuitively understood.

[0036] The method further includes step S620, establishing a resource release mapping of a load rating, and generating a release constraint through the resource release mapping and the load rating. The resource release mapping is a strategy that defines how to release system resources under different load ratings, which may include cache space, computing resources, storage resources, etc. For example, under high load conditions, some low-priority cache data may need to be released to reduce the device load. After the load rating and resource release mapping are established, a release constraint is generated according to the current load rating and resource release mapping. The release constraint refers to the specific requirements and restrictions on resource release under a specific load rating, which may include the type, quantity, time, etc. of released resources. Through the release constraint, it is ensured that the stability and performance of the system will not be damaged when releasing resources, while meeting the needs of users.

[0037] The system also includes step S630, which performs cache data management and control according to the release constraint, the reward result and the sequential sorting result. The cache data management and control is performed according to the release constraint, the reward result and the sequential sorting result. Specifically, according to the reward result, data with high cache hit rate, short response time and significant improvement on system performance is retained first. For high-value data, its cache time is appropriately increased or cache space is allocated first to ensure that when the device load is too high, the cache data can be released according to the predetermined strategy to relieve the load pressure; according to the actual effect of the cache strategy (such as cache hit rate, response time, etc.) and the reward mechanism, the effective cache strategy is rewarded to encourage it to continue to optimize, and the strategy with poor effect is adjusted or replaced; the cache data is sorted according to factors such as the access frequency and importance of the data, and data with high access frequency and importance is retained first, and data with low access frequency and importance is released; the performance indicators (such as cache hit rate, response time, etc.) and load status of the cache system are monitored in real time, and the cache data management and control strategy is adjusted in time according to the monitoring results to cope with changes in system load and changes in data access mode.

[0038] In a possible implementation, step S630 further includes step S631, inputting the real-time load evaluation and the predicted load evaluation into the response decision model to generate a response decision node. The response decision model is an evaluation model constructed based on a machine learning model (such as a neural network, a random forest, etc.), and is used to generate corresponding response decision nodes according to the input real-time load evaluation and the predicted load evaluation. The decision node represents the response measures or strategies that the cache device should take under different load conditions, such as increasing cache space, replacing low-value data, adjusting cache strategies, etc. The results of the real-time load evaluation and the predicted load evaluation are used as input and passed to the response decision model. The response decision model generates multiple response decision nodes based on the input load evaluation results and combined with preset algorithms and rules.

[0039] It also includes step S632, making fuzzy decisions through the response decision node, release constraints, the reward results and the order sorting results, and performing cache data management and control according to the fuzzy decision results. After obtaining the response decision node, a fuzzy decision is made in combination with the release constraints, reward results and order sorting results. Specifically, relevant data of the cache system, such as cache hit rate, cache space usage, data access mode, etc., are obtained, and the collected precise data is converted into a fuzzy set. According to the characteristics of the cache system and business needs, a fuzzy rule base is formulated, including decision conditions, decision results and corresponding fuzzy membership functions, etc. The fuzzified data is inferred using the fuzzy rule base to obtain multiple possible cache decision schemes and their corresponding fuzzy memberships, and then defuzzification is performed by selecting the maximum membership principle to convert the results of fuzzy reasoning into precise decision results. Finally, according to the defuzzified decision results, corresponding management and control operations are performed on the cache data, such as cache replacement, cache prefetching, etc.

[0040] In a possible implementation, step S600 further includes step S640, establishing a synchronization verification mechanism, and performing consistency verification of distributed encrypted storage data and cached data through the synchronization verification mechanism. A synchronization verification mechanism is established to perform consistency verification of distributed encrypted storage data and cached data, wherein the synchronization verification mechanism is a mechanism for checking data consistency between different data sources (here, distributed encrypted storage and cache) in a distributed system, ensuring that the representation and state of data remain consistent even in different data storage locations. Specifically, the synchronization verification mechanism needs to have a trigger point or condition, and when these conditions are met, the verification process will be started, such as setting regular verification or after data update; retrieve the corresponding data to be verified from the distributed encrypted storage and cache, and select a suitable verification algorithm to efficiently compare the data and accurately identify any inconsistencies, such as comparing the hash value, timestamp, etc. of the data, evaluate the comparison results, and determine whether the data is consistent.

[0041] It also includes step S650, judging whether the consistency verification result is within the preset deviation threshold. The preset deviation threshold is a deviation range set according to business needs and data characteristics (data sensitive), system fault tolerance, etc., which is used to judge the severity of data inconsistency, compare the analysis result with the preset deviation threshold, and judge whether the consistency verification result is within the preset deviation threshold. It also includes step S660, if the consistency verification result is within the preset deviation threshold, the cache data is directly updated. If all differences are within the threshold range, it is considered that the data consistency verification has passed, and the cache data can be directly updated. Specifically, the cache update strategy is optimized according to factors such as business needs and data update frequency. For example, a lazy loading strategy is used to reduce unnecessary cache update operations; or a write-after-update strategy is used to ensure that data is synchronized to the cache in a timely manner, and the cache update operation is monitored to ensure its correctness and performance.

[0042] In a possible implementation, step S650 further includes step S651, if the consistency verification result is not within the preset deviation threshold, an automatic anomaly detection instruction is generated. If the consistency verification result is not within the preset deviation threshold, an anomaly detection instruction is automatically generated to trigger subsequent data verification, problem diagnosis, anomaly reporting and other processes. It also includes step S652, performing data verification of the corresponding distributed encrypted storage data through the automatic anomaly detection instruction to generate a first verification result. Through the automatic anomaly detection instruction, the corresponding distributed encrypted storage data is verified, which may include data decryption, verifying the integrity of the data, ensuring that the data is not damaged during storage and transmission, and comparing the decrypted data with other reliable data sources, historical data, etc. to confirm the accuracy of the data, and generating a first verification result, indicating whether there is a problem with the distributed encrypted storage data itself.

[0043] It also includes step S653, reporting the data anomaly corresponding to the distributed encrypted storage data, feeding back the data anomaly to the user, and generating a second verification result based on the user's feedback data. If the first verification result shows that the distributed encrypted storage data does have an anomaly (such as data corruption, inconsistency, etc.), the anomaly information is reported to the user. After receiving the data anomaly report, the user may provide further information, explanations or correction suggestions. The system will perform further verification analysis based on the user's feedback to generate a second verification result. It also includes step S654, performing data update management of cached data based on the first verification result and the second verification result. The update of cached data is managed based on the first verification result and the second verification result. For example, if there is a problem with the distributed encrypted storage data, a repair plan is determined to repair the abnormal data, and the repaired data is resynchronized to the cache; if the cached data is outdated or invalid, the system chooses to refresh the cache, that is, reload the latest data from the distributed encrypted storage; if the data problem cannot be solved or the cached data is no longer needed, the system marks the cache item as invalid or deletes it to update the cached data.

[0044] In the above, refer to Figure 1 The cache data management method based on distributed encrypted storage according to an embodiment of the present invention is described in detail. Figure 2 A cache data management and control system based on distributed encrypted storage according to an embodiment of the present invention is described.

[0045] The cache data management and control system based on distributed encrypted storage according to the embodiment of the present invention is used to solve the technical problems of insufficient cache data privacy protection in existing cache data management, cache management ignoring the importance of the data itself and future access trends, which leads to low data cache hit rate and access efficiency and low cache data security, and realizes refined dynamic regulation of cache data, achieving the technical effect of improving the efficiency and security of data cache. The cache data management and control system based on distributed encrypted storage includes: a distributed encrypted storage data acquisition module 10, a data importance evaluation result establishment module 20, a cache data feature acquisition module 30, a sequential sorting result generation module 40, a reward result establishment module 50, and a cache data management and control module 60.

[0046] A distributed encrypted storage data acquisition module 10, wherein the distributed encrypted storage data acquisition module 10 is used to establish digital communication with a storage device and acquire distributed encrypted storage data of the storage device;

[0047] A data importance evaluation result establishment module 20, wherein the data importance evaluation result establishment module 20 is used to call a user's evaluation database, perform data importance evaluation on the distributed encrypted storage data through the evaluation database, establish a data importance evaluation result, and map the data importance evaluation result to cache data;

[0048] A cache data feature acquisition module 30, the cache data feature acquisition module 30 is used to acquire data features of cache data, the data features including data size features, access frequency features, and last access time features;

[0049] A sequence sorting result generating module 40, wherein the sequence sorting result generating module 40 is used to generate a sequence sorting result of cache data according to the data importance evaluation result and data characteristics;

[0050] A reward result establishment module 50, the reward result establishment module 50 is used to predict the access of cached data through historical data, establish the access prediction result, establish a reward network according to the access prediction result, perform access verification of cached data through the reward network, and establish the reward result;

[0051] The cache data control module 60 is used to obtain the device load of the cache device and perform cache data control according to the device load, reward results, and sequence sorting results.

[0052] The specific configuration of the reward result establishment module 50 will be described in detail below. The reward result establishment module 50 further includes: the reward network formula is as follows:

[0053] ;

[0054] in, Characterization time Down state Take Action The reward function is To monitor the time window length, represents the time step, Characterization in time step The discount factor, The weight coefficient that characterizes the cache hit, Characterization in time step Next data item Cache hits, Characterization in time step Next data item The predicted visit frequency, The weight coefficient that characterizes cache misses, Characterization in time step Next data item Cache misses, The weight coefficient that characterizes the use of cache resources, Representation in state Take Action Cache resource usage, Characterization in time step The cache collection below.

[0055] The specific configuration of the cache data control module 60 will be described in detail below. The cache data control module 60 may further include: performing load evaluation on the device load and establishing a load rating, wherein the load evaluation includes a real-time load evaluation and a predicted load evaluation; establishing a resource release mapping of the load rating, and generating a release constraint through the resource release mapping and the load rating; and performing cache data control according to the release constraint, the reward result, and the sequence sorting result.

[0056] The specific configuration of the cache data control module 60 will be described in detail below. The cache data control module 60 may further include: inputting the real-time load evaluation and the predicted load evaluation into the response decision model to generate a response decision node; performing fuzzy decision through the response decision node, the release constraint, the reward result and the sequence ranking result, and performing cache data control according to the fuzzy decision result.

[0057] The specific configuration of the reward result establishment module 50 will be described in detail below. The reward result establishment module 50 further includes: performing time series analysis on the historical data to establish a time series identifier; establishing an adaptive time series window through the time series identifier, performing long and short time series analysis on the historical data based on the adaptive time series window, and establishing a visit prediction result according to the long and short time series analysis result.

[0058] The specific configuration of the cache data control module 60 will be described in detail below. The cache data control module 60 may further include: establishing a synchronization verification mechanism, performing consistency verification of the distributed encrypted storage data and the cache data through the synchronization verification mechanism; determining whether the consistency verification result is within a preset deviation threshold; if the consistency verification result is within the preset deviation threshold, directly updating the cache data.

[0059] The specific configuration of the cache data control module 60 will be described in detail below. The cache data control module 60 may further include: if the consistency verification result is not within the preset deviation threshold, an automatic anomaly detection instruction is generated; data verification of the corresponding distributed encrypted storage data is performed through the automatic anomaly detection instruction to generate a first verification result; a data anomaly of the corresponding distributed encrypted storage data is reported, the data anomaly is fed back to the user, and a second verification result is generated according to the user's feedback data; and data update management of the cache data is performed according to the first verification result and the second verification result.

[0060] The cache data management and control system based on distributed encrypted storage provided in the embodiment of the present invention can execute the cache data management and control method based on distributed encrypted storage provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0061] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0062] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A cache data management and control method based on distributed encrypted storage, characterized in that: The method comprises: Establishing digital communication with the storage device to obtain distributed encrypted storage data of the storage device; Calling a user's evaluation database, performing data importance evaluation on the distributed encrypted storage data through the evaluation database, establishing a data importance evaluation result, and mapping the data importance evaluation result to cache data; Acquire data features of cached data, the data features including data size features, access frequency features, and last access time features; Generate a sequential sorting result of cached data according to the data importance evaluation result and data characteristics; Predicting access to cached data using historical data to establish access prediction results, establishing a reward network based on the access prediction results, verifying access to cached data using the reward network, and establishing reward results; The device load of the cache device is obtained, and cache data is managed and controlled according to the device load, reward results, and sequence sorting results.

2. The cache data management method based on distributed encrypted storage according to claim 1, characterized in that: The step of establishing a reward network according to the visit prediction result further includes: The reward network formula is as follows: ； in, Characterization time Down state Take Action The reward function is To monitor the time window length, represents the time step, Characterization in time step The discount factor, The weight coefficient that characterizes the cache hit, Characterization in time step Next data item Cache hits, Characterization in time step Next data item The predicted visit frequency, The weight coefficient that characterizes cache misses, Characterization in time step Next data item Cache misses, The weight coefficient that characterizes the use of cache resources, Representation in state Take Action Cache resource usage, Characterization in time step The cache collection below.

3. The cache data management method based on distributed encrypted storage according to claim 1, characterized in that: The cache data management and control according to the device load, reward result, and sequence sorting result also includes: Performing load evaluation on the device load and establishing a load rating, wherein the load evaluation includes real-time load evaluation and predicted load evaluation; Establishing a resource release mapping of the load rating, and generating a release constraint through the resource release mapping and the load rating; Cache data management is performed according to the release constraints, the reward results and the sequential sorting results.

4. The cache data management method based on distributed encrypted storage according to claim 3, characterized in that: The method further comprises: Input the real-time load evaluation and the predicted load evaluation into the response decision model to generate a response decision node; A fuzzy decision is made through the response decision node, the release constraint, the reward result and the sequential sorting result, and cache data management and control is performed according to the fuzzy decision result.

5. The cache data management method based on distributed encrypted storage according to claim 1, characterized in that: The predicting access to cached data using historical data and establishing access prediction results also includes: Performing time series analysis on the historical data and establishing time series identification; An adaptive timing window is established through the timing identifier, a long-short timing analysis of historical data is performed based on the adaptive timing window, and an access prediction result is established according to the long-short timing analysis result.

6. The cache data management method based on distributed encrypted storage according to claim 1, characterized in that: The method further comprises: Establishing a synchronization verification mechanism to verify the consistency of distributed encrypted storage data and cache data; Determine whether the consistency verification result is within the preset deviation threshold; If the consistency verification result is within the preset deviation threshold, the cache data is directly updated.

7. The cache data management method based on distributed encrypted storage according to claim 6, characterized in that: The step of determining whether the consistency verification result is within a preset deviation threshold further includes: If the consistency verification result is not within the preset deviation threshold, an automatic anomaly detection instruction is generated; Performing data verification of the corresponding distributed encrypted storage data through the automatic anomaly detection instruction to generate a first verification result; Reporting a data anomaly corresponding to the distributed encrypted storage data, feeding back the data anomaly to the user, and generating a second verification result according to the user's feedback data; Data update management of cache data is performed according to the first verification result and the second verification result.

8. A cache data management and control system based on distributed encrypted storage, characterized in that: The system is used to implement the cache data management method based on distributed encrypted storage according to any one of claims 1 to 7, and the system includes: A distributed encrypted storage data acquisition module, the distributed encrypted storage data acquisition module is used to establish digital communication with the storage device and acquire the distributed encrypted storage data of the storage device; A data importance evaluation result establishment module, the data importance evaluation result establishment module is used to call the user's evaluation database, perform data importance evaluation on the distributed encrypted storage data through the evaluation database, establish the data importance evaluation result, and map the data importance evaluation result to the cache data; A cache data feature acquisition module, the cache data feature acquisition module is used to acquire data features of cache data, the data features include data size features, access frequency features, and last access time features; A sequential sorting result generating module, wherein the sequential sorting result generating module is used to generate a sequential sorting result of the cached data according to the data importance evaluation result and the data feature; A reward result establishment module, the reward result establishment module is used to predict access to cached data through historical data, establish access prediction results, establish a reward network based on the access prediction results, perform access verification of cached data through the reward network, and establish reward results; A cache data control module is used to obtain the device load of the cache device and perform cache data control according to the device load, reward results, and sequence sorting results.

Images