Computer big data storage control system

Abstract

The invention provides a computer big data storage control system, and relates to the technical field of computers, and the system comprises a data evaluation module which is used for comprehensively calculating the real-time popularity of data, and the real-time popularity is obtained through the comprehensive calculation of the data access frequency and the access duration; performing quantitative evaluation on the real-time popularity of the data in combination with a multi-dimensional factor to obtain a value evaluation value of the data; analyzing the system resource information of the storage device on the computer to obtain a resource analysis result; and the intelligent contract execution module is used for carrying out comprehensive analysis and judgment on the real-time popularity, the value evaluation value and the system resource analysis result of the data so as to trigger adjustment of a storage strategy. According to the method, the real-time popularity model is constructed, the analytic hierarchy process is combined to achieve multi-dimensional value evaluation, the intelligent contract dynamically adjusts the storage strategy according to the evaluation result and the resource state, the data value and the storage medium are accurately matched, the problems that a traditional system is extensive in evaluation and rigid in strategy are solved, and efficient and accurate scheduling of storage resources is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a computer big data storage control system. Background Art

[0002] With the rapid development of information technology, the data volume in various industries has increased exponentially, and the complexity of data types and business scenarios has also been continuously improved, posing severe challenges to the intelligence, efficiency, and security of big data storage control systems. Traditional storage systems rely on single-dimensional data evaluation and static storage strategies, making it difficult to adapt to the dynamic changes in data value and diverse business needs, resulting in problems such as unreasonable storage resource allocation, high access latency for hot data, and imbalance between storage cost and performance. At the same time, in fields such as finance and healthcare where strict requirements are imposed on data security and compliance, traditional systems lack a full-link operation auditing mechanism, and it is difficult to trace key information such as changes in data storage locations and access histories, presenting problems of operation opacity and security risks.

[0003] There are significant deficiencies in the existing technology in data value evaluation: only judging data heat through a single index such as access frequency, ignoring core dimensions such as business importance and timeliness, resulting in high-value low-frequency data being misfiled to low-speed storage media, affecting the response efficiency of key services; the adjustment of storage strategies depends on manually preset rules and cannot real-time perceive the impact of system resource status and business events, easily leading to local node overload or resource waste. Therefore, it is necessary to provide a computer big data storage control system to solve the above technical problems. Summary of the Invention

[0004] The present invention provides a computer big data storage control system, which solves the problems raised in the above background art.

[0005] To solve the above technical problems, a computer big data storage control system provided by the present invention includes: A blockchain ledger module, configured to use blockchain technology to construct a storage management ledger, and record the storage location, access history, and adjustment records of storage strategies for each piece of data in the storage management ledger; A data evaluation module, configured to comprehensively calculate the real-time heat of data, where the real-time heat is obtained by comprehensively calculating the data access frequency and access duration; combining the real-time heat of data with multi-dimensional factors for quantitative evaluation to obtain a data value evaluation value; and then analyzing the system resource information of the storage device on the computer to obtain a resource analysis result, where the resource analysis result includes sufficient resources, resource tension, and moderate resources; A storage setting module, configured to identify the current storage device, including at least a local high-speed memory, a local low-speed memory, and a cloud storage area; The intelligent contract execution module is used to comprehensively analyze and judge the real-time popularity, value evaluation value, and system resource analysis results of data to trigger the adjustment of the storage strategy.

[0006] Preferably, the specific steps for comprehensively calculating the real-time popularity of the data are as follows: Set the statistical period T for the data access frequency, and count the number of accesses n of the data within this statistical period as the access frequency ; at the same time, record the duration of each data access , , calculate the average access duration ; set the preset frequency weight and duration weight , through the formula , calculate the basic popularity value of the data within the statistical period ; Identify the category C of the data, and calculate the basic popularity values of all data of the same category within the statistical period , where j represents the jth data among the data of the same category; calculate the category popularity value by calculating all the basic popularity values of the data of the same category , the formula is , m represents the number of data of the same category, and J represents the number of data of the same category; Use the cosine similarity algorithm to calculate the similarity corresponding to the data category, and denote the category to which the data D belongs as , and other categories as , and k≠1, calculate and The cosine similarity between them, the formula is , where and are the data feature vectors of the category respectively; select the similarity threshold Sth for the cosine similarity, and set the similarity weight of the similar category according to the similarity , where the calculation formula of the similarity weight is ; Perform weighted calculation on the similarity weight corresponding to the similar category of the data and the category popularity value to obtain the similar popularity value , the formula is ; Perform normalization and weighted processing on the basic popularity value, category popularity value, and similar popularity value, and use the formula to calculate the real-time popularity ; where h1 + h2 + h3 = 1, and h1, h2, and h3 represent the weights corresponding to the basic popularity value, category popularity value, and similar popularity value respectively.

[0007] Preferably, the real-time popularity of data is combined with multi-dimensional factors for quantitative evaluation to obtain the value evaluation value of the data. The specific evaluation is as follows: Obtain multi-dimensional factors; determine the multi-dimensional factor set , where represents business importance, represents timeliness, represents the degree of support for decision-making; Use the analytic hierarchy process to construct a judgment matrix , represents factor and The ratio of relative importance, calculate the eigenvector of the judgment matrix , this vector is the weight vector of each dimension factor, satisfying , u = 1 ; Obtain the real-time popularity H of the data; for each dimension factor Quantitatively score according to the preset scoring criteria to obtain the score ; Through the formula Get the value evaluation value V of the data.

[0008] Preferably, analyzing the system resource information of the storage device on the analysis computer to obtain the resource analysis result, the specific steps are as follows: Obtain the system resource information of the storage device on the computer, including the remaining space, CPU usage rate, and memory load; Identify the required capacity of the data, and subtract the required capacity of the data from the remaining space of the storage device to obtain the pre-remaining capacity; Perform weighted calculation on the pre-remaining capacity, CPU usage rate, and memory load with their preset weights to obtain the resource value; match the resource value with the preset resource value range group to generate the resource analysis result; among them, the resource threshold group includes the resource sufficient, resource tight, and resource moderate value ranges, and the resource analysis result includes resource sufficient, resource tight, and resource moderate.

[0009] Preferably, comprehensively analyze and judge the real-time popularity of the data, the value evaluation value, and the system resource analysis result to trigger the adjustment of the storage strategy, specifically: Obtain the real-time popularity of the data in the statistical period within the set time range, and calculate the growth rate of the data in the statistical period within this time range; perform weighted calculation on the real-time popularity, growth rate, and value evaluation value to obtain the data evaluation value; set the high value evaluation threshold and low value evaluation threshold of the data; If the data evaluation value is greater than its high - value evaluation threshold and the resource analysis result is that resources are sufficient, a data migration instruction is triggered; the data migration instruction is used to migrate the target data from the cloud storage area or the local low - speed memory to the local high - speed memory; When the data evaluation value is less than its low - value evaluation threshold: When the resource analysis result is that resources are sufficient or medium, a data archiving instruction is triggered; the data archiving instruction is used to compress the target data and then migrate it to the local low - speed memory or the cloud storage area; When the resource analysis result is that resources are tight, a data deletion instruction is preferentially triggered; the data deletion instruction is used to remove the target data from the local high - speed memory and only retain the backup in the cloud storage area; Among them, the data migration instruction is executed prior to the data archiving instruction and the data deletion instruction; All storage policy adjustment operations are recorded in the blockchain ledger module, including the operation time, data identifier, source storage location, target storage location, and trigger conditions, forming an immutable operation log.

[0010] Preferably, the present invention further includes: A data trend analysis module configured to generate a data pre - fetch instruction through historical data modeling and real - time business event analysis; A data pre - fetch module configured to pre - fetch the target data in advance according to the pre - fetch instruction output by the data trend analysis module and temporarily store it in the cache area; An asynchronous transmission module configured to transmit the target data in the cache area to the specified storage device during a period when the system resource utilization rate is lower than the preset idle threshold.

[0011] Preferably, the specific steps for analyzing historical data and real - time business trends are: Collect a historical data set including at least the access timestamp t of the data, access frequency f, and business operation type O ; Clean the data set, removing invalid data with abnormal timestamps, negative or out - of - range access frequencies; Use a time - series analysis algorithm to model the historical access frequency data and fit a prediction function , where are model parameters; determine the model order through the autocorrelation function and partial autocorrelation function, and optimize the model parameters using the maximum likelihood estimation method; Obtain real - time business event data , where includes event type, influence range, and duration; where r represents the index of the business event and b represents the number of business events; For each business event Calculate the influence coefficient ; According to the prediction function and the influence coefficient , calculate the corrected predicted access frequency , and the formula is ; When the predicted access frequency is greater than or equal to the preset high-frequency access threshold, generate a prefetch instruction including the data identifier and the target storage device and send it to the data prefetch module.

[0012] Preferably, the present invention further includes a data monitoring module for real-time monitoring of the system resource information of the storage device in the computer; wherein the system resource information includes the remaining space, CPU utilization rate, and the load condition of the memory to obtain the real-time utilization rate of the system resources.

[0013] Preferably, for each business event calculate the influence coefficient , specifically:[[]] Classify the business events, and extract the key feature parameters of each business event. The key feature parameters at least include the event type identifier , event influence range , event duration , event associated data label set , ; where g represents the number of associated data; Collect historical business event data and construct a training data set , where is the feature vector of the v-th historical business event, is the growth rate of the target data access frequency caused by this event; Then adopt a multiple linear regression model , where is the z-th feature parameter of the r-th business event, is the feature parameter weight, and b1 is the model intercept term; For real-time business events , extract their feature parameters, substitute their feature parameters into the multiple linear regression model, and calculate the influence coefficient ; where when new historical business event data appears, retrain the multiple linear regression model and update the feature parameter weight and the intercept term b1.

[0014] Compared with the related technology, a computer big data storage control system provided by the present invention has the following beneficial effects: 1. The present invention constructs a real-time popularity model by integrating data access frequency, duration, and category correlation, and realizes multi-dimensional value evaluation in combination with the analytic hierarchy process. The smart contract dynamically adjusts the storage strategy according to the evaluation results and resource status, accurately matches the data value with the storage medium, solves the problems of rough evaluation and rigid strategy in the traditional system, and realizes the efficient and accurate scheduling of storage resources.

[0015] 2. The present invention records the data storage location, access history, and policy adjustment log through the blockchain ledger module to form an immutable operation audit trail, ensuring the transparency and traceability of data operations from a technical level and enhancing the system trustworthiness.

[0016] 3. The present invention combines historical data modeling and real-time business event impact evaluation through the data trend analysis module, predicts high-frequency access data in advance and triggers prefetch instructions, and asynchronously transfers them to the target storage device during the system idle period, effectively reducing the access latency of hot data and enhancing the system response ability in the case of sudden high load scenarios.

[0017] In summary, through the collaborative innovation of multi-dimensional data evaluation, blockchain audit, and intelligent scheduling, the present invention realizes the intelligent management of the entire life cycle of big data storage. Brief Description of the Drawings

[0018] Figure 1 It is a schematic block diagram of a computer big data storage control system provided by the present invention. Detailed Embodiments

[0019] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit this disclosure. The singular forms of "group", "class", and "the" used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0022] Please refer to Figure 1 together. A computer big data storage control system includes: A blockchain ledger module for constructing a storage management ledger using blockchain technology, and recording the storage location, access history, and adjustment records of storage policies for each piece of data in the storage management ledger; A data evaluation module for comprehensively calculating the real-time popularity of data, where the real-time popularity is obtained by comprehensively calculating the data access frequency and access duration; quantifying and evaluating the real-time popularity of data in combination with multi-dimensional factors to obtain a value evaluation value of the data; and then analyzing the system resource information of the storage device on the computer to obtain a resource analysis result, where the resource analysis result includes sufficient resources, tight resources, and moderate resources; A storage setting module for identifying the current storage device, including at least a local high-speed memory, a local low-speed memory, and a cloud storage area; An intelligent contract execution module for comprehensively analyzing and judging the real-time popularity, value evaluation value, and system resource analysis result of data to trigger an adjustment of the storage policy.

[0023] In the present invention, the specific steps for comprehensively calculating the real-time popularity of data are: Set a statistical period T for the data access frequency, and count the number of accesses n of the data within this statistical period as the access frequency ; at the same time, record the duration of each data access , , calculate the average access duration ; set a preset frequency weight and a duration weight , and through the formula , calculate the basic popularity value of the data within the statistical period ; Identify the category C of the data, and calculate the basic popularity values of all data of the same category within the statistical period , where j represents the jth data among the data of the same category; calculate the category popularity value by calculating the basic popularity values of all data of the same category , the formula is , m represents the number of data of the same category, and J represents the number of data of the same category; The cosine similarity algorithm is used to calculate the similarity corresponding to the data category. Denote the category to which the data D belongs as , and denote other categories as , and k≠1. Calculate the cosine similarity between and . The formula is , where and are the data feature vectors of the category respectively. The features may include the business type, usage scenario, etc. of the data, which are specifically set according to the actual situation; Select the similarity threshold Sth for the cosine similarity, and set the similarity weight of the similar category according to the similarity , where the calculation formula of the similarity weight is ; Perform weighted calculation on the similarity weight corresponding to the similar category of the data and the category heat value to obtain the similar heat value . The formula is ; Perform normalization and weighting processing on the basic heat value, category heat value and similar heat value, and use the formula to calculate and obtain the real-time heat ; where h1 + h2 + h3 = 1, and h1, h2, and h3 respectively represent the weights corresponding to the basic heat value, category heat value and similar heat value, which can be specifically set by those skilled in the art according to the actual situation.

[0024] It should be noted that this part calculates the real-time heat by integrating factors such as data access frequency, duration, and category association, avoiding the one-sidedness of single-factor evaluation, and comprehensively reflecting the actual heat situation of the data; The value of the data can be evaluated through the real-time heat; Combining multi-dimensional factors such as business importance and timeliness can more accurately evaluate the data value, providing a scientific basis for subsequent storage strategy adjustment; At the same time, it also provides key data for the intelligent contract execution module. According to the real-time heat and related evaluation results, the intelligent contract can judge the importance and usage frequency of the data, so as to reasonably adjust the storage strategy, such as storing high-frequency hot data in high-speed devices to improve data access efficiency.

[0025] In the present invention, the real-time heat of the data is combined with multi-dimensional factors for quantitative evaluation to obtain the value evaluation value of the data. The specific evaluation is as follows: Obtain multi-dimensional factors; Determine the multi-dimensional factor set , where represents business importance, represents timeliness, Indicates the degree of support for decision-making; in the prior art, business importance can be determined based on the criticality of data in the business process, the scope and depth of its impact on business operations. For example, the importance of core business data is higher than that of auxiliary data; timeliness can be measured by the data generation time, update cycle, and the time sensitivity of the business scenarios involved in the data; the degree of support for decision-making is judged according to the value of the information provided by the data in the decision-making process and the degree of influence on the decision. For example, data that can directly affect strategic decisions has a higher degree of support for decision-making. Since these factors can be obtained through the prior art, no further clarification is made on the acquisition method in this application; Construct a judgment matrix using the analytic hierarchy process , Indicates a factor and The ratio of relative importance, which can be determined by expert scoring or data analysis, and calculate the eigenvector of the judgment matrix , and this vector is the weight vector of each dimension factor, satisfying , u = 1 ; Obtain the real-time popularity H of the data; for each dimension factor Quantify and score according to the preset scoring criteria to obtain a score ; Through the formula Obtain the value evaluation value V of the data.

[0026] In the present invention, analyzing the system resource information of the storage device on the computer to obtain a resource analysis result, and the specific steps are as follows: Obtain the system resource information of the storage device on the computer, including the remaining space, CPU usage rate, and memory load; Identify the required capacity of the data, and subtract the required capacity of the data from the remaining space of the storage device to obtain the pre-remaining capacity; Perform weighted calculation on the pre-remaining capacity, CPU usage rate, and memory load with their preset weights to obtain a resource value; match the resource value with the preset resource value range group to generate a resource analysis result; among them, the resource threshold group includes the value ranges of sufficient resources, tight resources, and moderate resources, and the resource analysis result includes sufficient resources, tight resources, and moderate resources; Among them, the specific judgment method of the judgment result is: match the resource value with the preset resource value range group. If the resource value is within the sufficient resource value range, it is determined as sufficient resources; if the resource value is within the tight resource value range, it is determined as tight resources; if the resource value is within the moderate resource value range, it is determined as moderate resources.

[0027] It should be noted that by obtaining the system resource information of the storage device, calculating the pre-remaining capacity, weighted calculating the resource value and matching it with the preset range, a resource analysis result is generated, providing a basis for the storage policy adjustment on the system resource status; by understanding the resource status of the storage device, the intelligent contract execution module can reasonably adjust the data storage policy according to the resource situation, such as migrating data to improve access efficiency when resources are sufficient, and deleting low-frequency and low-value data to free up space when resources are tense, so as to achieve the efficient utilization of storage resources and the optimization of system performance.

[0028] In the present invention, the real-time heat, value evaluation value and system resource analysis result of the data are comprehensively analyzed and judged to trigger the adjustment of the storage policy. Specifically: Obtain the real-time heat of the data in the statistical period within the set time range, and calculate the growth rate of the data in the statistical period within this time range; perform weighted calculation on the real-time heat, growth rate and value evaluation value to obtain a data evaluation value; set the high-value evaluation threshold and low-value evaluation threshold of the data; If the data evaluation value is greater than its high-value evaluation threshold and the resource analysis result is that resources are sufficient, a data migration instruction is triggered; the data migration instruction is used to migrate the target data from the cloud storage area or the local low-speed memory to the local high-speed memory; If the data evaluation value is less than its low-value evaluation threshold: When the resource analysis result is that resources are sufficient or moderate, a data archiving instruction is triggered; the data archiving instruction is used to compress the target data and then migrate it to the local low-speed memory or the cloud storage area; When the resource analysis result is that resources are tense, a data deletion instruction is preferentially triggered, and the data deletion instruction is used to remove the target data from the local high-speed memory and only retain the backup in the cloud storage area; Among them, the data migration instruction is executed prior to the data archiving instruction and the data deletion instruction; All storage policy adjustment operations are recorded in the blockchain ledger module, including the operation time, data identifier, source storage location, target storage location and triggering conditions, forming an immutable operation log.

[0029] It should be noted that by comprehensively analyzing the real-time heat of the data, the value evaluation value and the system resource status, the data storage location can be reasonably arranged according to the importance of the data, the usage frequency and the availability of the system resources. For example, migrating high-value and heat-growing data to the local high-speed memory to improve data access efficiency; archiving or deleting low-value data to free up storage resources and improve the overall utilization rate of storage resources.

[0030] In the present invention, the present invention further includes: A data trend analysis module, configured to generate data prefetch instructions through historical data modeling and real-time business event analysis; A data prefetch module, configured to prefetch target data in advance according to the prefetch instructions output by the data trend analysis module and temporarily store it in a cache area; An asynchronous transmission module, configured to transmit the target data in the cache area to a specified storage device during a period when the system resource utilization rate is lower than a preset idle threshold.

[0031] It should be noted that through the data trend analysis, prefetching and asynchronous transmission mechanisms, the data access performance is effectively improved and the system resource utilization is optimized. The data trend analysis module predicts the data access demand, generates prefetch instructions, enables the data prefetch module to prefetch data in advance and temporarily store it, reduces the data access latency. The asynchronous transmission module uses the system idle period to transmit data, avoiding occupying resources during the system busy period, improving the overall system performance, and ensuring that the hot data can be quickly accessed without affecting the normal operation of the system.

[0032] In the present invention, the specific steps for analyzing historical data and real-time business trends are as follows: Collect a historical data set including at least the access timestamp t of the data, the access frequency f, and the business operation type O ; Clean the data set, removing invalid data with abnormal timestamps, negative or out-of-range access frequencies; Use a time series analysis algorithm (specifically, such as the ARIMA model) to model the historical access frequency data and obtain a fitted prediction function , where are model parameters; Determine the model order through the autocorrelation function (ACF) and partial autocorrelation function (PACF), and optimize the model parameters using the maximum likelihood estimation method; Real-time obtain business event data , where includes the event type, the influence range, and the duration; where r represents the index of the business event and b represents the number of business events; For each business event Calculate the influence coefficient ; According to the prediction function and the influence coefficient , calculate the corrected predicted access frequency , the formula is ; When the predicted access frequency is greater than or equal to a preset high-frequency access threshold, generate a prefetch instruction including the data identifier and the target storage device and send it to the data prefetch module.

[0033] It should be noted that by collecting and cleaning historical data, using time series analysis algorithms to build models, and calculating influence coefficients in combination with real-time business event data, various factors are comprehensively considered to predict data access frequencies, making the prediction results more in line with the actual business situation; and based on the predicted access frequencies, prefetch instructions are generated when the preset high-frequency access threshold is reached, and the data prefetch module fetches data in advance according to the instructions, reducing data access waiting time, improving system response speed, and optimizing the user experience.

[0034] In the present invention, it further includes a data monitoring module for real-time monitoring of the system resource information of the storage device in the computer; wherein the system resource information includes the remaining space, CPU utilization rate, and memory load conditions to obtain the real-time utilization rate of the system resources.

[0035] In the present invention, for each business event calculate the influence coefficient , specifically: Classify the business events, extract the key feature parameters of each business event, and the key feature parameters at least include the event type identifier , event influence range , event duration , event-associated data label set , ; where g represents the number of associated data; Collect historical business event data and construct a training data set , where is the feature vector of the v-th historical business event, is the growth rate of the target data access frequency caused by this event; Then adopt a multiple linear regression model , where is the z-th feature parameter of the r-th business event, is the feature parameter weight, which can be determined by fitting the training data set using the least squares method, and b1 is the model intercept term; For real-time business events , extract their feature parameters, substitute their feature parameters into the multiple linear regression model, and calculate the influence coefficient ; where when new historical business event data appears, retrain the multiple linear regression model and update the feature parameter weight and the intercept term b1.

[0036] It should be noted that by quantifying the impact of business events on data access frequency, it provides key support for accurately predicting data access trends. By classifying and extracting the characteristic parameters of business events to construct a training set and using a multiple linear regression model to calculate the influence coefficient, it can accurately reflect the degree of influence of different business events on data access. When new data is added, the model is retrained to update the parameters, ensuring the accuracy and timeliness of the influence coefficient, thereby improving the adaptability of the prediction model to real-time business changes, making the generation of data prefetch instructions more scientific and reasonable, and optimizing data storage and access strategies.

[0037] Those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in this disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the following claims.

[0038] It should be understood that the present invention is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A computer big data storage control system, characterized in that Including: A blockchain ledger module, which is used to construct a storage management ledger using blockchain technology, and record the storage location, access history, and adjustment records of storage policies for each piece of data in the storage management ledger; A data evaluation module, which is used to comprehensively calculate the real-time popularity of data. The real-time popularity is obtained by comprehensively calculating the data access frequency and access duration; the real-time popularity of the data is combined with multi-dimensional factors for quantitative evaluation to obtain the value evaluation value of the data; then analyze the system resource information of the storage device on the computer to obtain the resource analysis result, where the resource analysis result includes sufficient resources, tight resources, and moderate resources; A storage setting module, which is used to identify the current storage device, including at least a local high-speed memory, a local low-speed memory, and a cloud storage area; A smart contract execution module, which is used to comprehensively analyze and judge the real-time popularity, value evaluation value, and system resource analysis result of the data to trigger the adjustment of the storage policy.

2. The computer big data storage control system according to claim 1, characterized in that The specific steps for comprehensively calculating the real-time popularity of the data: Set the statistical period T for data access frequency, and count the number of data accesses n within this statistical period as the access frequency ; At the same time, record the duration of each data access , , calculate the average access duration ; Set the preset frequency weight and duration weight , and through the formula , calculate the basic heat value of the data within the statistical period ; Identify the category C of the data, and calculate the basic popularity value of all data in the same category within the statistical period , where j represents the j-th data in the data of the same category; calculate the basic popularity values of all data in the same category to obtain the category popularity value , the formula is , m represents the number of data in the same category, and J represents the number of data in the same category; The cosine similarity algorithm is used to calculate the similarity corresponding to the data category, and the category to which the data D belongs is denoted as , and other categories are denoted as , and k≠1, calculate and The cosine similarity between them, the formula is , where and are the data feature vectors of the category respectively; the cosine similarity is within the selected similarity threshold Sth, and the similarity weight of the similar category is set according to the similarity , where the calculation formula of the similarity weight is ; The similarity weights corresponding to the similar categories of the data are weighted with the category heat value to obtain the similarity heat value , and the formula is ; Normalize and weight the basic heat value, category heat value, and similarity heat value, and use the formula to calculate the real-time heat ; where h1 + h2 + h3 = 1, and h1, h2, and h3 represent the weights corresponding to the basic heat value, category heat value, and similarity heat value respectively.

3. A computer big data storage control system according to claim 1, characterized in that, Combining the real-time popularity of the data with multi-dimensional factors for quantitative evaluation to obtain the value evaluation value of the data. The specific evaluation is as follows: Obtain multi-dimensional factors; determine the multi-dimensional factor set , where represents business importance,[[]] represents timeliness,[[]] represents the degree of support for decision-making; Construct a judgment matrix using the Analytic Hierarchy Process , represents the ratio of the relative importance of factor and , where represents the p-th dimensional factor, represents the q-th dimensional factor; calculate the eigenvector of the judgment matrix, and this vector is the weight vector of each dimensional factor, satisfying , u = 1 ; where, x×x represents the scale of the matrix, that is, the number of rows of the dimension includes × includes the number of columns, represents the weight of the u-th dimensional factor, and u is the index of the dimensional factor; Obtain the real-time popularity H of the data; for each dimensional factor Quantify and score according to the preset scoring criteria to obtain a score ; Through the formula Obtain the value evaluation value V of the data.

4. A computer big data storage control system according to claim 1, characterized in that, The specific steps for analyzing the system resource information of the storage device on the computer to obtain the resource analysis result are: Obtain the system resource information of the storage device on the computer, including the remaining space, CPU usage rate, and memory load; Identify the required capacity of the data, and subtract the required capacity of the data from the remaining space of the storage device to obtain the pre-remaining capacity; Perform weighted calculation on the pre-remaining capacity, CPU usage rate, and memory load with their preset weights to obtain a resource value; match the resource value with a preset resource value range group to generate a resource analysis result; among them, the resource threshold group includes the value range of sufficient resources, tight resources, and moderate resources, and the resource analysis result includes sufficient resources, tight resources, and moderate resources.

5. A computer big data storage control system according to claim 1, characterized in that, Comprehensively analyze and judge the real-time popularity, value evaluation value, and system resource analysis result of the data to trigger the adjustment of the storage policy. Specifically: Obtain the real-time popularity of the data in the statistical period within the set time range, and calculate the growth rate of the data in the statistical period within this time range; perform weighted calculation on the real-time popularity, growth rate, and value evaluation value to obtain a data evaluation value; set the high-value evaluation threshold and low-value evaluation threshold of the data; If the data evaluation value is greater than its high-value evaluation threshold and the resource analysis result is sufficient resources, then trigger a data migration instruction; the data migration instruction is used to migrate the target data from the cloud storage area or the local low-speed memory to the local high-speed memory; If the data evaluation value is less than its low-value evaluation threshold: When the resource analysis result is sufficient resources or moderate resources, trigger a data archiving instruction; the data archiving instruction is used to compress the target data and then migrate it to the local low-speed memory or the cloud storage area; When the resource analysis result is tight resources, preferentially trigger a data deletion instruction, and the data deletion instruction is used to remove the target data from the local high-speed memory and only retain the backup in the cloud storage area; Among them, the data migration instruction takes precedence over the data archiving instruction and the data deletion instruction; Record all storage policy adjustment operations in the blockchain ledger module, including operation time, data identifier, source storage location, target storage location, and trigger conditions, to form an immutable operation log.

6. A computer big data storage control system according to claim 1, characterized in that, It also includes: A data trend analysis module configured to generate data prefetch instructions through historical data modeling and real-time business event analysis; A data prefetch module configured to prefetch target data in advance according to the prefetch instructions output by the data trend analysis module and temporarily store it in a cache area; An asynchronous transmission module configured to transmit the target data in the cache area to a specified storage device during a period when the system resource utilization rate is lower than a preset idle threshold.

7. A computer big data storage control system according to claim 6, characterized in that, The specific steps for analyzing historical data and real-time business trends are: Collect a historical data set that includes at least the access timestamp t of the data, the access frequency f, and the business operation type O ; Clean the data set, and eliminate invalid data with abnormal timestamps, negative access frequencies, or access frequencies exceeding a reasonable range Use the time series analysis algorithm to model the historical access frequency data and obtain the prediction function by fitting , where are the model parameters; determine the model order through the autocorrelation function and partial autocorrelation function, and optimize the model parameters using the maximum likelihood estimation method; Obtain business event data in real time , identify the parameters in the business event data and record them as , where include event type, impact scope, and duration; where r represents the index of the business event and b represents the number of business events; For each business event Calculate the impact coefficient ; According to the prediction function and the influence coefficient , calculate the corrected predicted access frequency , and the formula is ; When the predicted access frequency is greater than or equal to a preset high-frequency access threshold, generate a prefetch instruction including a data identifier and a target storage device and send it to the data prefetch module.

8. A computer big data storage control system according to claim 1, characterized in that, It also includes a data monitoring module for real-time monitoring of system resource information of storage devices in a computer; the system resource information includes remaining space, CPU utilization rate, and memory load conditions to obtain the real-time utilization rate of system resources.

9. A computer big data storage control system according to claim 7, characterized in that, For each business event Calculate the impact coefficient , specifically as follows: Classify business events and extract key feature parameters for each business event. The key feature parameters at least include an event type identifier , the scope of event impact , the duration of the event , a set of event-associated data tags , ; Where g represents the number of associated data; Collect historical business event data and construct a training data set , where is the feature vector of the v-th historical business event, is the growth rate of the target data access frequency triggered by this event; Then use the multiple linear regression model , where is the z-th feature parameter of the r-th business event, is the feature parameter weight, and b1 is the model intercept term; For real-time service events , extract their characteristic parameters, substitute the characteristic parameters into the multiple linear regression model, and calculate the influence coefficient ; when new historical service event data appears, retrain the multiple linear regression model and update the characteristic parameter weights and the intercept term b1.

Images