Data storage method and system for a small model with a built-in knowledge base

Abstract

The present invention relates to the technical field of data storage, and specifically discloses a data storage method and system for a small model with a built-in knowledge base. The method includes the following steps: S1: Obtain the sample knowledge data to be stored, S2: Analyze the deviation of the storage capacity of the knowledge data, S3: Analyze the optimization of the storage performance of the knowledge data, S4: Analyze the fluctuation of the storage requirements of the knowledge data, S5: Evaluate the storage cost of the knowledge data, and S6: Complete the optimization of the knowledge data storage; The present invention judges the capacity deviation by calculating the obtained capacity deviation coefficient; respectively optimizes the storage performance and matches the storage requirements of the capacity deviation data through the storage performance optimization coefficient and the storage requirement fluctuation coefficient; and evaluates the storage cost, and optimizes the data storage method based on the evaluation result; The present invention can improve the storage performance, adapt to the dynamic fluctuation of data, reduce the storage cost and improve the management efficiency, providing a strong guarantee for the development and application of the small model with a built-in knowledge base.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a data storage method and system with a small model having a built-in knowledge base. Background Art

[0002] With the rapid development of artificial intelligence technology, small models with built-in knowledge bases have been widely used in various fields. Small models with built-in knowledge bases usually integrate a large amount of professional knowledge and can provide users with accurate and efficient information services. However, traditional small models with built-in knowledge bases face many challenges and limitations in data storage.

[0003] First, the traditional knowledge base storage method adopts a unified strategy and cannot optimize the characteristics of different types of knowledge data. The fixed storage structure is difficult to adapt to the dynamic changes in data volume and differences in access patterns, resulting in inefficient storage resource utilization, and often insufficient capacity or resource waste. In addition, there is a lack of accurate analysis of the deviation of knowledge data storage capacity, and it is impossible to identify the difference between data storage demand and benchmark value, resulting in unbalanced storage resource allocation. Secondly, the traditional knowledge base storage method has obvious deficiencies in performance optimization. Most traditional methods only focus on basic data reading and writing functions, ignoring the optimization of storage performance, resulting in slower query speed and affecting user experience. In addition, the traditional knowledge base storage method lacks analysis of storage demand fluctuations, and generally adopts a static storage allocation scheme, which cannot flexibly adjust resource allocation, resulting in unbalanced resource utilization. Finally, there is a lack of systematic consideration in cost control. With the growth of data scale, storage cost has become a key factor restricting the large-scale deployment of small models, but most systems have not established a complete cost evaluation model, making it difficult to balance storage efficiency, access speed and resource investment. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present invention provide a data storage method and system with a small model of a built-in knowledge base to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a data storage method with a small model of a knowledge base, comprising the following steps: S1: obtaining sample knowledge data to be stored, S2: knowledge data storage capacity deviation analysis, S3: knowledge data storage performance optimization analysis, S4: knowledge data storage demand fluctuation analysis, S5: knowledge data storage cost evaluation and S6: completing knowledge data storage optimization;

[0006] S1: Acquire sample knowledge data to be stored: collect new knowledge data to be stored according to a preset collection frequency, mark them as sample knowledge data to be stored, and mark the standard storage knowledge data built into the knowledge base small model as benchmark knowledge data;

[0007] S2: Analysis of Knowledge Data Storage Capacity Deviation: Analyze the storage capacity deviation ratios of the knowledge data of each sample to be stored and the reference knowledge data storage capacity deviation ratio to obtain the capacity deviation coefficients of the knowledge data of each sample to be stored, and perform deviation judgment on them. If the judgment result is that the capacity deviation is normal, further execute S3; if the judgment result is that the capacity deviation is abnormal, then execute S4;

[0008] S3: Analysis of Knowledge Data Storage Performance Optimization: Obtain the storage performance parameters of the knowledge data of each sample to be stored, analyze to obtain the storage performance optimization coefficients of the knowledge data of each sample to be stored, and optimize the storage performance of the knowledge data with a normal capacity deviation judgment result in S2;

[0009] S4: Analysis of Knowledge Data Storage Demand Fluctuation: Obtain the storage demand parameters of the knowledge data of each sample to be stored, analyze to obtain the storage demand fluctuation coefficients of the knowledge data of each sample to be stored, and match the storage demands of the knowledge data with an abnormal capacity deviation judgment result in S2;

[0010] S5: Conduct Knowledge Data Storage Cost Evaluation: Analyze the storage costs of the knowledge data of each sample to be stored, and optimize the data storage method according to the evaluation results;

[0011] S6: Complete Knowledge Data Storage Optimization: Store the knowledge data of each sample to be stored based on the data after storage performance optimization, the data after storage demand matching, and the optimized data storage method.

[0012] Preferably, the execution method for obtaining the knowledge data of the sample to be stored is specifically as follows:

[0013] Collect the newly added knowledge data to be stored at a preset collection frequency, mark it as the knowledge data of each sample to be stored, and number the knowledge data of each sample to be stored in sequence as 1, 2,..., i,..., n, where i represents the number of the knowledge data of each sample to be stored, and the preset collection frequency is specifically to collect at a frequency of once every three seconds.

[0014] Preferably, the execution method for the knowledge data storage capacity deviation analysis is specifically as follows:

[0015] Obtain the system storage capacity of the target storage device and the data volumes of the knowledge data of each sample to be stored, and substitute them into the formula to obtain the storage capacity deviation ratio of the i-th knowledge data sample to be stored, where represents the data volume of the i-th knowledge data sample to be stored, Sc represents the system storage capacity of the target storage device, and Uc represents the system used capacity of the target storage device;

[0016] The data volume of the reference knowledge data is obtained , and substitute it into the formula , and calculate the storage capacity deviation ratio of the reference knowledge data ;

[0017] Calculate the capacity deviation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0018] , where DCD i represents the capacity deviation coefficient of the i-th sample knowledge data to be stored, represents the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data. i represents the number of each sample knowledge data to be stored, and i = 1, 2, 3,..., n;

[0019] Read the capacity deviation coefficients of each sample knowledge data to be stored, judge the capacity deviation of each sample knowledge data to be stored, and compare the capacity deviation coefficients of each sample knowledge data to be stored with the preset capacity deviation coefficient threshold;

[0020] If the capacity deviation coefficient of a certain sample knowledge data to be stored is less than the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of the corresponding sample knowledge data to be stored is normal, generate a storage performance optimization signal, perform storage performance optimization on the corresponding sample knowledge data to be stored, and further execute S3;

[0021] If the capacity deviation coefficient of a certain sample knowledge data to be stored is greater than or equal to the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of the corresponding sample knowledge data to be stored is abnormal, generate a storage demand fluctuation analysis signal, perform storage demand fluctuation analysis on the corresponding sample knowledge data to be stored, and further execute S4.

[0022] Preferably, the method for obtaining the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data is specifically as follows:

[0023] Obtain the data volume of each sample knowledge data to be stored and the data volume of the reference knowledge data, and substitute them into the formula respectively to obtain the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data , where represents the data volume of the i-th sample knowledge data to be stored, represents the data volume of the reference knowledge data.

[0024] Preferably, the execution method of the storage performance optimization analysis of the knowledge data is specifically as follows:

[0025] Obtain the storage performance parameters of each sample knowledge data to be stored, where the storage performance parameters include throughput requirements, request response duration requirements, and compression ratio;

[0026] Calculate the storage performance optimization coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0027] , where POC i represents the storage performance optimization coefficient of the i-th sample knowledge data to be stored, Th i represents the throughput requirement of the i-th sample knowledge data to be stored, represents the preset minimum effective throughput, Rt i represents the request response duration requirement of the i-th sample knowledge data to be stored, represents the preset maximum acceptable request response duration, Co i represents the compression ratio of the i-th sample knowledge data to be stored;

[0028] Read the storage performance optimization coefficients of each sample knowledge data to be stored, and perform storage performance optimization on the knowledge data with normal capacity deviation in the judgment result in S2 through the storage performance optimization coefficients of each sample knowledge data to be stored.

[0029] Preferably, the execution method of the knowledge data storage requirement fluctuation analysis is specifically as follows:

[0030] Record the required storage amount, access frequency, and retention duration of each sample knowledge data to be stored as the storage requirement parameters of each sample knowledge data to be stored;

[0031] Read the maximum required storage amount , minimum required storage amount and average required storage amount of the i-th sample knowledge data to be stored, and substitute them into the formula respectively to calculate the required storage amount change rate Mr i of the i-th sample knowledge data to be stored;

[0032] Calculate the storage requirement fluctuation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0033] , where DFC i represents the storage requirement fluctuation coefficient of the i-th sample knowledge data to be stored, Mr i represents the required storage amount change rate of the i-th sample knowledge data to be stored, Fr i represents the access frequency change rate of the i-th sample knowledge data to be stored, Rd i represents the retention duration of the i-th sample knowledge data to be stored, represents a preset standard retention duration;

[0034] Read the storage requirement fluctuation coefficients of each sample knowledge data to be stored, and perform storage requirement matching analysis on the knowledge data with abnormal capacity deviation in the judgment result in S2 according to the storage requirement fluctuation coefficients of each sample knowledge data to be stored.

[0035] Preferably, the execution method for evaluating the storage cost of knowledge data is specifically as follows:

[0036] Obtain the storage cost data of each sample knowledge data to be stored;

[0037] Calculate the storage cost evaluation index of each sample knowledge data to be stored, and the specific calculation formula is as follows:

[0038] , where CEI i represents the storage cost evaluation index of the i-th sample knowledge data to be stored, represents the actual storage cost value of the j-th storage cost corresponding to the i-th sample knowledge data to be stored, represents the preset reference value of the j-th storage cost, represents the weight of the j-th storage cost, j represents the number of each storage cost, j = 1, 2, 3,..., m;

[0039] Read the storage cost evaluation indexes of each sample knowledge data to be stored, evaluate the storage costs of each sample knowledge data to be stored, compare the storage cost evaluation index of a certain sample knowledge data to be stored with the preset storage cost evaluation index threshold. If the storage cost evaluation index of a certain sample knowledge data to be stored is less than the preset storage cost evaluation index threshold, it is judged that the storage cost control state of this sample knowledge data to be stored is normal. If the storage cost evaluation index of a certain sample knowledge data to be stored is greater than or equal to the preset storage cost evaluation index threshold, it is judged that the storage cost control state of this sample knowledge data to be stored is abnormal. Mark the abnormal data of the storage cost control state of this sample knowledge data to be stored as the storage cost evaluation result of this sample knowledge data to be stored, and select the corresponding data storage method for data storage according to the evaluation result.

[0040] To achieve the above object, the present invention provides the following technical solution: a data storage system with a built-in knowledge base small model, implementing the above data storage method with a built-in knowledge base small model, including: a module for obtaining sample knowledge data to be stored, a module for analyzing the deviation of the storage capacity of knowledge data, a module for optimizing the analysis of the storage performance of knowledge data, a module for analyzing the fluctuation of the storage requirements of knowledge data, a module for evaluating the storage cost of knowledge data, and a module for completing the optimization of the storage of knowledge data;

[0041] Module for obtaining knowledge data of samples to be stored: It is used to collect newly added knowledge data to be stored according to a preset collection frequency, mark it as knowledge data of each sample to be stored, and mark the standard storage knowledge data built in the small knowledge base model as reference knowledge data;

[0042] Module for analyzing the deviation of knowledge data storage capacity: It is used to analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation ratio of the reference knowledge data, obtain the capacity deviation coefficient of each sample knowledge data to be stored, and conduct a deviation judgment on it. If the judgment result is that the capacity deviation is normal, the knowledge data storage performance optimization analysis module is further executed. If the judgment result is that the capacity deviation is abnormal, the knowledge data storage demand fluctuation analysis module is executed;

[0043] Module for optimizing the storage performance of knowledge data: It is used to obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficient of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data whose judgment result in the knowledge data storage capacity deviation analysis module is that the capacity deviation is normal;

[0044] Module for analyzing the fluctuation of knowledge data storage demand: It is used to obtain the storage demand parameters of each sample knowledge data to be stored, analyze and obtain the storage demand fluctuation coefficient of each sample knowledge data to be stored, and match the storage demand of the knowledge data whose judgment result in the knowledge data storage capacity deviation analysis module is that the capacity deviation is abnormal;

[0045] Module for evaluating the storage cost of knowledge data: It is used to analyze the storage cost of each sample knowledge data to be stored and optimize the data storage method according to the evaluation result;

[0046] Module for completing the optimization of knowledge data storage: Based on the data after storage performance optimization, the data after storage demand matching, and the optimized data storage method, the knowledge data of each sample to be stored is stored.

[0047] As described above, the data storage method and system with a built-in small knowledge base model provided by the present invention have at least the following beneficial effects:

[0048] The data storage method and system for a small model with a built-in knowledge base provided by the present invention obtain various sample knowledge data to be stored and reference knowledge data, analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation of the reference knowledge data, obtain the capacity deviation coefficient of each sample knowledge data to be stored, and make judgments on them. According to the judgment results, the data is processed. By using this method, knowledge data with abnormal storage capacity can be accurately identified and processed, ensuring the efficient operation of the knowledge base. At the same time, this method supports dynamic adjustment of storage strategies to adapt to the continuous changes of the knowledge base, improving the flexibility and scalability of storage. Through the calculated storage performance optimization coefficient, the storage performance of the knowledge data with normal capacity deviation in the judgment result is optimized; through the calculated storage demand fluctuation coefficient, the storage demand matching of the knowledge data with abnormal capacity deviation in the judgment result is carried out; by processing the data with different storage capacity deviation judgment results, not only the storage efficiency is improved, the efficient operation of the knowledge base is ensured, but also the flexible allocation of resources is realized, the utilization rate of storage resources is improved, and resource waste is avoided. Finally, by analyzing the storage cost, the storage cost evaluation index is obtained, and the data storage method is optimized according to the evaluation result, thereby reducing the storage cost, enhancing the accuracy and timeliness of knowledge base update, and improving the user experience and model performance. The present invention can also improve storage performance, adapt to data dynamic fluctuations, and improve management efficiency, providing a strong guarantee for the development and application of the small model with a built-in knowledge base. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention will be further described with reference to the accompanying drawings. However, the embodiments in the drawings do not constitute any limitation to the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on the following drawings without creative efforts.

[0050] Figure 1 It is a schematic flow chart of the data storage method for the small model with a built-in knowledge base of the present invention.

[0051] Figure 2 It is a schematic structural diagram of the data storage system for the small model with a built-in knowledge base of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] Embodiment 1

[0054] Please refer to Figure 1As shown in the figure, the present invention provides a data storage method for a small model with a built-in knowledge base, including the following steps: S1: Obtain the sample knowledge data to be stored; S2: Analyze the deviation of the storage capacity of the knowledge data; S3: Analyze the optimization of the storage performance of the knowledge data; S4: Analyze the fluctuation of the storage requirements of the knowledge data; S5: Evaluate the storage cost of the knowledge data; and S6: Complete the optimization of the knowledge data storage.

[0055] S1: Obtain the sample knowledge data to be stored: Collect the new knowledge data to be stored according to the preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built in the small model of the knowledge base as the reference knowledge data.

[0056] In this embodiment, it should be specifically noted that the execution manner of obtaining the sample knowledge data to be stored is as follows:

[0057] Collect the new knowledge data to be stored according to the preset collection frequency, mark it as each sample knowledge data to be stored, and number each sample knowledge data to be stored in sequence as 1, 2,..., i,..., n, where i represents the number of each sample knowledge data to be stored, and the preset collection frequency is specifically to collect once every three seconds.

[0058] S2: Analyze the deviation of the storage capacity of the knowledge data: Analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation ratio of the reference knowledge data to obtain the capacity deviation coefficient of each sample knowledge data to be stored, and perform a deviation judgment on it. If the judgment result is that the capacity deviation is normal, further execute S3; if the judgment result is that the capacity deviation is abnormal, then execute S4.

[0059] In this embodiment, it should be specifically noted that the execution manner of analyzing the deviation of the storage capacity of the knowledge data is as follows:

[0060] Obtain the system storage capacity of the target storage device and the data volume of each sample knowledge data to be stored, and substitute them into the formula , to obtain the storage capacity deviation ratio of the i-th sample knowledge data to be stored, where represents the data volume of the i-th sample knowledge data to be stored, Sc represents the system storage capacity of the target storage device, and Uc represents the system used capacity of the target storage device.

[0061] Obtain the data volume of the reference knowledge data, substitute it into the formula , and calculate the storage capacity deviation ratio of the reference knowledge data;

[0062] Calculate the capacity deviation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0063] , where DCD i represents the capacity deviation coefficient of the i-th sample knowledge data to be stored, represents the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data. i represents the number of each sample knowledge data to be stored, and i = 1, 2, 3,..., n;

[0064] Read the capacity deviation coefficients of each sample knowledge data to be stored, judge the capacity deviation of each sample knowledge data to be stored, and compare the capacity deviation coefficients of each sample knowledge data to be stored with a preset capacity deviation coefficient threshold;

[0065] If the capacity deviation coefficient of a certain sample knowledge data to be stored is less than the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of this sample knowledge data to be stored is normal, generate a storage performance optimization signal, perform storage performance optimization on the corresponding sample knowledge data to be stored, and further execute S3;

[0066] If the capacity deviation coefficient of a certain sample knowledge data to be stored is greater than or equal to the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of this sample knowledge data to be stored is abnormal, generate a storage demand fluctuation analysis signal, perform storage demand fluctuation analysis on the corresponding sample knowledge data to be stored, and further execute S4.

[0067] It should be specifically noted that the method for obtaining the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data is as follows:

[0068] Obtain the data volumes of each sample knowledge data to be stored and the data volume of the reference knowledge data, and substitute them into the formula respectively to obtain the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data , where represents the data volume of the i-th sample knowledge data to be stored, represents the data volume of the reference knowledge data.

[0069] S3: Knowledge data storage performance optimization analysis: Obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficients of each sample knowledge data to be stored, and perform storage performance optimization on the knowledge data with normal capacity deviation judged in S2;

[0070] In this embodiment, it should be specifically noted that the execution method of the knowledge data storage performance optimization analysis is as follows:

[0071] Obtain the storage performance parameters of each sample knowledge data to be stored, where the storage performance parameters include throughput requirements, request response duration requirements, and compression ratio;

[0072] Calculate the storage performance optimization coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0073] , where POC i represents the storage performance optimization coefficient of the i-th sample knowledge data to be stored, Th i represents the throughput requirement of the i-th sample knowledge data to be stored, represents the preset minimum effective throughput, Rt i represents the request response duration requirement of the i-th sample knowledge data to be stored, represents the preset maximum acceptable request response duration, Co i represents the compression ratio of the i-th sample knowledge data to be stored;

[0074] Read the storage performance optimization coefficients of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data with a normal capacity deviation judgment result in S2 through the storage performance optimization coefficients of each sample knowledge data to be stored;

[0075] It should be specifically noted that in the formula, the larger the throughput requirement Th i of the i-th sample knowledge data to be stored, the smaller the request response duration requirement Rt i of the i-th sample knowledge data to be stored, and the smaller the compression ratio Co i of the i-th sample knowledge data to be stored, the larger the storage performance optimization coefficient POC i of the i-th sample knowledge data to be stored, indicating that the storage performance of the i-th sample knowledge data to be stored is better. And the throughput requirement, request response duration requirement, and compression ratio of the i-th sample knowledge data to be stored do not affect each other.

[0076] S4: Analysis of storage demand fluctuations of knowledge data: Obtain the storage demand parameters of each sample knowledge data to be stored, analyze to obtain the storage demand fluctuation coefficients of each sample knowledge data to be stored, and perform storage demand matching on the knowledge data with an abnormal capacity deviation judgment result in S2;

[0077] In this embodiment, it should be specifically noted that the execution method of the storage demand fluctuation analysis of the knowledge data is as follows:

[0078] Record the required storage amount, access frequency, and retention duration of each sample knowledge data to be stored as the storage demand parameters of each sample knowledge data to be stored;

[0079] Read the maximum required storage capacity, minimum required storage capacity, and average required storage capacity of the i-th sample knowledge data to be stored, and substitute them into the formula respectively to calculate the change rate Mr of the required storage capacity of the i-th sample knowledge data to be stored; , minimum required storage capacity and average required storage capacity , and substitute them into the formula to calculate the change rate Mr of the required storage capacity of the i-th sample knowledge data to be stored i ;

[0080] Calculate the storage requirement fluctuation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0081] , where DFC i represents the storage requirement fluctuation coefficient of the i-th sample knowledge data to be stored, Mr i represents the change rate of the required storage capacity of the i-th sample knowledge data to be stored, Fr i represents the change rate of the access frequency of the i-th sample knowledge data to be stored, and Rd i represents the retention duration of the i-th sample knowledge data to be stored, represents the preset standard retention duration;

[0082] Read the storage requirement fluctuation coefficient of each sample knowledge data to be stored, and conduct a storage requirement matching analysis on the knowledge data with abnormal capacity deviation in the judgment result in S2 based on the storage requirement fluctuation coefficient of each sample knowledge data to be stored;

[0083] It should be specifically noted that in the formula, the greater the deviation of the retention duration Rd of the i-th sample knowledge data to be stored from the preset standard retention duration, the greater the change rate Mr of the required storage capacity, and the greater the change rate Fr of the access frequency, the greater the storage requirement fluctuation coefficient DFC of the i-th sample knowledge data to be stored, indicating that the storage requirement fluctuation of the i-th sample knowledge data to be stored is greater. And there is no mutual influence among the retention duration, the change rate of the required storage capacity, and the change rate of the access frequency of the i-th sample knowledge data to be stored. i The greater the deviation of the retention duration Rd of the i-th sample knowledge data to be stored from the preset standard retention duration, the greater the change rate Mr of the required storage capacity, and the greater the change rate Fr of the access frequency, the greater the storage requirement fluctuation coefficient DFC of the i-th sample knowledge data to be stored, indicating that the storage requirement fluctuation of the i-th sample knowledge data to be stored is greater. And there is no mutual influence among the retention duration, the change rate of the required storage capacity, and the change rate of the access frequency of the i-th sample knowledge data to be stored. i The greater the deviation of the retention duration Rd of the i-th sample knowledge data to be stored from the preset standard retention duration, the greater the change rate Mr of the required storage capacity, and the greater the change rate Fr of the access frequency, the greater the storage requirement fluctuation coefficient DFC of the i-th sample knowledge data to be stored, indicating that the storage requirement fluctuation of the i-th sample knowledge data to be stored is greater. And there is no mutual influence among the retention duration, the change rate of the required storage capacity, and the change rate of the access frequency of the i-th sample knowledge data to be stored. i The greater the deviation of the retention duration Rd of the i-th sample knowledge data to be stored from the preset standard retention duration, the greater the change rate Mr of the required storage capacity, and the greater the change rate Fr of the access frequency, the greater the storage requirement fluctuation coefficient DFC of the i-th sample knowledge data to be stored, indicating that the storage requirement fluctuation of the i-th sample knowledge data to be stored is greater. And there is no mutual influence among the retention duration, the change rate of the required storage capacity, and the change rate of the access frequency of the i-th sample knowledge data to be stored. i The greater the deviation of the retention duration Rd of the i-th sample knowledge data to be stored from the preset standard retention duration, the greater the change rate Mr of the required storage capacity, and the greater the change rate Fr of the access frequency, the greater the storage requirement fluctuation coefficient DFC of the i-th sample knowledge data to be stored, indicating that the storage requirement fluctuation of the i-th sample knowledge data to be stored is greater. And there is no mutual influence among the retention duration, the change rate of the required storage capacity, and the change rate of the access frequency of the i-th sample knowledge data to be stored.

[0084] It should be specifically noted that the specific calculation formula of the change rate of the access frequency of the i-th sample knowledge data to be stored is as follows:

[0085] From the formula , calculate the change rate Fr of the access frequency of the i-th sample knowledge data to be stored i , where represents the maximum access frequency of the i-th sample knowledge data to be stored, represents the minimum access frequency of the i-th sample knowledge data to be stored, represents the average access frequency of the i-th sample knowledge data to be stored;

[0086] In this embodiment, a specific embodiment is provided as follows:

[0087] Read the storage demand fluctuation coefficients of each sample knowledge data to be stored. When the storage demand fluctuation coefficient of the i-th sample knowledge data to be stored , it indicates that the storage demand of this sample knowledge data belongs to severe fluctuation, and a corresponding storage method demand matching strategy is generated. Specifically: adopt an elastic storage architecture (such as dynamic cloud storage expansion), enable multi-level caching for high-access-frequency data, and shorten the storage cycle of data with an over-standard retention duration;

[0088] When , it indicates that the storage demand of this sample knowledge data belongs to moderate fluctuation, and a corresponding storage method demand matching strategy is generated. Specifically: regularly monitor the change in storage volume, reserve 10% - 20% elastic space; and allocate storage resources in advance;

[0089] When , it indicates that the storage demand of this sample knowledge data belongs to mild fluctuation, and a corresponding storage method demand matching strategy is generated. Specifically: maintain the existing conventional storage strategy.

[0090] S5: Conduct an assessment of the storage cost of knowledge data: Analyze the storage costs of each sample knowledge data to be stored, and optimize the data storage method according to the evaluation results;

[0091] In this embodiment, it should be specifically noted that the execution method of the assessment of the storage cost of knowledge data is as follows:

[0092] Obtain the storage cost data of each sample knowledge data to be stored, and the storage cost data includes but is not limited to hardware cost, maintenance cost, energy consumption cost, data migration cost, and operation and maintenance cost;

[0093] Calculate the storage cost evaluation index of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0094] , where CEI i represents the storage cost evaluation index of the i-th sample knowledge data to be stored, represents the actual storage cost value of the j-th storage cost corresponding to the i-th sample knowledge data to be stored, represents the preset benchmark value of the j-th storage cost, represents the weight of the j-th storage cost, and j represents the number of each storage cost, j = 1, 2, 3,..., m;

[0095] Read the storage cost evaluation index of each sample knowledge data to be stored, evaluate the storage cost of each sample knowledge data to be stored, compare the storage cost evaluation index of a certain sample knowledge data to be stored with the preset storage cost evaluation index threshold. If the storage cost evaluation index of a certain sample knowledge data to be stored is less than the preset storage cost evaluation index threshold, it is determined that the storage cost control status of this sample knowledge data to be stored is normal. If the storage cost evaluation index of a certain sample knowledge data to be stored is greater than or equal to the preset storage cost evaluation index threshold, it is determined that the storage cost control status of this sample knowledge data to be stored is abnormal. Mark the abnormal data of the storage cost control status of this sample knowledge data to be stored as the storage cost evaluation result of this sample knowledge data to be stored, and select the corresponding data storage method for data storage according to the evaluation result.

[0096] S6: Complete the optimization of knowledge data storage: Store each sample knowledge data to be stored based on the data optimized for storage performance, the data matched for storage requirements, and the optimized data storage method.

[0097] Embodiment 2

[0098] Please refer to Figure 2 As shown, the present invention provides a data storage system with a built-in knowledge base small model, including a module for obtaining sample knowledge data to be stored, a module for analyzing the deviation of knowledge data storage capacity, a module for optimizing and analyzing the storage performance of knowledge data, a module for analyzing the fluctuation of knowledge data storage requirements, a module for evaluating the storage cost of knowledge data, and a module for completing the optimization of knowledge data storage;

[0099] Module for obtaining sample knowledge data to be stored: It is used to collect the newly added knowledge data to be stored according to the preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built in the knowledge base small model as the reference knowledge data;

[0100] Module for analyzing the deviation of knowledge data storage capacity: It is used to analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation ratio of the reference knowledge data, obtain the capacity deviation coefficient of each sample knowledge data to be stored, and perform deviation judgment on it. If the judgment result is that the capacity deviation is normal, further execute the module for optimizing and analyzing the storage performance of knowledge data. If the judgment result is that the capacity deviation is abnormal, then execute the module for analyzing the fluctuation of knowledge data storage requirements;

[0101] Module for optimizing and analyzing the storage performance of knowledge data: It is used to obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficient of each sample knowledge data to be stored, and perform storage performance optimization on the knowledge data whose judgment result in the module for analyzing the deviation of knowledge data storage capacity is that the capacity deviation is normal;

[0102] Knowledge Data Storage Requirement Fluctuation Analysis Module: It is used to obtain the storage requirement parameters of each sample knowledge data to be stored, analyze and obtain the storage requirement fluctuation coefficients of each sample knowledge data to be stored, and perform storage requirement matching on the knowledge data with abnormal capacity deviation judgment results in the Knowledge Data Storage Capacity Deviation Analysis Module;

[0103] Knowledge Data Storage Cost Evaluation Module: It is used to analyze the storage costs of each sample knowledge data to be stored, and optimize the data storage method according to the evaluation results;

[0104] Knowledge Data Storage Optimization Completion Module: Store each sample knowledge data to be stored based on the data optimized for storage performance, the data after storage requirement matching, and the optimized data storage method.

[0105] Finally: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0106] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or replacements, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

1. A data storage method for a small model with a built-in knowledge base, characterized in that, It includes the following steps: S1: Obtain the sample knowledge data to be stored: Collect the newly added knowledge data to be stored according to the preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard stored knowledge data built in the knowledge base small model as the reference knowledge data; S2: Analyze the storage capacity deviation of knowledge data: Analyze the storage capacity deviation ratios of each sample knowledge data to be stored and the reference knowledge data, obtain the capacity deviation coefficients of each sample knowledge data to be stored, and perform deviation judgment on them. If the judgment result is that the capacity deviation is normal, further execute S3. If the judgment result is that the capacity deviation is abnormal, then execute S4; S3: Analyze the storage performance optimization of knowledge data: Obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficients of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data with normal capacity deviation judged in S2; S4: Analyze the storage demand fluctuation of knowledge data: Obtain the storage demand parameters of each sample knowledge data to be stored, analyze and obtain the storage demand fluctuation coefficients of each sample knowledge data to be stored, and match the storage demands of the knowledge data with abnormal capacity deviation judged in S2; For severe fluctuations, adopt an elastic storage architecture, enable multi-level caching for high-access-frequency data, and shorten the storage cycle of data with an over-standard retention duration; For moderate fluctuations, regularly monitor the change in storage volume, reserve a 10% - 20% elastic space; and allocate storage resources in advance; For mild fluctuations, maintain the existing conventional storage strategy; S5: Conduct an assessment of the storage cost of knowledge data: Analyze the storage costs of each sample knowledge data to be stored, and optimize the data storage method according to the evaluation results; S6: Complete the optimization of knowledge data storage: Store each sample knowledge data to be stored based on the data after storage performance optimization, the data after storage demand matching, and the optimized data storage method.

2. The data storage method of the small model with a built-in knowledge base according to claim 1, characterized in that: The execution method of obtaining the sample knowledge data to be stored is specifically as follows: Collect the newly added knowledge data to be stored according to the preset collection frequency, mark it as each sample knowledge data to be stored, and number each sample knowledge data to be stored in sequence as 1, 2,..., i,..., n, where i represents the number of each sample knowledge data to be stored, and the preset collection frequency is specifically to collect at a frequency of once every three seconds.

3. The data storage method of the small model with a built-in knowledge base according to claim 1, characterized in that: The execution method of the storage capacity deviation analysis of knowledge data is specifically as follows: Obtain the system storage capacity of the target storage device and the data volume of each sample knowledge data to be stored, and substitute them into the formula respectively , to obtain the storage capacity deviation ratio of the i-th sample knowledge data to be stored , where represents the data volume of the i-th sample knowledge data to be stored, Sc represents the system storage capacity of the target storage device, and Uc represents the system used capacity of the target storage device; The data volume of the reference knowledge data , substitute it into the formula , and calculate the storage capacity deviation ratio of the reference knowledge data ; Calculate the capacity deviation coefficients of each sample knowledge data to be stored, and the specific calculation formula is as follows: , where DCD i represents the capacity deviation coefficient of the i-th sample knowledge data to be stored, represents the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data, and i represents the number of each sample knowledge data to be stored, i = 1, 2, 3,..., n; Read the capacity deviation coefficients of each sample knowledge data to be stored, judge the capacity deviation of each sample knowledge data to be stored, and compare the capacity deviation coefficients of each sample knowledge data to be stored with the preset capacity deviation coefficient threshold; If the capacity deviation coefficient of a certain sample knowledge data to be stored is less than the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of this sample knowledge data to be stored is normal, generate a storage performance optimization signal, optimize the storage performance of the corresponding sample knowledge data to be stored, and further execute S3; If the capacity deviation coefficient of a sample knowledge data to be stored is greater than or equal to a preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of the sample knowledge data to be stored is abnormal, a storage demand fluctuation analysis signal is generated, and the corresponding sample knowledge data to be stored is subjected to storage demand fluctuation analysis, and S4 is further executed.

4. The data storage method of the small model with a built-in knowledge base according to claim 3, wherein: The method for obtaining the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data is specifically as follows: Obtain the data volumes of each sample knowledge data to be stored and the data volume of the reference knowledge data, and substitute them into the formula respectively , to obtain the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data , where represents the data volume of the i-th sample knowledge data to be stored represents the data volume of the reference knowledge data 5. The data storage method of the small model with a built-in knowledge base according to claim 1, characterized in that: The execution method of the knowledge data storage performance optimization analysis is specifically as follows: Obtain the storage performance parameters of each sample knowledge data to be stored, where the storage performance parameters include throughput requirements, request response duration requirements, and compression ratio; Calculate the storage performance optimization coefficient of each sample knowledge data to be stored, and the specific calculation formula is as follows: , where POC i represents the storage performance optimization coefficient of the i-th sample knowledge data to be stored, Th i represents the throughput requirement of the i-th sample knowledge data to be stored, represents the preset minimum effective throughput, Rt i represents the request response duration requirement of the i-th sample knowledge data to be stored, represents the preset maximum acceptable request response duration, Co i represents the compression ratio of the i-th sample knowledge data to be stored; Read the storage performance optimization coefficient of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data with a normal capacity deviation judgment result in S2 through the storage performance optimization coefficient of each sample knowledge data to be stored.

6. The data storage method of the small model with a built-in knowledge base according to claim 1, characterized in that: The execution method of the knowledge data storage demand fluctuation analysis is specifically as follows: Record the required storage volume, access frequency, and retention duration of each sample knowledge data to be stored as the storage demand parameters of each sample knowledge data to be stored; Read the maximum required storage capacity, minimum required storage capacity, and average required storage capacity of the i-th sample knowledge data to be stored, and substitute them into the formula respectively to calculate the change rate Mr of the required storage capacity of the i-th sample knowledge data to be stored i ; and the minimum required storage capacity as well as the average required storage capacity , and substitute them into the formula respectively to calculate the change rate Mr of the required storage capacity of the i-th sample knowledge data to be stored i ; Calculate the storage demand fluctuation coefficient of each sample knowledge data to be stored, and the specific calculation formula is as follows: , where DFC i represents the storage requirement fluctuation coefficient of the i-th sample knowledge data to be stored, Mr i represents the change rate of the required storage amount of the i-th sample knowledge data to be stored, Fr i represents the change rate of the access frequency of the i-th sample knowledge data to be stored, Rd i represents the retention duration of the i-th sample knowledge data to be stored, represents the preset standard retention duration; Read the storage demand fluctuation coefficient of each sample knowledge data to be stored, and perform storage demand matching analysis on the knowledge data with an abnormal capacity deviation judgment result in S2 according to the storage demand fluctuation coefficient of each sample knowledge data to be stored.

7. The data storage method of the small model with a built-in knowledge base according to claim 1, wherein: The execution method of performing knowledge data storage cost evaluation is specifically as follows: Obtain the storage cost data of each sample knowledge data to be stored; Calculate the storage cost evaluation index of each sample knowledge data to be stored, and the specific calculation formula is as follows: , where CEI i represents the storage cost evaluation index of the i-th sample knowledge data to be stored, represents the actual value of the storage cost of the j-th storage cost corresponding to the i-th sample knowledge data to be stored, represents the preset reference value of the j-th storage cost, represents the weight of the j-th storage cost, and j represents the number of each storage cost, j = 1, 2, 3,..., m; Read the storage cost evaluation index of each sample knowledge data to be stored, evaluate the storage cost of each sample knowledge data to be stored, compare the storage cost evaluation index of a sample knowledge data to be stored with a preset storage cost evaluation index threshold. If the storage cost evaluation index of a sample knowledge data to be stored is less than the preset storage cost evaluation index threshold, it is judged that the storage cost control status of the sample knowledge data to be stored is normal. If the storage cost evaluation index of a sample knowledge data to be stored is greater than or equal to the preset storage cost evaluation index threshold, it is judged that the storage cost control status of the sample knowledge data to be stored is abnormal, mark the abnormal data of the storage cost control status of the sample knowledge data to be stored as the storage cost evaluation result of the sample knowledge data to be stored, and select the corresponding data storage method for data storage according to the evaluation result.

8. A data storage system for a small model with a built-in knowledge base, which is used to implement the data storage method for a small model with a built-in knowledge base according to any one of claims 1-7 above, characterized in that, Including: A module for obtaining sample knowledge data to be stored: used to collect new knowledge data to be stored at a preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built in the knowledge base small model as the reference knowledge data; Knowledge Data Storage Capacity Deviation Analysis Module: It is used to analyze the storage capacity deviation ratios of each sample knowledge data to be stored and the reference knowledge data storage capacity deviation ratio, obtain the capacity deviation coefficients of each sample knowledge data to be stored, and perform deviation judgment on them. If the judgment result is that the capacity deviation is normal, the knowledge data storage performance optimization analysis module is further executed. If the judgment result is that the capacity deviation is abnormal, the knowledge data storage demand fluctuation analysis module is executed; Knowledge Data Storage Performance Optimization Analysis Module: It is used to obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficients of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data with a normal capacity deviation judgment result in the knowledge data storage capacity deviation analysis module; Knowledge Data Storage Demand Fluctuation Analysis Module: It is used to obtain the storage demand parameters of each sample knowledge data to be stored, analyze and obtain the storage demand fluctuation coefficients of each sample knowledge data to be stored, and match the storage demands of the knowledge data with an abnormal capacity deviation judgment result in the knowledge data storage capacity deviation analysis module; Knowledge Data Storage Cost Evaluation Module: It is used to analyze the storage costs of each sample knowledge data to be stored and optimize the data storage method according to the evaluation results; Knowledge Data Storage Optimization Completion Module: Store each sample knowledge data to be stored based on the data after storage performance optimization, the data after storage demand matching, and the optimized data storage method.

Images