Data storage method and system with knowledge base small model

Abstract

The invention relates to the technical field of data storage, and particularly discloses a data storage method and system with a knowledge base small model. The method comprises the following steps of S1, obtaining to-be-stored sample knowledge data, S2, analyzing knowledge data storage capacity deviation, S3, analyzing knowledge data storage performance optimization, S4, analyzing knowledge data storage demand fluctuation, S5, evaluating knowledge data storage cost and S6, completing knowledge data storage optimization. According to the invention, capacity deviation judgment is carried out through the calculated capacity deviation coefficient; performing storage performance optimization and storage demand matching on the capacity deviation data through a storage performance optimization coefficient and a storage demand fluctuation coefficient; the storage cost is evaluated, and the data storage mode is optimized based on an evaluation result; according to the method, the storage performance can be improved, data dynamic fluctuation is adapted, the storage cost is reduced, the management efficiency is improved, and a powerful guarantee is provided for development and application of a small model with a knowledge base.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and particularly to a data storage method and system for a small model with a built-in knowledge base. Background Art

[0002] With the rapid development of artificial intelligence technology, small models with built-in knowledge bases have been widely used in various fields. Small models with built-in knowledge bases usually integrate a large amount of professional knowledge and can provide accurate and efficient information services for users. However, traditional small models with built-in knowledge bases face many challenges and limitations in data storage.

[0003] Firstly, traditional knowledge base storage methods adopt a unified strategy and cannot be optimized according to the characteristics of different types of knowledge data. The fixed storage structure is difficult to adapt to the dynamic changes in data volume and differences in access patterns, resulting in low utilization efficiency of storage resources, and often there are situations of insufficient capacity or resource waste; and there is a lack of accurate analysis of the storage capacity deviation of knowledge data and the difference between the data storage requirements and the benchmark value cannot be identified, leading to an imbalance in storage resource allocation. Secondly, traditional knowledge base storage methods have obvious deficiencies in performance optimization. Most traditional methods only focus on basic data reading and writing functions and ignore the optimization of storage performance, resulting in slower query speeds and affecting the user experience. In addition, traditional knowledge base storage methods lack analysis of storage demand fluctuations and generally adopt static storage allocation schemes, unable to flexibly adjust resource allocation, resulting in unbalanced resource utilization. Finally, there is a lack of systematic consideration in cost control. As the data scale grows, storage cost has become a key factor restricting the large-scale deployment of small models, but most systems have not established a perfect cost evaluation model and it is difficult to balance storage efficiency, access speed, and resource investment. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a data storage method and system for a small model with a built-in knowledge base to solve the problems raised in the above background art.

[0005] To achieve the above object, the present invention provides the following technical solution: A data storage method for a small model with a built-in knowledge base, including the following steps: S1: Obtain sample knowledge data to be stored, S2: Analyze the storage capacity deviation of knowledge data, S3: Analyze the storage performance optimization of knowledge data, S4: Analyze the storage demand fluctuation of knowledge data, S5: Conduct a storage cost assessment of knowledge data, and S6: Complete the storage optimization of knowledge data;

[0006] S1: Obtain sample knowledge data to be stored: Collect newly added knowledge data to be stored at a preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built into the small model of the knowledge base as benchmark knowledge data;

[0007] S2: Analysis of Knowledge Data Storage Capacity Deviation: Analyze the storage capacity deviation ratios of each sample knowledge data to be stored and the benchmark knowledge data storage capacity deviation ratio to obtain the capacity deviation coefficients of each sample knowledge data to be stored, and perform deviation judgment on them. If the judgment result is that the capacity deviation is normal, further execute S3; if the judgment result is that the capacity deviation is abnormal, then execute S4;

[0008] S3: Analysis of Knowledge Data Storage Performance Optimization: Obtain the storage performance parameters of each sample knowledge data to be stored, analyze to obtain the storage performance optimization coefficients of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data whose judgment result in S2 is normal in terms of capacity deviation;

[0009] S4: Analysis of Knowledge Data Storage Demand Fluctuation: Obtain the storage demand parameters of each sample knowledge data to be stored, analyze to obtain the storage demand fluctuation coefficients of each sample knowledge data to be stored, and match the storage demands of the knowledge data whose judgment result in S2 is abnormal in terms of capacity deviation;

[0010] S5: Conduct Evaluation of Knowledge Data Storage Cost: Analyze the storage costs of each sample knowledge data to be stored, and optimize the data storage method according to the evaluation results;

[0011] S6: Complete Optimization of Knowledge Data Storage: Store each sample knowledge data to be stored based on the data after storage performance optimization, the data after storage demand matching, and the optimized data storage method.

[0012] Preferably, the execution method for obtaining the sample knowledge data to be stored is as follows:

[0013] Collect the new knowledge data to be stored according to a preset collection frequency, mark it as each sample knowledge data to be stored, and number each sample knowledge data to be stored in sequence as 1, 2,..., i,..., n, where i represents the number of each sample knowledge data to be stored, and the preset collection frequency is specifically to collect at a frequency of once every three seconds.

[0014] Preferably, the execution method for the knowledge data storage capacity deviation analysis is as follows:

[0015] Obtain the system storage capacity of the target storage device and the data volume of each sample knowledge data to be stored, and substitute them into the formula , to obtain the storage capacity deviation ratio of the i-th sample knowledge data to be stored, where represents the data volume of the i-th sample knowledge data to be stored, Sc represents the system storage capacity of the target storage device, and Uc represents the system used capacity of the target storage device;

[0016] The data volume of the reference knowledge data is obtained , and substitute it into the formula , and calculate the storage capacity deviation ratio of the reference knowledge data ;

[0017] Calculate the capacity deviation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0018] , where DCD i represents the capacity deviation coefficient of the i-th sample knowledge data to be stored, represents the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data. i represents the number of each sample knowledge data to be stored, and i = 1, 2, 3,..., n;

[0019] Read the capacity deviation coefficients of each sample knowledge data to be stored, judge the capacity deviation of each sample knowledge data to be stored, and compare the capacity deviation coefficients of each sample knowledge data to be stored with the preset capacity deviation coefficient threshold;

[0020] If the capacity deviation coefficient of a certain sample knowledge data to be stored is less than the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of the corresponding sample knowledge data to be stored is normal, generate a storage performance optimization signal, perform storage performance optimization on the corresponding sample knowledge data to be stored, and further execute S3;

[0021] If the capacity deviation coefficient of a certain sample knowledge data to be stored is greater than or equal to the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of the corresponding sample knowledge data to be stored is abnormal, generate a storage demand fluctuation analysis signal, perform storage demand fluctuation analysis on the corresponding sample knowledge data to be stored, and further execute S4.

[0022] Preferably, the method for obtaining the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data is specifically as follows:

[0023] Obtain the data volume of each sample knowledge data to be stored and the data volume of the reference knowledge data, and substitute them into the formula respectively to obtain the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data , where represents the data volume of the i-th sample knowledge data to be stored, represents the data volume of the reference knowledge data.

[0024] Preferably, the execution method of the storage performance optimization analysis of the knowledge data is specifically as follows:

[0025] Obtain the storage performance parameters of each sample knowledge data to be stored, where the storage performance parameters include throughput requirements, request response duration requirements, and compression ratio;

[0026] Calculate the storage performance optimization coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0027] , where POC i represents the storage performance optimization coefficient of the i-th sample knowledge data to be stored, Th i represents the throughput requirement of the i-th sample knowledge data to be stored, represents the preset minimum effective throughput, Rt i represents the request response duration requirement of the i-th sample knowledge data to be stored, represents the preset maximum acceptable request response duration, Co i represents the compression ratio of the i-th sample knowledge data to be stored;

[0028] Read the storage performance optimization coefficients of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data with a normal capacity deviation judgment result in S2 through the storage performance optimization coefficients of each sample knowledge data to be stored.

[0029] Preferably, the execution method of the knowledge data storage demand fluctuation analysis is specifically as follows:

[0030] Record the required storage amount, access frequency, and retention duration of each sample knowledge data to be stored as the storage demand parameters of each sample knowledge data to be stored;

[0031] Read the maximum required storage amount 、minimum required storage amount and average required storage amount of the i-th sample knowledge data to be stored, and substitute them into the formula respectively to calculate the required storage amount change rate Mr i of the i-th sample knowledge data to be stored;

[0032] Calculate the storage demand fluctuation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0033] , where DFC i represents the storage demand fluctuation coefficient of the i-th sample knowledge data to be stored, Mr i represents the required storage amount change rate of the i-th sample knowledge data to be stored, Fr i represents the access frequency change rate of the i-th sample knowledge data to be stored, Rd i represents the retention duration of the i-th sample knowledge data to be stored, represents a preset standard retention duration;

[0034] Read the storage requirement fluctuation coefficients of each sample knowledge data to be stored, and perform storage requirement matching analysis on the knowledge data with abnormal capacity deviation in the judgment result in S2 according to the storage requirement fluctuation coefficients of each sample knowledge data to be stored.

[0035] Preferably, the execution method for evaluating the storage cost of knowledge data is specifically as follows:

[0036] Obtain the storage cost data of each sample knowledge data to be stored;

[0037] Calculate the storage cost evaluation index of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0038] , where CEI i represents the storage cost evaluation index of the i-th sample knowledge data to be stored, represents the actual storage cost value of the j-th storage cost corresponding to the i-th sample knowledge data to be stored, represents the preset benchmark value of the j-th storage cost, represents the weight of the j-th storage cost, and j represents the number of each storage cost, j = 1, 2, 3,..., m;

[0039] Read the storage cost evaluation indexes of each sample knowledge data to be stored, evaluate the storage costs of each sample knowledge data to be stored, compare the storage cost evaluation index of a certain sample knowledge data to be stored with the preset storage cost evaluation index threshold. If the storage cost evaluation index of a certain sample knowledge data to be stored is less than the preset storage cost evaluation index threshold, it is determined that the storage cost control status of this sample knowledge data to be stored is normal. If the storage cost evaluation index of a certain sample knowledge data to be stored is greater than or equal to the preset storage cost evaluation index threshold, it is determined that the storage cost control status of this sample knowledge data to be stored is abnormal. Mark the abnormal data of the storage cost control status of this sample knowledge data to be stored as the storage cost evaluation result of this sample knowledge data to be stored, and select the corresponding data storage method for data storage according to the evaluation result.

[0040] To achieve the above object, the present invention provides the following technical solution: a data storage system with a built-in knowledge base small model, implementing the above data storage method with a built-in knowledge base small model, including: a module for obtaining sample knowledge data to be stored, a module for analyzing the deviation of the storage capacity of knowledge data, a module for optimizing the analysis of the storage performance of knowledge data, a module for analyzing the fluctuation of the storage requirements of knowledge data, a module for evaluating the storage cost of knowledge data, and a module for completing the optimization of the storage of knowledge data;

[0041] Module for obtaining knowledge data of samples to be stored: It is used to collect newly added knowledge data to be stored according to a preset acquisition frequency, mark it as knowledge data of each sample to be stored, and mark the standard storage knowledge data built in the small knowledge base model as reference knowledge data;

[0042] Module for analyzing the deviation of knowledge data storage capacity: It is used to analyze the storage capacity deviation ratios of knowledge data of each sample to be stored and the storage capacity deviation ratio of reference knowledge data, obtain the capacity deviation coefficients of knowledge data of each sample to be stored, and conduct deviation judgment on them. If the judgment result is that the capacity deviation is normal, the module for optimizing the storage performance of knowledge data is further executed. If the judgment result is that the capacity deviation is abnormal, the module for analyzing the fluctuation of knowledge data storage requirements is executed;

[0043] Module for optimizing the storage performance of knowledge data: It is used to obtain the storage performance parameters of knowledge data of each sample to be stored, analyze and obtain the storage performance optimization coefficients of knowledge data of each sample to be stored, and optimize the storage performance of the knowledge data whose judgment result in the module for analyzing the deviation of knowledge data storage capacity is that the capacity deviation is normal;

[0044] Module for analyzing the fluctuation of knowledge data storage requirements: It is used to obtain the storage requirement parameters of knowledge data of each sample to be stored, analyze and obtain the storage requirement fluctuation coefficients of knowledge data of each sample to be stored, and match the storage requirements of the knowledge data whose judgment result in the module for analyzing the deviation of knowledge data storage capacity is that the capacity deviation is abnormal;

[0045] Module for evaluating the storage cost of knowledge data: It is used to analyze the storage costs of knowledge data of each sample to be stored, and optimize the data storage method according to the evaluation results;

[0046] Module for completing the optimization of knowledge data storage: Based on the data after storage performance optimization, the data after storage requirement matching, and the optimized data storage method, the knowledge data of each sample to be stored is stored.

[0047] As described above, the data storage method and system with a built-in small knowledge base model provided by the present invention have at least the following beneficial effects:

[0048] The data storage method and system for the small model with a built-in knowledge base provided by the present invention obtain various sample knowledge data to be stored and reference knowledge data, analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation of the reference knowledge data, obtain the capacity deviation coefficient of each sample knowledge data to be stored, and judge it. According to the judgment result, the data is processed. By adopting this method, knowledge data with abnormal storage capacity can be accurately identified and processed to ensure the efficient operation of the knowledge base. At the same time, this method supports dynamic adjustment of storage strategies to adapt to the continuous changes of the knowledge base, improving the flexibility and scalability of storage. Through the calculated storage performance optimization coefficient, the storage performance of the knowledge data with normal capacity deviation in the judgment result is optimized; through the calculated storage demand fluctuation coefficient, the storage demand matching of the knowledge data with abnormal capacity deviation in the judgment result is carried out; by processing the data with different storage capacity deviation judgment results, not only the storage efficiency is improved, the efficient operation of the knowledge base is ensured, but also the flexible allocation of resources is realized, the utilization rate of storage resources is improved, and resource waste is avoided. Finally, by analyzing the storage cost, the storage cost evaluation index is obtained, and the data storage method is optimized according to the evaluation result, thereby reducing the storage cost, enhancing the accuracy and timeliness of knowledge base update, and improving the user experience and model performance. The present invention can also improve the storage performance, adapt to the dynamic fluctuation of data, and improve the management efficiency, providing a strong guarantee for the development and application of the small model with a built-in knowledge base. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention will be further described with reference to the accompanying drawings. However, the embodiments in the drawings do not constitute any limitation to the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the following drawings.

[0050] Figure 1 It is a schematic flow chart of the data storage method for the small model with a built-in knowledge base of the present invention.

[0051] Figure 2 It is a schematic structural diagram of the data storage system for the small model with a built-in knowledge base of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] Embodiment 1

[0054] Please refer to Figure 1As shown in the figure, the present invention provides a data storage method for a small model with a built-in knowledge base, including the following steps: S1: Obtain sample knowledge data to be stored, S2: Analyze the deviation of the storage capacity of knowledge data, S3: Analyze the optimization of the storage performance of knowledge data, S4: Analyze the fluctuation of the storage requirements of knowledge data, S5: Evaluate the storage cost of knowledge data, and S6: Complete the optimization of the storage of knowledge data;

[0055] S1: Obtain sample knowledge data to be stored: Collect the newly added knowledge data to be stored according to a preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built in the small model of the knowledge base as the reference knowledge data;

[0056] In this embodiment, it should be specifically noted that the execution manner of obtaining the sample knowledge data to be stored is as follows:

[0057] Collect the newly added knowledge data to be stored according to a preset collection frequency, mark it as each sample knowledge data to be stored, and number each sample knowledge data to be stored in sequence, numbered 1, 2,..., i,..., n in turn, where i represents the number of each sample knowledge data to be stored, and the preset collection frequency is specifically to collect according to a collection frequency of once every three seconds.

[0058] S2: Analyze the deviation of the storage capacity of knowledge data: Analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation ratio of the reference knowledge data to obtain the capacity deviation coefficient of each sample knowledge data to be stored, and perform a deviation judgment on it. If the judgment result is that the capacity deviation is normal, further execute S3. If the judgment result is that the capacity deviation is abnormal, then execute S4;

[0059] In this embodiment, it should be specifically noted that the execution manner of analyzing the deviation of the storage capacity of knowledge data is as follows:

[0060] Obtain the system storage capacity of the target storage device and the data volume of each sample knowledge data to be stored, and substitute them into the formula , to obtain the storage capacity deviation ratio of the i-th sample knowledge data to be stored, where represents the data volume of the i-th sample knowledge data to be stored, Sc represents the system storage capacity of the target storage device, and Uc represents the system used capacity of the target storage device;

[0061] Obtain the data volume of the reference knowledge data, substitute it into the formula , and calculate to obtain the storage capacity deviation ratio of the reference knowledge data;

[0062] Calculate the capacity deviation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0063] , where DCD i represents the capacity deviation coefficient of the i-th sample knowledge data to be stored, represents the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data. i represents the number of each sample knowledge data to be stored, and i = 1, 2, 3,..., n;

[0064] Read the capacity deviation coefficients of each sample knowledge data to be stored, judge the capacity deviation of each sample knowledge data to be stored, and compare the capacity deviation coefficients of each sample knowledge data to be stored with a preset capacity deviation coefficient threshold;

[0065] If the capacity deviation coefficient of a certain sample knowledge data to be stored is less than the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of this sample knowledge data to be stored is normal, generate a storage performance optimization signal, perform storage performance optimization on the corresponding sample knowledge data to be stored, and further execute S3;

[0066] If the capacity deviation coefficient of a certain sample knowledge data to be stored is greater than or equal to the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of this sample knowledge data to be stored is abnormal, generate a storage demand fluctuation analysis signal, perform storage demand fluctuation analysis on the corresponding sample knowledge data to be stored, and further execute S4.

[0067] It should be specifically noted that the method for obtaining the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data is as follows:

[0068] Obtain the data volume of each sample knowledge data to be stored and the data volume of the reference knowledge data, and substitute them into the formula to obtain the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data , where represents the data volume of the i-th sample knowledge data to be stored, represents the data volume of the reference knowledge data.

[0069] S3: Knowledge data storage performance optimization analysis: Obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficients of each sample knowledge data to be stored, and perform storage performance optimization on the knowledge data with normal capacity deviation judged in S2;

[0070] In this embodiment, it should be specifically noted that the execution method of the knowledge data storage performance optimization analysis is as follows:

[0071] Obtain the storage performance parameters of each sample knowledge data to be stored, where the storage performance parameters include throughput requirements, request response duration requirements, and compression ratio;

[0072] Calculate the storage performance optimization coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0073] , where POC i represents the storage performance optimization coefficient of the i-th sample knowledge data to be stored, Th i represents the throughput requirement of the i-th sample knowledge data to be stored, represents the preset minimum effective throughput, Rt i represents the request response duration requirement of the i-th sample knowledge data to be stored, represents the preset maximum acceptable request response duration, Co i represents the compression ratio of the i-th sample knowledge data to be stored;

[0074] Read the storage performance optimization coefficients of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data with a normal capacity deviation judgment result in S2 through the storage performance optimization coefficients of each sample knowledge data to be stored;

[0075] It should be specifically noted that in the formula, the larger the throughput requirement Th i of the i-th sample knowledge data to be stored, the smaller the request response duration requirement Rt i , and the smaller the compression ratio Co i , the larger the storage performance optimization coefficient POC i of the i-th sample knowledge data to be stored, indicating that the storage performance of the i-th sample knowledge data to be stored is better. And the throughput requirement, request response duration requirement, and compression ratio of the i-th sample knowledge data to be stored do not affect each other.

[0076] S4: Analysis of storage demand fluctuations of knowledge data: Obtain the storage demand parameters of each sample knowledge data to be stored, analyze the storage demand fluctuation coefficients of each sample knowledge data to be stored, and perform storage demand matching on the knowledge data with an abnormal capacity deviation judgment result in S2;

[0077] In this embodiment, it should be specifically noted that the execution method of the storage demand fluctuation analysis of knowledge data is as follows:

[0078] Record the required storage amount, access frequency, and retention duration of each sample knowledge data to be stored as the storage demand parameters of each sample knowledge data to be stored;

[0079] Read the maximum required storage capacity of the i-th sample knowledge data to be stored , minimum storage required and the average storage required , respectively substitute them into the formula , calculate the required storage capacity change rate Mr of the i-th sample knowledge data to be stored i ;

[0080] Calculate the storage demand fluctuation coefficient of each sample knowledge data to be stored. The calculation formula is as follows:

[0081] , where DFC i represents the storage demand fluctuation coefficient of the i-th sample knowledge data to be stored, Mr i represents the change rate of the storage demand of the i-th sample knowledge data to be stored, Fr i Rd represents the access frequency change rate of the i-th sample knowledge data to be stored. i Indicates the retention time of the i-th sample knowledge data to be stored, Indicates the preset standard retention time;

[0082] Read the storage demand fluctuation coefficient of each sample knowledge data to be stored, and perform storage demand matching analysis on the knowledge data judged as having abnormal capacity deviation in S2 based on the storage demand fluctuation coefficient of each sample knowledge data to be stored;

[0083] It should be noted that in the formula, the retention time Rd of the i-th sample knowledge data to be stored is i The more it deviates from the preset standard retention time, the more storage demand change rate Mr i The larger the access frequency change rate Fr i The larger the value is, the greater the storage demand fluctuation coefficient DFC of the i-th sample knowledge data to be stored. i The larger the value is, the greater the fluctuation of the storage demand of the i-th sample knowledge data to be stored is. And the retention time of the i-th sample knowledge data to be stored, the change rate of the required storage volume and the change rate of the access frequency will not affect each other.

[0084] It should be specifically noted that the calculation formula for the access frequency change rate of the i-th sample knowledge data to be stored is as follows:

[0085] By formula , calculate the access frequency change rate Fr of the i-th sample knowledge data to be stored i ,in, represents the maximum access frequency of the i-th sample knowledge data to be stored, represents the minimum access frequency of the i-th sample knowledge data to be stored, represents the average access frequency of the i-th sample knowledge data to be stored;

[0086] In this embodiment, a specific embodiment is provided as follows:

[0087] Read the storage requirement fluctuation coefficients of each sample knowledge data to be stored. When the storage requirement fluctuation coefficient of the i-th sample knowledge data to be stored , it indicates that the storage requirement of this sample knowledge data belongs to severe fluctuation, and a corresponding storage method requirement matching strategy is generated. Specifically: adopt an elastic storage architecture (such as dynamic expansion of cloud storage), enable multi-level caching for high-access-frequency data, and shorten the storage cycle of data with an over-standard retention duration;

[0088] When , it indicates that the storage requirement of this sample knowledge data belongs to moderate fluctuation, and a corresponding storage method requirement matching strategy is generated. Specifically: regularly monitor the change in storage volume, reserve 10% - 20% elastic space; and allocate storage resources in advance;

[0089] When , it indicates that the storage requirement of this sample knowledge data belongs to mild fluctuation, and a corresponding storage method requirement matching strategy is generated. Specifically: maintain the existing conventional storage strategy.

[0090] S5: Conduct an evaluation of the storage cost of knowledge data: Analyze the storage cost of each sample knowledge data to be stored, and optimize the data storage method according to the evaluation results;

[0091] In this embodiment, it should be specifically noted that the execution method of the above-mentioned evaluation of the storage cost of knowledge data is as follows:

[0092] Obtain the storage cost data of each sample knowledge data to be stored. The storage cost data includes but is not limited to hardware cost, maintenance cost, energy consumption cost, data migration cost, and operation and maintenance cost;

[0093] Calculate the storage cost evaluation index of each sample knowledge data to be stored. The specific calculation formula is as follows:

[0094] , where CEI i represents the storage cost evaluation index of the i-th sample knowledge data to be stored, represents the actual storage cost value of the j-th storage cost corresponding to the i-th sample knowledge data to be stored, represents the preset benchmark value of the j-th storage cost, represents the weight of the j-th storage cost, and j represents the number of each storage cost, j = 1, 2, 3,..., m;

[0095] Read the storage cost evaluation index of each sample knowledge data to be stored, evaluate the storage cost of each sample knowledge data to be stored, compare the storage cost evaluation index of a sample knowledge data to be stored with the preset storage cost evaluation index threshold. If the storage cost evaluation index of a sample knowledge data to be stored is less than the preset storage cost evaluation index threshold, it is determined that the storage cost control status of this sample knowledge data to be stored is normal. If the storage cost evaluation index of a sample knowledge data to be stored is greater than or equal to the preset storage cost evaluation index threshold, it is determined that the storage cost control status of this sample knowledge data to be stored is abnormal. Mark the abnormal data of the storage cost control status of this sample knowledge data to be stored as the storage cost evaluation result of this sample knowledge data to be stored, and screen the corresponding data storage method according to the evaluation result for data storage.

[0096] S6: Complete the optimization of knowledge data storage: Store each sample knowledge data to be stored based on the data optimized for storage performance, the data matched with storage requirements, and the optimized data storage method.

[0097] Embodiment 2

[0098] Please refer to Figure 2 As shown, the present invention provides a data storage system with a built-in knowledge base small model, including a module for acquiring sample knowledge data to be stored, a module for analyzing the deviation of knowledge data storage capacity, a module for optimizing and analyzing the storage performance of knowledge data, a module for analyzing the fluctuation of knowledge data storage requirements, a module for evaluating the storage cost of knowledge data, and a module for completing the optimization of knowledge data storage;

[0099] The module for acquiring sample knowledge data to be stored: It is used to collect the newly added knowledge data to be stored according to the preset acquisition frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built in the knowledge base small model as the reference knowledge data;

[0100] The module for analyzing the deviation of knowledge data storage capacity: It is used to analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation ratio of the reference knowledge data, obtain the capacity deviation coefficient of each sample knowledge data to be stored, and perform deviation judgment on it. If the judgment result is that the capacity deviation is normal, further execute the module for optimizing and analyzing the storage performance of knowledge data. If the judgment result is that the capacity deviation is abnormal, then execute the module for analyzing the fluctuation of knowledge data storage requirements;

[0101] The module for optimizing and analyzing the storage performance of knowledge data: It is used to obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficient of each sample knowledge data to be stored, and perform storage performance optimization on the knowledge data whose judgment result in the module for analyzing the deviation of knowledge data storage capacity is that the capacity deviation is normal;

[0102] Knowledge data storage requirement fluctuation analysis module: It is used to obtain the storage requirement parameters of each sample knowledge data to be stored, analyze and obtain the storage requirement fluctuation coefficients of each sample knowledge data to be stored, and perform storage requirement matching on the knowledge data with abnormal capacity deviation judgment results in the knowledge data storage capacity deviation analysis module;

[0103] Knowledge data storage cost evaluation module: It is used to analyze the storage costs of each sample knowledge data to be stored, and optimize the data storage method according to the evaluation results;

[0104] Completed knowledge data storage optimization module: Store each sample knowledge data to be stored based on the data optimized for storage performance, the data after storage requirement matching, and the optimized data storage method.

[0105] Finally: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0106] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or replacements, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A data storage method for a small model with a built-in knowledge base, characterized in that, It includes the following steps: S1: Obtain the sample knowledge data to be stored: Collect the newly added knowledge data to be stored at a preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built into the knowledge base small model as the reference knowledge data; S2: Analysis of the storage capacity deviation of knowledge data: Analyze the storage capacity deviation ratios of each sample knowledge data to be stored and the storage capacity deviation ratio of the reference knowledge data to obtain the capacity deviation coefficients of each sample knowledge data to be stored, and perform deviation judgment on them. If the judgment result is that the capacity deviation is normal, further execute S3. If the judgment result is that the capacity deviation is abnormal, then execute S4; S3: Analysis of the storage performance optimization of knowledge data: Obtain the storage performance parameters of each sample knowledge data to be stored, analyze to obtain the storage performance optimization coefficients of each sample knowledge data to be stored, and perform storage performance optimization on the knowledge data with normal capacity deviation in S2; S4: Analysis of the storage demand fluctuation of knowledge data: Obtain the storage demand parameters of each sample knowledge data to be stored, analyze to obtain the storage demand fluctuation coefficients of each sample knowledge data to be stored, and perform storage demand matching on the knowledge data with abnormal capacity deviation in S2; S5: Conduct an assessment of the storage cost of knowledge data: Analyze the storage costs of each sample knowledge data to be stored, and optimize the data storage method according to the assessment results; S6: Complete the optimization of knowledge data storage: Store each sample knowledge data to be stored based on the data after storage performance optimization, the data after storage demand matching, and the optimized data storage method.

2. The data storage method of the small model with a built-in knowledge base according to claim 1, wherein: The execution method of obtaining the sample knowledge data to be stored is specifically as follows: Collect the newly added knowledge data to be stored at a preset collection frequency, mark it as each sample knowledge data to be stored, and number each sample knowledge data to be stored in sequence as 1, 2,..., i,..., n, where i represents the number of each sample knowledge data to be stored. The preset collection frequency is specifically to collect at a frequency of once every three seconds.

3. The data storage method of the small model with a built-in knowledge base according to claim 1, characterized in that: The execution method of the storage capacity deviation analysis of knowledge data is specifically as follows: Obtain the system storage capacity of the target storage device and the data volume of each sample knowledge data to be stored, and substitute them into the formula respectively , to obtain the storage capacity deviation ratio of the i-th sample knowledge data to be stored , where represents the data volume of the i-th sample knowledge data to be stored, Sc represents the system storage capacity of the target storage device, and Uc represents the system used capacity of the target storage device; The data volume of the reference knowledge data , substitute it into the formula , and calculate the storage capacity deviation ratio of the reference knowledge data ; Calculate the capacity deviation coefficients of each sample knowledge data to be stored. The specific calculation formula is as follows: , where DCD i represents the capacity deviation coefficient of the i-th sample knowledge data to be stored, represents the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data, and i represents the number of each sample knowledge data to be stored, i = 1, 2, 3,..., n; Read the capacity deviation coefficients of each sample knowledge data to be stored, judge the capacity deviation of each sample knowledge data to be stored, and compare the capacity deviation coefficients of each sample knowledge data to be stored with the preset capacity deviation coefficient threshold; If the capacity deviation coefficient of a certain sample knowledge data to be stored is less than the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of this sample knowledge data to be stored is normal, generate a storage performance optimization signal, perform storage performance optimization on the corresponding sample knowledge data to be stored, and further execute S3; If the capacity deviation coefficient of a certain sample knowledge data to be stored is greater than or equal to the preset capacity deviation coefficient threshold, the judgment result is that the capacity deviation of this sample knowledge data to be stored is abnormal, generate a storage demand fluctuation analysis signal, perform storage demand fluctuation analysis on the corresponding sample knowledge data to be stored, and further execute S4.

4. The data storage method of the small model with a built-in knowledge base according to claim 3, wherein: The method for obtaining the data volume deviation value between the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data is specifically as follows: Obtain the data volumes of each sample knowledge data to be stored and the reference knowledge data, and substitute them into the formula respectively , to obtain the data volume deviation value of the data volume of the i-th sample knowledge data to be stored and the data volume of the reference knowledge data , where represents the data volume of the i-th sample knowledge data to be stored represents the data volume of the reference knowledge data 5. The data storage method of the small model with a built-in knowledge base according to claim 1, characterized in that: The execution method of the storage performance optimization analysis of the knowledge data is specifically as follows: Obtain the storage performance parameters of each sample knowledge data to be stored, where the storage performance parameters include throughput requirements, request response duration requirements, and compression ratio; Calculate the storage performance optimization coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows: , where POC i represents the storage performance optimization coefficient of the i-th sample knowledge data to be stored, Th i represents the throughput requirement of the i-th sample knowledge data to be stored, represents the preset minimum effective throughput, Rt i represents the request response duration requirement of the i-th sample knowledge data to be stored, represents the preset maximum acceptable request response duration, Co i represents the compression ratio of the i-th sample knowledge data to be stored; Read the storage performance optimization coefficient of each sample knowledge data to be stored, and perform storage performance optimization on the knowledge data with normal capacity deviation in the judgment result in S2 through the storage performance optimization coefficient of each sample knowledge data to be stored.

6. The data storage method of the small model with a built-in knowledge base according to claim 1, wherein: The execution method of the storage demand fluctuation analysis of the knowledge data is specifically as follows: Record the required storage volume, access frequency, and retention duration of each sample knowledge data to be stored as the storage demand parameters of each sample knowledge data to be stored; Read the maximum required storage volume of the i-th sample knowledge data to be stored , the minimum required storage volume and the average required storage volume , and substitute them into the formula respectively to calculate the change rate Mr of the required storage volume of the i-th sample knowledge data to be stored i ; Calculate the storage demand fluctuation coefficient of each sample knowledge data to be stored. The specific calculation formula is as follows: , where DFC i represents the storage requirement fluctuation coefficient of the i-th sample knowledge data to be stored, Mr i represents the change rate of the required storage amount of the i-th sample knowledge data to be stored, Fr i represents the change rate of the access frequency of the i-th sample knowledge data to be stored, Rd i represents the retention duration of the i-th sample knowledge data to be stored, represents the preset standard retention duration; Read the storage demand fluctuation coefficient of each sample knowledge data to be stored, and perform storage demand matching analysis on the knowledge data with abnormal capacity deviation in the judgment result in S2 through the storage demand fluctuation coefficient of each sample knowledge data to be stored.

7. The data storage method of the small model with a built-in knowledge base according to claim 1, characterized in that: The execution method of performing storage cost evaluation of the knowledge data is specifically as follows: Obtain the storage cost data of each sample knowledge data to be stored; Calculate the storage cost evaluation index of each sample knowledge data to be stored. The specific calculation formula is as follows: , where CEI i represents the storage cost evaluation index of the i-th sample knowledge data to be stored, represents the actual value of the storage cost of the j-th storage cost corresponding to the i-th sample knowledge data to be stored, represents the preset reference value of the j-th storage cost, represents the weight of the j-th storage cost, where j represents the number of each storage cost, and j = 1, 2, 3,..., m; Read the storage cost evaluation index of each sample knowledge data to be stored, evaluate the storage cost of each sample knowledge data to be stored, compare the storage cost evaluation index of a certain sample knowledge data to be stored with the preset storage cost evaluation index threshold. If the storage cost evaluation index of a certain sample knowledge data to be stored is less than the preset storage cost evaluation index threshold, it is judged that the storage cost control status of this sample knowledge data to be stored is normal. If the storage cost evaluation index of a certain sample knowledge data to be stored is greater than or equal to the preset storage cost evaluation index threshold, it is judged that the storage cost control status of this sample knowledge data to be stored is abnormal. Mark the abnormal data of the storage cost control status of this sample knowledge data to be stored as the storage cost evaluation result of this sample knowledge data to be stored, and select the corresponding data storage method for data storage according to the evaluation result.

8. A data storage system for a small model with a built-in knowledge base, which is used to implement the data storage method for a small model with a built-in knowledge base according to any one of the above claims 1-7, characterized in that, Including: Module for obtaining sample knowledge data to be stored: used to collect new knowledge data to be stored at a preset collection frequency, mark it as each sample knowledge data to be stored, and mark the standard storage knowledge data built in the knowledge base small model as the reference knowledge data; Knowledge data storage capacity deviation analysis module: used to analyze the storage capacity deviation ratio of each sample knowledge data to be stored and the storage capacity deviation ratio of the reference knowledge data, obtain the capacity deviation coefficient of each sample knowledge data to be stored, and perform deviation judgment on it. If the judgment result is normal capacity deviation, further execute the knowledge data storage performance optimization analysis module. If the judgment result is abnormal capacity deviation, then execute the knowledge data storage demand fluctuation analysis module; Knowledge data storage performance optimization analysis module: It is used to obtain the storage performance parameters of each sample knowledge data to be stored, analyze and obtain the storage performance optimization coefficient of each sample knowledge data to be stored, and optimize the storage performance of the knowledge data with a normal capacity deviation judgment result in the knowledge data storage capacity deviation analysis module; Knowledge data storage demand fluctuation analysis module: It is used to obtain the storage demand parameters of each sample knowledge data to be stored, analyze and obtain the storage demand fluctuation coefficient of each sample knowledge data to be stored, and match the storage demand of the knowledge data with an abnormal capacity deviation judgment result in the knowledge data storage capacity deviation analysis module; Knowledge data storage cost evaluation module: It is used to analyze the storage cost of each sample knowledge data to be stored and optimize the data storage method according to the evaluation result; Knowledge data storage optimization completion module: Store each sample knowledge data to be stored based on the data after storage performance optimization, the data after storage demand matching, and the optimized data storage method.

Images