Intelligent hierarchical storage system for database files
By using a decision tree module and information gain calculation, the classification of database files is dynamically adjusted, solving the problem of chaotic file priority control in existing technologies. This achieves efficient file storage differentiation and real-time adjustment, thereby improving database management efficiency.
Patent Information
- Application Number
- CN202411289841.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-14
AI Technical Summary
The lack of automated and intelligent mechanisms in existing technologies to regulate the priority of files in the database leads to chaotic file classification and management, especially in the case of massive amounts of data, resulting in storage disorder and affecting data retrieval efficiency.
A decision tree module is used to build an intelligent hierarchical storage system for database files. By setting the call frequency and storage time baseline, the system dynamically adjusts file classification using information gain and empirical entropy calculations, and automatically divides high-speed and low-speed storage areas.
It enables intelligent and real-time adjustment of database files, improves partitioning efficiency, reduces labor costs, and ensures that files are stored reasonably according to their importance.
Smart Images

Figure CN118796840B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to information processing technology, and more specifically, to an intelligent hierarchical storage system for database files. Background Art
[0002] As we all know, databases can be used to store and call a large number of files. Large databases store a huge number of files. However, these different files stored in the database often have different statuses.
[0003] Files in a database are often prioritized by frequency of use. Over a period of time, files with a higher frequency of use are likely to become the most sensitive files in the database and should also be stored in the database at a priority level.
[0004] In addition to the call frequency, you may also need to consider the storage time of the file. It is generally believed that the files recently stored in the database are more important than the "old" files in the database at the current moment. Therefore, the storage time is also an important indicator for determining the priority of files in the database.
[0005] In other words, both the call frequency and storage time may affect the status of files in the database. However, in reality, people often rely on experience to infer the importance of related files, and do not set up an effective mechanism to automatically and intelligently control the priority of files in the database. This will cause great confusion in the classification and management of files in the database. Once the database storage volume reaches a massive level, it will bring great disorder to the database storage and cause great inconvenience in data call. Summary of the invention
[0006] The present invention provides a database file intelligent hierarchical storage system, which effectively solves many defects in the prior art.
[0007] Specifically, the present invention provides an intelligent hierarchical storage system for database files, which includes a database, a decision tree module, and a judgment scheduling module. The database is divided into a high-speed storage area and a low-speed storage area to form a high-speed and low-speed storage classification of files, and a call frequency reference value A0 and a storage time reference value B0 are set. In the call frequency classification, files with a call frequency higher than the call frequency reference value A0 are classified as high-frequency files, and files with a call frequency not higher than the call frequency reference value A0 are classified as low-frequency files; in the storage time classification, files with a storage time greater than the storage time reference value B0 are classified as old files, and files with a storage time not greater than the storage time reference value B0 are classified as new files. At an initial time T0, all files in the database form a 0th data set M0. The decision tree module is based on the 0th data set M0, with the call frequency classification as the root node and the high and low speed storage classification as the leaf node, to construct a call frequency decision tree at time T0, and calculate the information gain θA0 of the call frequency at time T0; then based on the 0th data set M0, with the storage time classification as the root node and the high and low speed storage classification as the leaf node, Construct a storage time decision tree at time T0, calculate the information gain θB0 of the storage time at time T0, compare the information gains θA0 and θB0, select the classification corresponding to the maximum value of the information gains θA0 and θB0 as the indicator classification J0 at time T0, and evolve from time T0 to time T1. All files in the database are updated to the first data set M1. The decision tree module is based on the first data set M1, with the call frequency classification and storage time classification as the root node, and the high and low speed storage classification as the leaf nodes, respectively. Draw the call frequency decision tree at time T1 and the storage time decision tree at time T1, calculate the information gain θA1 of the call frequency at time T1 and the information gain θB1 of the storage time respectively, compare the information gains θA1 and θB1, select the classification corresponding to the maximum value of the information gains θA1 and θB1 as the index classification J1 at time T1, determine whether the index classifications J0 and J1 are consistent with each other by the scheduling module, if they are consistent, they form the same index classification, and divide the first data set M1 into high-speed storage area and low-speed storage area based on the same index classification.
[0008] Preferably, at time T0, the decision tree module calculates the empirical entropy H0 (D) at time T0 based on the 0th data set M0 with high and low speed storage classification. The decision tree module constructs a call frequency decision tree at time T0 based on the 0th data set M0, calculates the empirical conditional entropy H0 (D|A) about the call frequency classification, and the information gain θA0 = H0 (D) - H0 (D|A) of the call frequency classification at time T0 is obtained by subtracting the empirical conditional entropy H0 (D) from the empirical entropy H0 (D) at time T0.
[0009] Preferably, the decision tree module constructs a storage time decision tree at time T0 based on the 0th data set, calculates the empirical conditional entropy H0(D|B) about the storage time classification, and subtracts the empirical conditional entropy H0(D|B) from the empirical entropy H0(D) at time T0 to obtain the information gain θB0= H0(D)-H0(D|B) of the storage time classification at time T0.
[0010] Preferably, at time T0, if θA0≥θB0, the call frequency classification corresponding to the information gain θA0 is selected as the indicator classification J0; if θA0<θB0, the storage time classification corresponding to the information gain θB0 is selected as the indicator classification J0.
[0011] Preferably, at time T1, the decision tree module calculates the empirical entropy H1 (D) at time T1 based on the first data set M1 with high and low speed storage classification. The decision tree module constructs a call frequency decision tree at time T1 based on the first data set M1, and calculates the empirical conditional entropy H1 (D|A) about the call frequency classification. The information gain θA1 = H1 (D) - H1 (D|A) of the call frequency classification at time T1 is obtained by subtracting the empirical conditional entropy H1 (D|A) from the empirical entropy H1 (D) at time T1.
[0012] Preferably, the decision tree module constructs a storage time decision tree at time T1 based on the first data set, calculates the empirical conditional entropy H1 (D|B) about the storage time classification, and subtracts the empirical conditional entropy H1 (D|B) from the empirical entropy H1 (D) at time T1 to obtain the information gain θB1 = H1 (D) - H1 (D|B) of the storage time classification at time T1.
[0013] Preferably, at time T1, if θA1≥θB1, the call frequency classification corresponding to the information gain θA1 is selected as the indicator classification J1; if θA1<θB1, the storage time classification corresponding to the information gain θB1 is selected as the indicator classification J1.
[0014] Preferably, the total data set of the database at any time is collectively referred to as the total data set M, the total number of files in the total data set M is set to D, the number of files in the high-speed storage area is C1, and the number of files in the low-speed storage area is C2, then the calculation formula of the empirical entropy H (D) is:
[0015] The empirical conditional entropy in the call frequency decision tree at any time is collectively referred to as H (D|A), and its calculation formula is:
[0016]
[0017] Among them, parameter A i The integer i takes the values 1 and 2 respectively, so the parameter A iIn the summation formula, it is expressed as A1 and A2, where A1 represents the number of high-frequency files in the data set M, and A2 represents the number of low-frequency files in the data set M. The parameter A ik In the formula, the integer i takes the values 1 and 2 respectively, and the integer k also takes the values 1 and 2 respectively. According to the values of i and k, four parameters A are formed. 11 , A 12 , A 21 , A 22 , A 11 Indicates the number of high-frequency files in the high-speed storage area, A 12 Indicates the number of high-frequency files in the low-speed storage area, A 21 Indicates the number of low-frequency files in the high-speed storage area, A 22 It represents the number of low-frequency files in the low-speed storage area. The empirical conditional entropy in the storage time decision tree at any time is collectively referred to as H (D|B), and its calculation formula is:
[0018]
[0019] Among them, parameter B i’ The integer i' takes the values 1 and 2 respectively, and the parameter B i’ In the summation formula, it is expressed as B1 and B2, where B1 represents the number of old files in the data set M, and B2 represents the number of new files in the data set M. The parameter B i’k’ In the formula, the integer i' takes the values 1 and 2 respectively, and the integer k' also takes the values 1 and 2 respectively. Therefore, according to the values of i' and k', four parameters B are formed. 11 , B 12 , B 21 , B 22 , B 11 Indicates the number of old files in the high-speed storage area, B 12 Indicates the number of old files in the slow storage area, B 21 Indicates the number of new files in the high-speed storage area, B 22 Indicates the number of new files in the slow storage area.
[0020] In summary, the present invention provides a database file intelligent hierarchical storage system, which performs two-level decision tree classification on the data set at each moment under the classification concept of the decision tree, and compares the classification at different moments using the classification-related characteristics of information gain, thereby dividing the current data set into high-speed and low-speed storage areas in the most suitable way at the moment. The present invention completely gets rid of the drawbacks of dividing files completely relying on manual experience in the past by intelligent and automated means, and can adjust the partitions in real time, greatly improving the efficiency of database partitioning and saving labor costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be discussed below. Obviously, the technical solutions described in conjunction with the drawings are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments and their drawings can be obtained based on the embodiments shown in these drawings without paying creative work.
[0022] Figure 1 The operating flow chart of the database file intelligent hierarchical storage system according to the present invention is shown. DETAILED DESCRIPTION
[0023] The technical solutions of various embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments described in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0024] In summary, the present invention provides a database file intelligent hierarchical storage system, which uses decision trees to build a database classification mechanism, and judges the correlation of different types of data based on empirical conditional entropy, and intelligently regulates the distribution and storage of different data in the database to different areas in the database based on this correlation. Figure 1 The specific operation of the system provided by the present invention is described in detail.
[0025] Figure 1 The operating flow chart of the database file intelligent hierarchical storage system according to the present invention is shown.
[0026] In the database constructed by this system, "important" files are stored in high-speed storage areas (such as flash drives, SSDs, etc.), and "unimportant" files are stored in low-speed storage areas. In other words, the database is divided into high-speed storage areas and low-speed storage areas according to the importance of the stored files.
[0027] Generally speaking, the high-speed storage area is much smaller in space than the low-speed storage area for the convenience of users.
[0028] However, the so-called "important" and "unimportant" documents must not be divided based on experience, otherwise we will fall into the dilemma of existing technology.
[0029] However, the dilemma in the division of "important" / "unimportant" lies in that the division of importance should be distinguished at least according to two dimensions: call frequency and storage time. In fact, there is no clear technical standard to indicate which of these two dimensions is more critical. Additionally, it is even more difficult that for the same file, the call frequency and storage time will change over time. For example, a file just stored in the database today is a "new" file in terms of storage time, but it will become an "old" file after a period of time (such as one month or two months). For the same file in the database, if its call frequency is relatively high during this period and becomes sparse in the next period, then for this file, the dimension of "call frequency" will also change over time, which also increases the difficulty of classifying files in the database.
[0030] The technical concept of the system provided by the present invention lies in that as historical events evolve, a decision tree is used to gradually improve the planning of data files in the database.
[0031] At the initial T0 moment, the decision tree module of the system takes all files in the database as the 0th total data set M0, and takes the high-speed storage area and the low-speed storage area as the high-low speed storage classification, and calculates the empirical entropy H0(D) at the T0 moment.
[0032] Then, still at the initial T0 moment, a call frequency reference value A0 and a storage time reference value B0 are set. Then, classified by call frequency, among the files in the 0th total data set within the time period T before the T0 moment, files with a call frequency higher than the call frequency reference value A0 are classified as high-frequency files, and conversely, files with a call frequency not higher than the call frequency reference value A0 are classified as low-frequency files.
[0033] For example, if the call frequency reference value A0 is set to 3 times per month and the time period is set to 2 months, and a certain file in the database has been called 5 times in the past 2 months at the T0 moment, then the call frequency is 5 / 2 = 2.5 times per month < A0, so this file is classified as a low-frequency file; if a certain file has been called 8 times in the past 2 months at the T0 moment, then the call frequency is 8 / 2 = 4 times per month > A0, so this file is classified as a high-frequency file.
[0034] Classified by storage time, among the files in the 0th total data set, files with a storage time greater than the storage time reference value B0 are classified as old files, and conversely, files with a storage time not greater than the storage time reference value B0 are classified as new files.
[0035] For example, if the storage time reference value B0 is set to 6 months, then at time T0, a file has been stored in the database for 1 year, and the storage time of the file is greater than the storage time reference value B0, and the file is classified as an old file. If at time T0, a file has been stored in the database for only 3 months, and the storage time of the file is less than the storage time reference value B0, the file is classified as a new file.
[0036] Therefore, there are two dimensions of classification for the total data set, namely, call frequency classification and storage time classification.
[0037] The decision tree module is based on the total data set 0, with the call frequency classification as the root node and the high and low speed storage classification as the leaf nodes, to construct a call frequency decision tree at time T0. Based on the call frequency decision tree at time T0, the empirical conditional entropy H0 (D|A) can be calculated.
[0038] Subtracting the empirical entropy H0(D) from the empirical conditional entropy H0(D|A) at time T0 gives the information gain θA0= H0(D)-H0(D|A) of the call frequency at time T0.
[0039] In parallel, the decision tree module builds a storage time decision tree at time T0 based on the total data set 0, with the storage time classification as the root node and the high and low speed storage classification as the leaf nodes. Based on the storage time decision tree at time T0, the empirical conditional entropy H0 (D|B) can be calculated.
[0040] Subtracting the empirical conditional entropy H0(D|B) from the empirical entropy H0(D) at time T0 gives the information gain θB0= H0(D)-H0(D|B) of the storage time at time T0.
[0041] Compare the information gains θA0 and θB0, and select the classification corresponding to the maximum value of the information gains θA0 and θB0 as the indicator classification J0 at time T0.
[0042] For example, at time T0, if θA0≥θB0, the call frequency category corresponding to the larger information gain θA0 of the two information gains is used as the indicator category J0. On the contrary, if θA0<θB0, the storage time category corresponding to the larger information gain θB0 of the two information gains is used as the indicator category J0.
[0043] From time T0 to time T1, since the database will inevitably have files added or deleted after time T, the database data set is updated from the 0th data set M0 to the 1st data set M1. The call frequency reference value A0 and the storage time reference value B0 are not updated.
[0044] In addition, during the period from time T0 to time T1, the calling frequency and storage time of files will also change accordingly. Some files will transition from high-frequency files to low-frequency files, and the storage time of all original files will be lengthened.
[0045] Similar to the above moment T0, at moment T1, the decision tree module of the system uses all files in the database as the first data set M1, and high-speed storage area and low-speed storage area as high-speed and low-speed storage classifications, and calculates the empirical entropy H1 (D) at moment T1.
[0046] The decision tree module is based on the first data set M1, with the call frequency classification as the root node and the high and low speed storage classification as the leaf nodes, to construct a call frequency decision tree at time T1. Based on the call frequency decision tree at time T1, the empirical conditional entropy H1 (D|A) can be calculated.
[0047] Subtracting the empirical entropy H1(D) from the empirical conditional entropy H1(D|A) at time T1 gives the information gain θA1= H1(D)-H1(D|A) of the call frequency at time T1.
[0048] In parallel, the decision tree module constructs a storage time decision tree at time T1 based on the first data set, with the storage time classification as the root node and the high and low speed storage classification as the leaf nodes. Based on the storage time decision tree at time T1, the empirical conditional entropy H1 (D|B) can be calculated.
[0049] Subtracting the empirical entropy H1(D) from the empirical conditional entropy H1(D|B) at time T1 gives the information gain θB1= H1(D)-H1(D|B) of the call frequency at time T1.
[0050] Compare the information gains θA1 and θB1, and select the category corresponding to the maximum value of the information gains θA1 and θB1 as the indicator category J1 at time T1.
[0051] At this point, the system provided by the present invention starts the judgment scheduling module. The judgment scheduling module judges whether the index classifications J0 and J1 are consistent. If the two index classifications are consistent and thus form the same index classification, the first data set M1 is divided into a high-speed storage area and a low-speed storage area based on the same index classification.
[0052] For example, if the same indicator classification is storage time classification, then the files in the first data set M1 whose storage time is greater than the storage time reference value B0 will all be classified as "old" files and belong to the low-speed storage area; conversely, the files in the first data set M1 whose storage time is less than or equal to the storage time reference value B0 will all be classified as "new" files and belong to the high-speed storage area.
[0053] If the same indicator classification is a call frequency classification, the files in the first data set M1 whose call frequency is greater than the call frequency reference value A0 will all be classified as high-frequency files and belong to the high-speed storage area; conversely, the files in the first data set M1 whose call frequency is less than or equal to the call frequency reference value A0 will all be classified as low-frequency files and belong to the low-speed storage area.
[0054] If the scheduling module determines that the index categories J0 and J1 are inconsistent, the distribution of the first data set M1 in the high-speed storage area and the low-speed storage area at the current time T1 is maintained unchanged. In other words, no operation is currently required on the database.
[0055] Time further evolves. From time T1 to time T2 after a time period T, the data set in the database is updated to the second data set M2, and the decision tree module calculates the indicator classification J2 accordingly. Similarly, the judgment scheduling module determines whether the indicator classifications J1 and J2 are consistent. If the two indicator classifications are consistent and thus form the same indicator classification, the second data set M2 is divided into a high-speed storage area and a low-speed storage area based on the same indicator classification.
[0056] As time goes by, based on the above steps, the same operation can be performed at T3, T4 and other times in the future.
[0057] The process of the present invention has been basically introduced here. Here, the calculation of the above-mentioned empirical entropy and empirical conditional entropy can also be introduced. Since the methods of classifying the data set at the corresponding time and calculating the relevant entropy are completely consistent at each time node (for example, T0, T1, T2, etc.), they are introduced uniformly here.
[0058] The total data set of the database at any time is collectively referred to as the data set M. The total number of files in the data set M is set to D, the number of files in the high-speed storage area is C1, and the number of files in the low-speed storage area is C2. The calculation formula of the empirical entropy H (D) is:
[0059] The empirical conditional entropy in the call frequency decision tree at any time is collectively referred to as H (D|A), and its calculation formula is:
[0060]
[0061] Among them, in the above summation formula, parameter A i The integer i takes the values 1 and 2 respectively. In other words, the parameter A i In fact, it is expressed as A1 and A2 in the summation formula, where A1 represents the number of high-frequency files in the data set M whose call frequency is higher than the call frequency benchmark value, and A2 represents the number of low-frequency files in the data set M whose call frequency is not higher than the call frequency benchmark value. ikIn the formula, the integer i takes the values 1 and 2 respectively, and the integer k also takes the values 1 and 2 respectively. Therefore, according to the values of i and k, four parameters A are formed. 11 , A 12 , A 21 , A 22 . A 11 Indicates the number of high-frequency files in the high-speed storage area, A 12 Indicates the number of high-frequency files in the low-speed storage area, A 21 Indicates the number of low-frequency files in the high-speed storage area, A 22 Indicates the number of low-frequency files in the low-speed storage area.
[0062] Similarly, the empirical conditional entropy in the storage time decision tree at any time is collectively referred to as H(D|B), and its calculation formula is:
[0063]
[0064] Among them, in the above summation formula, parameter B i’ The integer i' takes the values 1 and 2 respectively. In other words, the parameter B i’ In fact, it is expressed as B1 and B2 in the summation formula, where B1 represents the number of old files in the data set M whose storage time is greater than the storage time reference value B0, and B2 represents the number of new files in the data set M whose storage time is not greater than the storage time reference value B0. Parameter B i’k’ In the formula, the integer i' takes the values 1 and 2 respectively, and the integer k' also takes the values 1 and 2 respectively. Therefore, according to the values of i' and k', four parameters B are formed. 11 , B 12 , B 21 , B 22 . B 11 Indicates the number of old files in the high-speed storage area, B 12 Indicates the number of old files in the slow storage area, B 21 Indicates the number of new files in the high-speed storage area, B 22 Indicates the number of new files in the slow storage area.
[0065] The present invention has been basically introduced here. In summary, the present invention provides an intelligent hierarchical storage system for database files. Under the classification concept of decision trees, a two-level decision tree classification is performed on the data set at each moment, and the classification correlation characteristics of information gain are used to compare the classifications at different moments, so as to divide the current data set into high-speed and low-speed storage areas in the most suitable way at the moment. The present invention completely gets rid of the drawbacks of dividing files completely relying on manual experience in the past by intelligent and automated means, and can adjust the partitions in real time, greatly improving the efficiency of database partitioning and saving labor costs.
[0066] The above description is only an exemplary embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. An intelligent hierarchical storage system for database files, It is characterized in that The system includes a database, a decision tree module, and a judgment scheduling module. The database is divided into a high-speed storage area and a low-speed storage area to form a high-speed and low-speed storage classification of files, set a call frequency reference value A0 and a storage time reference value B0, In the call frequency classification, files with a call frequency higher than the call frequency reference value A0 are classified as high-frequency files, and files with a call frequency not higher than the call frequency reference value A0 are classified as low-frequency files; in the storage time classification, files with a storage time greater than the storage time reference value B0 are classified as old files, and files with a storage time not greater than the storage time reference value B0 are classified as new files, where: At the initial time T0, all files in the database form the 0th data set M0. The decision tree module is based on the 0th data set M0, with the call frequency classification as the root node and the high and low speed storage classification as the leaf nodes. The call frequency decision tree at time T0 is constructed and the information gain θA of the call frequency at time T0 is calculated. 0 ; Based on the 0th data set M0, the storage time classification is used as the root node, and the high and low speed storage classification is used as the leaf node to construct the storage time decision tree at time T0, and calculate the information gain θB of the storage time at time T0 0 , compare the information gain θA 0 and θB 0 The size of the information gain θA is selected 0 and θB 0 The category corresponding to the maximum value in is taken as the index category J0 at time T0. From time T0 to time T1, all files in the database are updated to the first data set M1. The decision tree module is based on the first data set M1, with the call frequency classification and storage time classification as the root node, and the high and low speed storage classification as the leaf node. The call frequency decision tree at time T1 and the storage time decision tree at time T1 are constructed respectively, and the information gain θA of the call frequency at time T1 is calculated respectively. 1 and the information gain θB of the storage time 1 , compare the information gain θA 1 and θB 1 The size of the information gain θA is selected 1 and θB 1 The category corresponding to the maximum value in is taken as the index category J1 at time T1. The scheduling module determines whether the indicator classifications J0 and J1 are consistent. If they are consistent, they form the same indicator classification. Based on the same indicator classification, the first data set M1 is divided into a high-speed storage area and a low-speed storage area.
2. The system according to claim 1, It is characterized in that At time T0, the decision tree module calculates the empirical entropy H at time T0 based on the total data set M0 of the 0th data set by high and low speed storage classification. 0 (D), The decision tree module builds a call frequency decision tree at time T0 based on the total data set M0 of the 0th time, and calculates the empirical conditional entropy H about the call frequency classification 0 (D|A), The empirical entropy H at time T0 0 (D) Subtract the empirical conditional entropy H 0 (D|A) gives the information gain θA of the call frequency classification at time T0 0 = H 0 (D)-H 0 (D|A).
3. The system according to claim 2, It is characterized in that The decision tree module builds a storage time decision tree at time T0 based on the total set of data 0, and calculates the empirical conditional entropy H for storage time classification. 0 (D|B), The empirical entropy H at time T0 0 (D) Subtract the empirical conditional entropy H 0 (D|B) gives the information gain θB of the storage time classification at time T0 0 = H 0 (D)-H 0 (D|B).
4. The system according to claim 3, It is characterized in that At time T0, if θA 0 ≥θB 0 , then select the information gain θA 0 The corresponding call frequency classification is used as the indicator classification J0; if θA 0 <θB 0 , then select the information gain θB 0 The corresponding storage time classification is used as the index classification J0.
5. The system according to claim 1, It is characterized in that At time T1, the decision tree module calculates the empirical entropy H at time T1 based on the first data set M1 and stores the classification at high and low speeds. 1 (D), The decision tree module builds a call frequency decision tree at time T1 based on the first data set M1 and calculates the empirical conditional entropy H for the call frequency classification. 1 (D|A), The empirical entropy H at time T1 1 (D) Subtract the empirical conditional entropy H 1 (D|A) gives the information gain θA of the call frequency classification at time T1 1 = H 1 (D)-H 1 (D|A).
6. The system according to claim 5, It is characterized in that The decision tree module builds a storage time decision tree at time T1 based on the first data set and calculates the empirical conditional entropy H for storage time classification. 1 (D|B), The empirical entropy H at time T1 1 (D) Subtract the empirical conditional entropy H 1 (D|B) gives the information gain θB of the storage time classification at time T1 1 = H 1 (D)-H 1 (D|B).
7. The system according to claim 6, It is characterized in that At time T1, if θA 1 ≥θB 1 , then select the information gain θA 1 The corresponding call frequency classification is used as the indicator classification J1; if θA 1 <θB 1 , then select the information gain θB 1 The corresponding storage time classification is used as the index classification J1.
8. The system according to any one of claims 2 to 7, It is characterized in that The total data set of the database at any time is collectively referred to as the data set M. The total number of files in the data set M is set to D, and the number of files in the high-speed storage area is C. 1 , the number of files in the low-speed storage area is C 2 , then the calculation formula of empirical entropy H (D) is: , The empirical conditional entropy in the call frequency decision tree at any time is collectively referred to as H (D|A), and its calculation formula is: , Among them, parameter A i The integer i takes the values 1 and 2 respectively, so the parameter A i Form two parameters A 1 and A 2 , A 1 A represents the number of high-frequency files in the total data set M. 2 Indicates the number of low-frequency files in the total data set M, parameter A ik In the formula, the integer i takes the values 1 and 2 respectively, and the integer k also takes the values 1 and 2 respectively. According to the values of i and k, four parameters A are formed. 11 , A 12 , A 21 , A 22 , A 11 Indicates the number of high-frequency files in the high-speed storage area, A 12 Indicates the number of high-frequency files in the low-speed storage area, A 21 Indicates the number of low-frequency files in the high-speed storage area, A 22 Indicates the number of low-frequency files in the low-speed storage area. The empirical conditional entropy in the storage time decision tree at any time is collectively referred to as H (D|B), and its calculation formula is: , Among them, parameter B i’ The integer i' takes the values 1 and 2 respectively, and the parameter B i’ Form two parameters B 1 and B 2 , B 1 represents the number of old files in the total data set M, B 2 Indicates the number of new files in the total data set M, parameter B i’k’ In the formula, the integer i' takes the values 1 and 2 respectively, and the integer k' also takes the values 1 and 2 respectively. Therefore, according to the values of i' and k', four parameters B are formed. 11 , B 12 , B 21 , B 22 , B 11 Indicates the number of old files in the high-speed storage area, B 12 Indicates the number of old files in the slow storage area, B 21 Indicates the number of new files in the high-speed storage area, B 22 Indicates the number of new files in the slow storage area.
Citation Information
Patent Citations
Novel efficient filtering method for accelerating similar text search
CN115840810A
Distributed intelligent warehouse scheduling system based on artificial intelligence
CN117910921A
