Hierarchical storage method and device of data file, equipment and medium

Abstract

The invention relates to the technical field of cloud storage, can be applied to business system platforms of financial science and technology, medical treatment and health and the like, and discloses a hierarchical storage method, device and equipment for data files and a medium, and the method comprises the following steps: acquiring the data files in a storage unit, and extracting data characteristic parameters in the data files; performing multi-dimensional quantitative analysis on the data feature parameters to obtain a dimensional feature analysis value; identifying a dynamic weight of the data feature parameter, and performing weighted calculation on the data feature parameter according to the dimension feature analysis value and the dynamic weight to obtain a dynamic storage analysis value; identifying a data storage type label according to the dynamic storage analysis value; matching the data storage type label with strategies in a storage strategy library to obtain a target storage strategy; determining a storage hierarchy according to the target storage strategy; a storage address of the data file in the storage hierarchy is identified and the data file is stored in the storage address. The accuracy of hierarchical storage of the data files can be improved.

Description

Technical Field

[0001] This invention relates to the field of cloud storage technology, and in particular to a method, apparatus, device, and medium for hierarchical storage of data files. Background Technology

[0002] In big data applications, HDFS, as the core storage system, needs to handle massive amounts of data, including hot data for real-time computing, warm data for regular business access, and cold data for historical archives. Different types of data have varying requirements for storage latency and capacity support. Meanwhile, media such as NVMe SSDs, HDDs, and object storage each have their own performance and cost characteristics, and data access frequency and recent access time dynamically change with business development. Efficient storage needs to be adapted to these objective conditions. Therefore, to meet the storage adaptation and dynamic management needs of different types of data, the HDFS storage architecture and data tiering mechanism need to be optimized to improve the accuracy of hierarchical data file storage.

[0003] In the healthcare field, a hospital uses the HDFS storage system to manage data such as electronic medical records, medical images, and historical medical records. Existing technology does not differentiate based on data access characteristics, employing homogeneous storage media and a copy strategy for real-time outpatient medical records, periodically reviewed inpatient images, and long-term archived medical records from many years ago. It fails to perform hierarchical evaluation based on data access frequency, recent access time, and file size, ultimately resulting in low accuracy in hierarchical data file storage.

[0004] In the fintech business, a financial institution uses the HDFS storage system to manage its fintech business data, which includes real-time transaction logs, daily reconciliation files, and risk control audit files from many years ago. Existing technology does not differentiate based on data access characteristics, using homogeneous storage media and a uniform replication strategy for frequently accessed real-time transaction logs, periodically retrieved reconciliation files, and infrequently archived risk control files. It fails to consider factors such as the most recent access time and file size differences for hierarchical evaluation, ultimately resulting in low accuracy in the tiered storage of data files.

[0005] In existing technologies, after acquiring data files within a storage unit, only a single data feature parameter is extracted, failing to achieve multi-dimensional feature coverage. When performing quantitative analysis on feature parameters, there is a lack of scientific algorithm support, making it impossible to obtain accurate dimensional feature analysis values. Furthermore, dynamic weights are not identified; dynamic storage analysis values ​​are calculated only through simple numerical superposition. When identifying storage type labels, fixed threshold judgments are relied upon. When matching preset storage strategies, different media and replication strategies are not adapted. There is also no transparent mechanism to automatically adjust storage configurations during data migration, resulting in low accuracy of hierarchical storage of data files. Summary of the Invention

[0006] This invention provides a method, apparatus, device, and medium for hierarchical storage of data files to solve the problem of low accuracy in hierarchical storage of data files.

[0007] Firstly, a hierarchical storage method for data files is provided, including: Obtain the data file from the storage unit and extract the data feature parameters from the data file; Perform multi-dimensional quantitative analysis on the data feature parameters to obtain the dimensional feature analysis values ​​corresponding to the data feature parameters; Identify the dynamic weights corresponding to the data feature parameters, and perform weighted calculations on the data feature parameters based on the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values; Identify the data storage type tag corresponding to the data file based on the dynamic storage analysis value; The data storage type label is matched with the policies in the preset storage policy library to obtain the target storage policy for the data file; The storage level of the data file is determined according to the target storage strategy; The storage address of the data file in the storage level is identified using a preset transparent migration mechanism, and the data file is stored in the storage address.

[0008] Secondly, a hierarchical storage device for data files is provided, comprising: The data feature parameter extraction module is used to obtain data files in the storage unit and extract data feature parameters from the data files. The dimension feature analysis value acquisition module is used to perform multi-dimensional quantitative analysis on the data feature parameters to obtain the dimension feature analysis values ​​corresponding to the data feature parameters. The dynamic storage analysis value acquisition module is used to identify the dynamic weights corresponding to the data feature parameters, and to perform weighted calculations on the data feature parameters according to the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values. A data storage type label recognition module is used to identify the data storage type label corresponding to the data file based on the dynamic storage analysis value; The target storage strategy acquisition module is used to match the data storage type tag with the strategies in the preset storage strategy library to obtain the target storage strategy of the data file; A storage level determination module is used to determine the storage level of the data file according to the target storage strategy; The storage address identification module is used to identify the storage address of the data file in the storage level using a preset transparent migration mechanism, and to store the data file in the storage address.

[0009] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the hierarchical storage method for the data file described above.

[0010] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the hierarchical storage method for the data file described above.

[0011] In the above-mentioned hierarchical storage method, apparatus, device, and medium for data files, a client can obtain a data file in a storage unit and extract data feature parameters from the data file; perform multi-dimensional quantitative analysis on the data feature parameters to obtain dimensional feature analysis values ​​corresponding to the data feature parameters; identify the dynamic weights corresponding to the data feature parameters, and perform weighted calculations on the data feature parameters according to the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values; identify the data storage type label corresponding to the data file according to the dynamic storage analysis values; match the data storage type label with strategies in a preset storage strategy library to obtain the target storage strategy for the data file; determine the storage level of the data file according to the target storage strategy; and utilize a preset transparent migration... The migration mechanism identifies the storage address of the data file in the storage level and stores the data file at the storage address. In this invention, three-dimensional data feature parameters such as access frequency, recent access time, and file size are extracted to overcome the limitations of single-parameter analysis. Multi-dimensional quantitative analysis is performed through access frequency weight score calculation, recent access time exponential decay algorithm, and file size curve scoring rules to ensure accurate dimensional feature analysis values. Dynamic storage analysis values ​​are obtained by weighting the three-dimensional parameters with reasonable weights, which can accurately identify hot, warm, and cold data storage type labels. Matching a preset three-media tiered strategy library can determine the appropriate storage level. Then, the transparent migration mechanism automatically adjusts the replication strategy and media to accurately locate the storage address, improve storage resource utilization, reduce costs, and solve the problem of low accuracy in data file tiered storage. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of an application environment for a hierarchical storage method for data files according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a hierarchical storage method for data files according to an embodiment of the present invention; Figure 3 yes Figure 2 A flowchart illustrating a specific implementation method of step S3; Figure 4 yes Figure 2 A flowchart illustrating a specific implementation of step S4; Figure 5 This is a schematic diagram of a hierarchical storage device for data files according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 7 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The hierarchical storage method for data files provided in this embodiment of the invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can obtain data files from the storage unit through the client, extract data feature parameters from the data files, perform multi-dimensional quantitative analysis on the data feature parameters to obtain the dimensional feature analysis values ​​corresponding to the data feature parameters, identify the dynamic weights corresponding to the data feature parameters, and perform weighted calculations on the data feature parameters according to the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values; identify the data storage type label corresponding to the data file according to the dynamic storage analysis values; match the data storage type label with the strategies in the preset storage strategy library to obtain the target storage strategy for the data file; determine the storage level of the data file according to the target storage strategy; and use a preset transparent migration mechanism to identify the storage level of the data file in the storage unit. This invention describes the storage address in the storage hierarchy and stores the data file in the storage address. It extracts three-dimensional data feature parameters—access frequency, recent access time, and file size—to overcome the limitations of single-parameter analysis. Multi-dimensional quantitative analysis is performed through access frequency weighting, recent access time exponential decay algorithm, and file size curve scoring rules to ensure accurate dimensional feature analysis values. Dynamic storage analysis values ​​are obtained by weighting the three-dimensional parameters appropriately, accurately identifying hot, warm, and cold data storage type labels. Matching a preset three-media tiered strategy library determines the appropriate storage hierarchy. Furthermore, a transparent migration mechanism automatically adjusts the replication strategy and media, accurately locating the storage address, improving storage resource utilization, and reducing costs, thus solving the problem of low accuracy in data file tiered storage. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster composed of multiple servers. The invention is described in detail below through specific embodiments.

[0016] Please see Figure 2 As shown, Figure 2 A flowchart illustrating a hierarchical storage method for data files provided in an embodiment of the present invention includes the following steps: S1. Obtain the data file in the storage unit and extract the data feature parameters from the data file.

[0017] In this embodiment of the invention, the storage unit refers to the different levels of storage media and corresponding storage nodes that carry data in the HDFS hot and cold data tiered storage scheme; the data file refers to various types of data stored in these storage units and classified according to access frequency and purpose.

[0018] In detail, the process of retrieving data files from storage units is based on a three-media tiered architecture to determine the storage units. The hot data layer consists of DateNodes based on NVMe SSD media and employing a three-replica high-availability mechanism; the warm data layer consists of DateNodes based on HDD media and employing EC encoding technology; and the cold data layer consists of a lake-bottom storage system employing EC storage and object storage media. The namespace is made transparent by relying on HDFS Federation, and the required data files are located and retrieved from the corresponding hot, warm, and cold data layer storage units in conjunction with the storage policies defined in hdfs-site.xml.

[0019] In this embodiment of the invention, the data feature parameters refer to three types of core parameters used for hierarchical evaluation of data files, specifically the access frequency, recent access time, and file size of the data file.

[0020] In this embodiment of the invention, extracting data feature parameters from the data file includes: Extract the original access information of the data file within a preset access time period; Based on the original access information, identify the access frequency parameter and access timestamp parameter of the data file within the access time period; Extract the storage attribute information of the data file, quantify the file capacity data in the storage attribute information, and obtain the file data volume parameter of the data file. The access frequency parameter, the access timestamp parameter, and the file data volume parameter are integrated to obtain the data feature parameters of the data file.

[0021] Specifically, the preset access time period is the past 62 days; the raw access information consists of the number of times a data file has been accessed within the past 62 days and the latest access time of that file. Leveraging the namespace transparency achieved through a three-media layered architecture and HDFS Federation, the raw access information is extracted from the target data file in the corresponding storage unit by collecting the number of accesses and the latest access time within the past 62 days.

[0022] Specifically, the access frequency parameter refers to the number of times a data file is accessed within a preset access time period, and the access timestamp parameter refers to the latest access time of the data file within the preset access time period. Based on the extracted original access information of the data file over the past 62 days, and relying on the namespace transparency achieved through the three-media layered architecture and HDFS Federation, the number of accesses over the past 62 days is directly identified from the original access information as the access frequency parameter, and the latest access time is identified as the access timestamp parameter.

[0023] Furthermore, storage attribute information refers to the storage-related characteristics of a data file, with its core including data file capacity information; file capacity data refers to the actual size of the data file itself; and file data volume parameters are parameters that can be used for data grading and evaluation after the file capacity data has been quantified according to preset rules. Based on the namespace transparency achieved through a three-media layered architecture and HDFS Federation, storage attribute information containing file capacity information is extracted from the target data file of the corresponding storage unit. Then, the file capacity data is mapped to a specific range of 0-100 points according to the curve scoring rules set in the scheme for quantification, thereby obtaining file data volume parameters that can be used for subsequent data grading.

[0024] Furthermore, the data feature parameters are obtained by integrating the access frequency parameter, the access timestamp parameter, and the file data volume parameter. This is carried out based on a three-dimensional evaluation model of access frequency, recent access time, and file size. First, the access frequency parameter is weighted according to rules, the access timestamp parameter is scored using an exponential decay algorithm, and the file data volume parameter is mapped to a score of 0-100 using a curve scoring rule. Then, the feature integration is completed by calculating the average of the three scores.

[0025] For example, in a medical scenario, for real-time heart rate monitoring data of inpatients and historical medical records of outpatients stored in a hospital's HDFS, the original access information of the past 62 days is first extracted, and then the access frequency parameter and access timestamp parameter are identified based on this information. Next, the file capacity data in the storage attribute information is extracted and quantified according to the curve scoring rule to obtain the file data volume parameter. Finally, the scores of the three types of parameters are integrated to calculate a dynamic score, and the data feature parameters of the corresponding medical data are obtained.

[0026] For example, in a financial scenario, for real-time stock trading data and user historical billing data stored in HDFS, the raw access information of the past 62 days is extracted, and access frequency parameters and access timestamp parameters are identified accordingly. Then, the file capacity data in the storage attribute information is extracted, and the file data volume parameter is quantified according to the curve scoring rule. Finally, the scores of the three types of parameters are integrated to calculate a dynamic score, and the data feature parameters of the two types of financial data are obtained.

[0027] S2. Perform multi-dimensional quantitative analysis on the data feature parameters to obtain the dimensional feature analysis values ​​corresponding to the data feature parameters.

[0028] In this embodiment of the invention, the dimensional feature analysis value refers to the specific score obtained by each feature parameter through its corresponding quantification calculation rule when performing multi-dimensional quantification analysis on the three data feature parameters of data access frequency, recent access time, and file size.

[0029] In this embodiment of the invention, the step of performing multi-dimensional quantitative analysis on the data feature parameters to obtain the dimensional feature analysis values ​​corresponding to the data feature parameters includes: Calculate the access frequency score corresponding to the access frequency parameter among the data feature parameters; The access timestamp parameter in the data feature parameters is subjected to time decay processing to obtain the access timestamp score of the access timestamp parameter; The file data volume parameter in the data feature parameters is mapped to a preset score range, and the file data volume score of the file data volume parameter is determined according to the score range; The arithmetic mean of the access frequency score, the access timestamp score, and the file data volume score is used to obtain the dimensional feature analysis value corresponding to the data feature parameter.

[0030] In detail, the access frequency score is a specific score obtained by quantifying the data characteristic parameter of access frequency based on the number of times the data has been accessed in the past 62 days. If the number of times the data has been accessed in the past 62 days is 0, the access frequency score is 0. If the number of times the data has been accessed in the past 62 days is greater than or equal to 63 times (one read / write operation + daily access), the access frequency score is 100. If the number of times the data has been accessed in the past 62 days is greater than 0 times but less than 63 times, the access frequency score is calculated according to the formula (number of accesses / 62) * 100 * 0.95.

[0031] Specifically, the access timestamp score is a score obtained by quantifying and analyzing the access timestamp in the data feature parameters. First, the latest access time of the file is taken as the access timestamp parameter. Then, an exponential decay algorithm is used to process this parameter. The score will decrease exponentially over time according to a set rule, and the final score is the access timestamp score.

[0032] Furthermore, the preset score range is a specific score range set in the data for mapping the file data size parameter, specifically 0-100 points; the file data size score is the specific score obtained after mapping the file data size parameter to the preset score range according to the corresponding scoring rules. First, determine the specific file size value corresponding to the file data size parameter, and then map the file size value to the preset score range of 0-100 points according to the curve scoring rules set in the data. The specific score obtained after mapping is the file data size score.

[0033] Furthermore, the access frequency score is calculated based on the number of accesses in the past 62 days, the access timestamp score is obtained by taking the latest access time of the file and applying an exponential decay algorithm, and the file data volume score is obtained by mapping the file size to the 0-100 score range according to the curve scoring rules. Then, these three separately calculated scores are summed, and the sum is divided by 3. Through this arithmetic mean operation, the dimensional feature analysis value corresponding to the data feature parameters is finally obtained.

[0034] For example, in a medical scenario, when calculating dimensional feature analysis values ​​for patient treatment data, the access frequency score is calculated first: Severe case data from the past 62 days, accessed ≥63 times daily with read / write operations, receives 100 points; rehabilitation data, without access, receives 0 points; outpatient data, accessed 31 times, is weighted at (31 / 62)^1000.95, resulting in 47.5 points. Next, the access timestamp score is processed: Severe case data accessed today receives 100 points; rehabilitation data accessed 3 days ago is weighted at 0.95 for each day that has passed. Next, the file data volume is mapped to a score range of 0-100. 10GB of high-definition critical care images are scored as 80 points according to the curve, and 100KB of rehabilitation reports are scored as 40 points. Finally, the scores of the three categories are summed and then divided by 3. For example, critical care data (100+100+80) / 3≈93.3, rehabilitation data (0+85.7+40) / 3≈41.9, to obtain the corresponding dimension feature analysis values.

[0035] For example, in a financial scenario, when calculating the feature analysis values ​​for user real-time transaction data, recent month transfer records, and annual financial statements, the access frequency score is calculated first: real-time transaction data from the past 62 days, due to daily access and read / write operations ≥ 63 times, receives 100 points; recent month transfer records, accessed 31 times, are weighted at (31 / 62)1000.95, receiving 47.5 points; and annual reports, without access, receive 0 points. Next, the access timestamp score is processed: real-time data accessed today receives 100 points; transfer records accessed 5 days ago are weighted at 0.95 for each day that has passed. The report retrieved 30 days ago The file data volume is mapped to a score range of 0-100. 1GB of real-time data gets 90 points, 500MB of transfer records gets 60 points, and 200MB of reports gets 30 points. Finally, the scores of the three categories are summed and divided by 3. For example, real-time data (100+100+90) / 3≈96.67, to obtain the corresponding dimension feature analysis value.

[0036] S3. Identify the dynamic weights corresponding to the data feature parameters, and perform weighted calculations on the data feature parameters based on the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values.

[0037] In this embodiment of the invention, the dynamic weight is a weight score that dynamically changes with the actual data situation, calculated according to the corresponding rules for the three data feature parameters of access frequency, recent access time, and file size in the dynamic hierarchical algorithm of the HDFS cold and hot data hierarchical storage method based on lake bottom storage and EC storage.

[0038] In detail, for the three data feature parameters of access frequency, recent access time, and file size, corresponding scores are calculated according to specific rules. The dynamic weight score of access frequency is calculated based on the number of accesses in the past 62 days, the dynamic weight score of recent access time is taken from the latest access time of the file and an exponential decay algorithm is used, and the dynamic weight score of file size is determined based on the curve scoring rule, which maps the file size to a specific range of 0-100 points.

[0039] In this embodiment of the invention, the dynamic storage analysis value is a value obtained by weighted calculation based on the dimensional feature analysis values ​​and dynamic weights corresponding to the three core feature parameters of the data in the HDFS hot and cold data tiered storage method based on lake bottom storage and EC storage. This value is the core basis for determining whether the data belongs to hot data (dynamic storage analysis value ≥ 95), warm data (80 ≤ dynamic storage analysis value < 95), or cold data (dynamic storage analysis value < 80).

[0040] In this embodiment of the invention, reference is made to Figure 3 As shown, the step of weighting the data feature parameters according to the dimensional feature analysis value and the dynamic weight to obtain the dynamic storage analysis value includes: S31. Associate the dimensional feature analysis value with the data feature parameter to obtain the feature association parameter; S32. Dynamically assign weights to the feature association parameters according to the dynamic weights to obtain the target feature parameters; S33. Perform weighted calculations on the target feature parameters to obtain the dynamic storage analysis values ​​of the data feature parameters.

[0041] In detail, feature association parameters refer to the parameter combinations formed by matching the three core data feature parameters of HDFS data with their corresponding dimensional feature analysis values ​​one-to-one. For the HDFS data to be analyzed, the three data feature parameters of access frequency, recent access time, and file size are first determined. Then, the dimensional feature analysis value corresponding to each parameter is calculated according to given rules. Finally, each data feature parameter is matched with its calculated dimensional feature analysis value to obtain the feature association parameters.

[0042] Specifically, the target feature parameters refer to the set of parameters formed after weighting the feature association parameters according to the dynamic weighting rules corresponding to each data feature parameter. This set includes each data feature parameter, its corresponding dimensional feature analysis value, and a defined dynamic weight. First, the associated and matched data feature parameters and dimensional feature analysis values ​​are determined. Then, based on the preset dynamic weighting calculation rules in the data, corresponding dynamic weights are assigned to each data feature parameter and its corresponding dimensional feature analysis value in the feature association parameters, ensuring that each association combination has a defined dynamic weight, thereby obtaining the target feature parameters.

[0043] Furthermore, the dynamic weight score corresponding to the access frequency is first extracted from the target feature parameters. Then, according to the weighted calculation rule of "dynamic score = (access frequency score + recent access time score + file size score) / 3", the three extracted dynamic weight scores are added together and divided by 3. The result is the dynamic storage analysis value of the data feature parameters.

[0044] For example, in a medical setting, for patient diagnosis and treatment data, the data feature parameters are first associated and matched with their dimensional feature analysis values ​​to obtain feature association parameters; then, weights are assigned to them according to dynamic weight rules to form target feature parameters; finally, dynamic storage analysis values ​​are obtained by weighting by (access frequency score + recent access time score + file size score) / 3, and the frequently accessed critical illness data is stored in a hot layer, the occasionally used data is stored in a warm layer, and the archived data is stored in a cold layer.

[0045] For example, in a financial scenario, for bank transaction data, the data feature parameters are first matched with their dimensional feature analysis values ​​to obtain feature association parameters; then, weights are assigned to them according to dynamic weight rules to form target feature parameters; finally, dynamic storage analysis values ​​are obtained by weighting by (access frequency score + recent access time score + file size score) / 3, and high-frequency real-time transfer records are stored in the hot layer and low-frequency reconciliation files are stored in the cold layer.

[0046] S4. Identify the data storage type tag corresponding to the data file based on the dynamic storage analysis value.

[0047] In this embodiment of the invention, the data storage type label refers to the label assigned to the data file after classifying it based on a dynamic score calculated according to three dimensions: access frequency, recent access time, and file size. Specifically, it includes a "hot data" label when the dynamic score is ≥95, a "cold data" label when the dynamic score is <80, and a "warm data" label when the dynamic score is between 80 (inclusive) and 95 (exclusive). Different labels correspond to different storage media and strategies to achieve layered storage of hot and cold data in HDFS.

[0048] In this embodiment of the invention, reference is made to Figure 4As shown, the step of identifying the data storage type tag corresponding to the data file based on the dynamic storage analysis value includes: S41. When the dynamic storage analysis value is greater than the preset first data type score threshold, the data storage type label corresponding to the data file is identified as the first label. S42. When the dynamic storage analysis value is less than the preset second data type score threshold, the data storage type label corresponding to the data file is identified as the second label. S43. When the dynamic storage analysis value is greater than the second data type score threshold and less than the first data type score threshold, the data storage type label corresponding to the data file is identified as the third label.

[0049] In detail, the preset first data type score threshold is 95; the first label is "hot data". First, a three-dimensional evaluation model is constructed based on the access frequency, recent access time, and file size of the data file to calculate the score of each dimension. The access frequency score is calculated based on the number of accesses in the past 62 days, the recent access time score is based on the latest access time of the file and uses an exponential decay algorithm, and the file size score is mapped to the 0-100 score range based on the file size value according to the curve scoring rules. Then, the dynamic storage analysis value is calculated through a dynamic scoring formula. Subsequently, the dynamic storage analysis value is compared with the preset data type score threshold of 95. When the dynamic storage analysis value is greater than (or equal to) the threshold, the data storage type label corresponding to the data file is identified as "hot data" as the first label.

[0050] Specifically, the preset score threshold for the second data type is 80; the second label is "cold data". A three-dimensional evaluation model is constructed based on the access frequency, recent access time, and file size of the data file to calculate the score for each dimension. The access frequency score is calculated based on the number of accesses in the past 62 days, the recent access time score is based on the latest access time of the file and uses an exponential decay algorithm, and the file size score is based on the actual file size and mapped to the 0-100 score range according to the curve scoring rules. Then, a dynamic storage analysis value is calculated using a formula. When the dynamic storage analysis value is less than the preset data type score threshold of 80, the data storage type label corresponding to the data file is identified as the second label "cold data".

[0051] Furthermore, the third label is "warm data". A three-dimensional evaluation model is constructed based on the access frequency, recent access time, and file size of the data file to calculate the score for each dimension. The access frequency score is calculated based on the number of accesses in the past 62 days. The recent access time score is based on the latest access time of the file and uses an exponential decay algorithm. The file size score is mapped to the 0-100 score range based on the file size according to the curve scoring rules. Then, a dynamic storage analysis value is calculated using a formula. When the dynamic storage analysis value is greater than the threshold of 80 and less than the threshold of 95, the data storage type label corresponding to the data file is identified as the third label "warm data".

[0052] S5. Match the data storage type label with the policies in the preset storage policy library to obtain the target storage policy for the data file.

[0053] In this embodiment of the invention, the preset storage strategy library is a predefined set of storage rules corresponding to different storage types of data; the target storage strategy is a specific storage rule that is matched from the preset storage strategy library and applicable to the data file based on the storage type label obtained by the data through a dynamic hierarchical algorithm.

[0054] In this embodiment of the invention, the step of matching the data storage type tag with policies in a preset storage policy library to obtain the target storage policy for the data file includes: Extract the key information on storage requirements corresponding to the data storage type tags; The key information of the storage requirements is matched with the strategies in the preset storage strategy library to obtain a set of candidate storage strategies for the data storage type label; Based on the basic data attributes of the data file, the candidate storage strategy set is filtered to obtain a subset of candidate storage strategies; The candidate storage strategy subset is prioritized to obtain the storage strategy sequence for the data file. The storage strategy with the highest priority in the storage strategy sequence is determined as the target storage strategy for the data file.

[0055] In detail, key storage requirement information refers to the storage media type, replication or EC encoding strategy, and preset storage path configuration corresponding to the hot, warm, and cold storage type tags of the data. A dynamic score is calculated based on a three-dimensional evaluation model of access frequency, recent access time, and file size. Data is tagged as hot, warm, or cold data according to the dynamic score: ≥95, 80≤dynamic score<95, and dynamic score<80, respectively. Then, based on this tag, the required storage media, replication / EC strategy, and storage path information are extracted from the preset storage strategy library, thus completing the extraction of key storage requirement information.

[0056] Specifically, the candidate storage strategy set refers to the set of all preset storage strategies selected from the preset storage strategy library that completely match the key information of storage requirements corresponding to the data storage type label. A dynamic score is calculated based on a three-dimensional evaluation model of access frequency, recent access time, and file size. The storage type label of the data is determined based on the dynamic score, and then the key information of storage requirements corresponding to that label is extracted. Finally, all preset storage strategies that match the extracted key information of storage requirements are selected from the preset storage strategy library. These selected preset storage strategies together constitute the candidate storage strategy set for that data storage type label.

[0057] Furthermore, basic data attributes refer to the core data features used to assess the popularity of data files, specifically including the access frequency, recent access time, and file size. These three are also the core inputs to the three-dimensional evaluation model in the dynamic grading algorithm. The candidate storage strategy subset refers to the set of storage strategies that better match the actual characteristics of the data file, obtained by further filtering from the candidate storage strategy set based on the specific basic data attributes of the data file. First, the basic data attributes of the data file are obtained. Based on these attributes, a dynamic score is calculated using the three-dimensional evaluation model to determine the data storage type label. Then, a candidate storage strategy set is obtained by matching the labels. Finally, the candidate storage strategy set is further filtered based on the specific basic data attributes of the data file to obtain the final candidate storage strategy subset.

[0058] Furthermore, a storage strategy sequence refers to an ordered list of storage strategies formed by sorting a subset of candidate storage strategies according to their matching degree with the basic data attributes of the data file or their priority in meeting business needs. A subset of candidate storage strategies is selected based on the specific basic data attributes of the data file. Then, the subset is sorted according to the matching degree between each strategy and the basic data attributes of the data file or a preset priority rule, thus obtaining the storage strategy sequence for the data file.

[0059] Furthermore, a three-dimensional evaluation model based on the access frequency, recent access time, and file size of data files is used to calculate dynamic scores. These scores are then used to determine the data storage type labels: hot data, warm data, and cold data. Next, key storage requirement information corresponding to these labels is extracted and matched against a preset storage policy library containing rules such as the dfs.datanode.data.dir storage paths ([SSD] / data / hot, [HDD] / data / warm, [ARCHIVE] / data / cold) and dfs.storage.policy.enabled switches defined in hdfs-site.xml. This yields a set of candidate storage policies. A subset of these policies is then selected based on the specific basic data attributes of the data file. This subset is sorted according to its matching degree with the basic data attributes or its priority in meeting business requirements, forming a storage policy sequence. Finally, the storage policy with the highest priority in this sequence is selected as the target storage policy for the data file.

[0060] S6. Determine the storage level of the data file according to the target storage strategy.

[0061] In this embodiment of the invention, the storage tier refers to a tier based on a three-media layered architecture, each adapted to different hot and cold data storage needs. Specifically, it includes a hot data tier (using a three-replica high-availability mechanism, paired with high-performance storage media such as NVMe SSDs, supporting low-latency real-time access, used to store hot data such as shuffle data and data written on the same day), a warm data tier (using EC encoding technology, paired with large-capacity storage media such as HDDs, supporting real-time access with a certain latency, used to store warm data such as data with read and write records in the past 62 days), and a cold data tier (using a lakebed storage system, combining object storage media and EC encoding to optimize storage density, supporting massive data storage, used to store data that meets the cold data determination criteria).

[0062] In detail, the storage media type, replication or EC encoding strategy and preset storage path associated with the target storage strategy of the data file are clearly defined. Then, based on the preset three-media tiered architecture, the storage characteristics corresponding to the target storage strategy are matched with the adaptation characteristics of each storage tier. For example, if the target storage strategy includes NVMe SSD and three replicas, it corresponds to the hot data tier; if it includes HDD and EC, it corresponds to the warm data tier; and if it includes object storage and EC, it corresponds to the cold data tier. This determines the storage tier of the data file.

[0063] S7. Use a preset transparent migration mechanism to identify the storage address of the data file in the storage level, and store the data file in the storage address.

[0064] In this embodiment of the invention, the preset transparent migration mechanism refers to making the namespace transparent through HDFS Federation, so that the replica strategy can be automatically triggered when data is migrated between cold and hot layers; the storage address refers to the path preset in hdfs-site.xml that corresponds to different storage levels.

[0065] In this embodiment of the invention, identifying the storage address of the data file in the storage level using a preset transparent migration mechanism includes: The storage policy configuration information of the data file in the storage tier is extracted using a preset transparent migration mechanism; The logical access identifier of the data file is determined based on the storage policy configuration information; Query the storage medium path of the data file in the storage level according to the logical access identifier; The storage address of the data file in the storage hierarchy is identified based on the storage medium path.

[0066] In detail, the storage policy configuration information refers to the storage-related configurations predefined in hdfs-site.xml, including the paths set in dfs.datanode.data.dir corresponding to different storage tiers ([SSD] / data / hot corresponds to the hot data tier, [HDD] / data / warm corresponds to the warm data tier, and [ARCHIVE] / data / cold corresponds to the cold data tier), the dfs.storage.policy.enabled=true configuration for enabling storage policies, and the replica and media policies corresponding to different storage tiers (hot data uses a replica count of 3 and NVMe media, warm data uses a replica count of 1+EC policy and HDD media, and cold data uses a replica count of 1+EC policy and object storage media). Namespace transparency is achieved through HDFS Federation. First, a dynamic hierarchical algorithm is used, and then the corresponding storage policy is marked for data files using the hdfs storagepolicies command. After that, the hdfsmover migration daemon is started. When data files are migrated between cold and hot layers, the transparent migration mechanism automatically extracts the storage policy configuration information such as the path, replica, and media policy corresponding to the storage layer to which the data file belongs, which is preset in hdfs-site.xml, and triggers the replica policy adjustment.

[0067] Specifically, a logical access identifier refers to the preset logical path of a data file within the HDFS namespace, corresponding to its hot, warm, or cold storage tier. This identifier allows users or applications to access data without being aware of the physical storage medium or specific physical location. Through a dynamic hierarchical algorithm and based on storage policy configuration information, data files are associated with preset logical paths of their respective tiers. These associated preset logical paths are the logical access identifiers of the data files. Furthermore, leveraging the namespace transparency features implemented by HDFS Federation, it is ensured that once the logical access identifier is determined, users or applications can directly access the data through this identifier, and the logical access identifier remains unchanged when data is migrated between tiers.

[0068] Furthermore, the storage medium path refers to the actual directory path predefined in hdfs-site.xml, which corresponds one-to-one with each of the hot, warm, and cold storage tiers and is associated with specific storage media. Specifically, it is [SSD] / data / hot for the hot data tier, [HDD] / data / warm for the warm data tier, and [ARCHIVE] / data / cold for the cold data tier. The logical access identifier itself corresponds to the preset logical path of the storage tier to which the data file belongs. Relying on the namespace transparency feature implemented by HDFS Federation, after the data file has determined its tier and obtained the corresponding logical access identifier through the dynamic hierarchical algorithm, it can be directly matched to the specific storage medium path of the storage tier corresponding to the pre-configured logical access identifier in hdfs-site.xml.

[0069] Furthermore, the preset storage medium path is a specific path that corresponds one-to-one with each of the hot, warm, and cold storage levels. The storage level to which the data file belongs has been determined through a dynamic hierarchical algorithm. At the same time, relying on the namespace transparency implemented by HDFS Federation, the level to which the data file belongs is directly associated with a preset storage medium path. Therefore, by simply using the storage medium path corresponding to the storage level to which the data file belongs, the storage medium path can be directly identified as the storage address of the data file in the corresponding storage level. Even if the data is migrated between levels according to the transparent migration mechanism, its storage address in the target level will be determined and identified based on the storage medium path corresponding to the target storage level.

[0070] As can be seen, the above solution extracts three-dimensional data feature parameters—access frequency, recent access time, and file size—to overcome the limitations of single-parameter analysis. Multi-dimensional quantitative analysis is performed through access frequency weighting, recent access time exponential decay algorithms, and file size curve scoring rules to ensure accurate dimensional feature analysis values. Dynamic storage analysis values ​​are obtained by weighting the three-dimensional parameters appropriately, accurately identifying hot, warm, and cold data storage type labels. Matching a pre-defined three-media tiered strategy library determines the appropriate storage level. Furthermore, a transparent migration mechanism automatically adjusts the replication strategy and media, accurately locating storage addresses, improving storage resource utilization, and reducing costs. This solution addresses the problem of low accuracy in tiered data file storage.

[0071] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0072] In one embodiment, a hierarchical storage device for data files is provided, the hierarchical storage device 100 of which corresponds one-to-one with the hierarchical storage method for data files in the above embodiments. For example... Figure 5 As shown, the hierarchical storage device 100 for the data file includes a data feature parameter extraction module 101, a dimensional feature analysis value acquisition module 102, a dynamic storage analysis value acquisition module 103, a data storage type label identification module 104, a target storage strategy acquisition module 105, a storage level determination module 106, and a storage address identification module 107. Detailed descriptions of each functional module are as follows: The data feature parameter extraction module 101 is used to obtain a data file in the storage unit and extract data feature parameters from the data file. The dimension feature analysis value acquisition module 102 is used to perform multi-dimensional quantitative analysis on the data feature parameters to obtain the dimension feature analysis value corresponding to the data feature parameters. The dynamic storage analysis value acquisition module 103 is used to identify the dynamic weights corresponding to the data feature parameters, and to perform weighted calculations on the data feature parameters according to the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values. The data storage type label recognition module 104 is used to identify the data storage type label corresponding to the data file based on the dynamic storage analysis value; The target storage strategy acquisition module 105 is used to match the data storage type tag with the strategy in the preset storage strategy library to obtain the target storage strategy of the data file. Storage level determination module 106 is used to determine the storage level of the data file according to the target storage strategy; The storage address identification module 107 is used to identify the storage address of the data file in the storage level using a preset transparent migration mechanism, and to store the data file in the storage address.

[0073] In one embodiment, the data feature parameter extraction module 101, when extracting data feature parameters from the data file, is used to: Extract the original access information of the data file within a preset access time period; Based on the original access information, identify the access frequency parameter and access timestamp parameter of the data file within the access time period; Extract the storage attribute information of the data file, quantify the file capacity data in the storage attribute information, and obtain the file data volume parameter of the data file. The access frequency parameter, the access timestamp parameter, and the file data volume parameter are integrated to obtain the data feature parameters of the data file.

[0074] In one embodiment, the dimensional feature analysis value acquisition module 102, when performing multi-dimensional quantization analysis on the data feature parameters to obtain the dimensional feature analysis values ​​corresponding to the data feature parameters, is used to: Calculate the access frequency score corresponding to the access frequency parameter among the data feature parameters; The access timestamp parameter in the data feature parameters is subjected to time decay processing to obtain the access timestamp score of the access timestamp parameter; The file data volume parameter in the data feature parameters is mapped to a preset score range, and the file data volume score of the file data volume parameter is determined according to the score range; The arithmetic mean of the access frequency score, the access timestamp score, and the file data volume score is used to obtain the dimensional feature analysis value corresponding to the data feature parameter.

[0075] In one embodiment, the dynamic storage analysis value acquisition module 103, when performing a weighted calculation of the data feature parameters based on the dimensional feature analysis value and the dynamic weight to obtain the dynamic storage analysis value, is used to: The dimensional feature analysis values ​​are correlated with the data feature parameters to obtain feature correlation parameters; The feature association parameters are dynamically weighted according to the dynamic weights to obtain the target feature parameters. The target feature parameters are weighted and calculated to obtain the dynamic storage analysis value of the data feature parameters.

[0076] In one embodiment, the data storage type label identification module 104, when performing the task of identifying the data storage type label corresponding to the data file based on the dynamic storage analysis value, is used to: When the dynamic storage analysis value is greater than the preset first data type score threshold, the data storage type label corresponding to the data file is identified as the first label; When the dynamic storage analysis value is less than the preset second data type score threshold, the data storage type label corresponding to the data file is identified as the second label; When the dynamic storage analysis value is greater than the second data type score threshold and less than the first data type score threshold, the data storage type label corresponding to the data file is identified as the third label.

[0077] In one embodiment, the target storage policy acquisition module 105, when performing the matching of the data storage type tag with policies in a preset storage policy library to obtain the target storage policy of the data file, is configured to: Extract the key information on storage requirements corresponding to the data storage type tags; The key information of the storage requirements is matched with the strategies in the preset storage strategy library to obtain a set of candidate storage strategies for the data storage type label; Based on the basic data attributes of the data file, the candidate storage strategy set is filtered to obtain a subset of candidate storage strategies; The candidate storage strategy subset is prioritized to obtain the storage strategy sequence for the data file. The storage strategy with the highest priority in the storage strategy sequence is determined as the target storage strategy for the data file.

[0078] In one embodiment, the storage address identification module 107, when performing the identification of the storage address of the data file in the storage level using a preset transparent migration mechanism, is used to: The storage policy configuration information of the data file in the storage tier is extracted using a preset transparent migration mechanism; The logical access identifier of the data file is determined based on the storage policy configuration information; Query the storage medium path of the data file in the storage level according to the logical access identifier; The storage address of the data file in the storage hierarchy is identified based on the storage medium path.

[0079] This invention provides a hierarchical storage device for data files, extracting three-dimensional data feature parameters such as access frequency, recent access time, and file size, overcoming the limitations of single-parameter analysis. It performs multi-dimensional quantitative analysis through access frequency weighted score calculation, recent access time exponential decay algorithm, and file size curve scoring rules, ensuring accurate dimensional feature analysis values. Dynamic storage analysis values ​​are obtained by weighting the three-dimensional parameters appropriately, accurately identifying hot, warm, and cold data storage type labels. Matching a preset three-media hierarchical strategy library determines the appropriate storage level. Furthermore, a transparent migration mechanism automatically adjusts the replication strategy and media, accurately locating storage addresses, improving storage resource utilization, and reducing costs, thus solving the problem of low accuracy in hierarchical data file storage.

[0080] Specific limitations regarding the hierarchical storage device for data files can be found in the above description of the hierarchical storage method for data files, and will not be repeated here. Each module in the aforementioned hierarchical storage device for data files can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0081] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a hierarchical data file storage method on the server side.

[0082] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements client-side functions or steps of a hierarchical data file storage method.

[0083] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain the data file from the storage unit and extract the data feature parameters from the data file; Perform multi-dimensional quantitative analysis on the data feature parameters to obtain the dimensional feature analysis values ​​corresponding to the data feature parameters; Identify the dynamic weights corresponding to the data feature parameters, and perform weighted calculations on the data feature parameters based on the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values; Identify the data storage type tag corresponding to the data file based on the dynamic storage analysis value; The data storage type label is matched with the policies in the preset storage policy library to obtain the target storage policy for the data file; The storage level of the data file is determined according to the target storage strategy; The storage address of the data file in the storage level is identified using a preset transparent migration mechanism, and the data file is stored in the storage address.

[0084] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain the data file from the storage unit and extract the data feature parameters from the data file; Perform multi-dimensional quantitative analysis on the data feature parameters to obtain the dimensional feature analysis values ​​corresponding to the data feature parameters; Identify the dynamic weights corresponding to the data feature parameters, and perform weighted calculations on the data feature parameters based on the dimensional feature analysis values ​​and the dynamic weights to obtain dynamic storage analysis values; Identify the data storage type tag corresponding to the data file based on the dynamic storage analysis value; The data storage type label is matched with the policies in the preset storage policy library to obtain the target storage policy for the data file; The storage level of the data file is determined according to the target storage strategy; The storage address of the data file in the storage level is identified using a preset transparent migration mechanism, and the data file is stored in the storage address.

[0085] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0086] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0087] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0088] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

[0089] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A tiered storage method of data files, characterized by, The method comprises the following steps: acquiring a data file in a storage unit, and extracting a data feature parameter in the data file; performing multi-dimensional quantitative analysis on the data feature parameter to obtain a dimensional feature analysis value corresponding to the data feature parameter; identifying a dynamic weight corresponding to the data feature parameter, and performing weighted calculation on the data feature parameter according to the dimensional feature analysis value and the dynamic weight to obtain a dynamic storage analysis value; identifying a data storage type label corresponding to the data file according to the dynamic storage analysis value; matching the data storage type label with a strategy in a preset storage strategy library to obtain a target storage strategy of the data file; determining a storage level of the data file according to the target storage strategy; identifying a storage address of the data file in the storage level by using a preset transparent migration mechanism, and storing the data file into the storage address.

2. The hierarchical storage method of data files as claimed in claim 1, wherein, The step of extracting the data feature parameter in the data file comprises the following steps: extracting original access information of the data file in a preset access time period; identifying an access frequency parameter and an access timestamp parameter of the data file in the access time period according to the original access information; extracting storage attribute information of the data file, and performing quantitative processing on file capacity data in the storage attribute information to obtain a file data volume parameter of the data file; performing feature integration on the access frequency parameter, the access timestamp parameter and the file data volume parameter to obtain the data feature parameter of the data file.

3. The hierarchical storage method of data files according to claim 1, wherein, The step of performing multi-dimensional quantitative analysis on the data feature parameter to obtain a dimensional feature analysis value corresponding to the data feature parameter comprises the following steps: calculating an access frequency score corresponding to the access frequency parameter in the data feature parameter; performing time attenuation processing on the access timestamp parameter in the data feature parameter to obtain an access timestamp score of the access timestamp parameter; mapping the file data volume parameter in the data feature parameter into a preset score interval, and determining a file data volume score of the file data volume parameter according to the score interval; performing arithmetic mean operation on the access frequency score, the access timestamp score and the file data volume score to obtain the dimensional feature analysis value corresponding to the data feature parameter.

4. The hierarchical storage method of data files according to claim 1, wherein, The step of performing weighted calculation on the data feature parameter according to the dimensional feature analysis value and the dynamic weight to obtain a dynamic storage analysis value comprises the following steps: associating the dimensional feature analysis value with the data feature parameter to obtain a feature association parameter; performing dynamic weight distribution on the feature association parameter according to the dynamic weight to obtain a target feature parameter; performing weighted calculation on the target feature parameter to obtain the dynamic storage analysis value of the data feature parameter.

5. The hierarchical storage method of data files according to claim 1, wherein, The step of identifying a data storage type label corresponding to the data file according to the dynamic storage analysis value comprises the following steps: when the dynamic storage analysis value is greater than a preset first data type score threshold, identifying the data storage type label corresponding to the data file as a first label; When the dynamic storage analysis value is less than a preset second data type score threshold, a data storage type label corresponding to the data file is identified as a second label; When the dynamic storage analysis value is greater than the second data type score threshold and less than the first data type score threshold, a data storage type label corresponding to the data file is identified as a third label.

6. The hierarchical storage method of data files according to claim 1, wherein, The matching of the data storage type label with a strategy in a preset storage strategy library to obtain a target storage strategy of the data file comprises: extracting storage requirement key information corresponding to the data storage type label; matching the storage requirement key information with a strategy in a preset storage strategy library to obtain a candidate storage strategy set of the data storage type label; filtering the candidate storage strategy set according to the basic data attribute of the data file to obtain a candidate storage strategy subset; performing priority sorting on the candidate storage strategy subset to obtain a storage strategy sequence of the data file; determining a storage strategy with the highest priority in the storage strategy sequence as the target storage strategy of the data file.

7. The hierarchical storage method of data files according to claim 1, wherein, The identification of the storage address of the data file in the storage hierarchy by using a preset transparent migration mechanism comprises: extracting storage strategy configuration information of the data file in the storage hierarchy by using a preset transparent migration mechanism; determining a logical access identifier of the data file according to the storage strategy configuration information; inquiring a storage medium path of the data file in the storage hierarchy according to the logical access identifier; identifying a storage address of the data file in the storage hierarchy according to the storage medium path.

8. A tiered storage apparatus for data files, characterized by, comprise: a data feature parameter extraction module configured to acquire a data file in a storage unit and extract a data feature parameter in the data file; a dimension feature analysis value acquisition module configured to perform multi-dimensional quantitative analysis on the data feature parameter to obtain a dimension feature analysis value corresponding to the data feature parameter; a dynamic storage analysis value acquisition module configured to identify a dynamic weight corresponding to the data feature parameter, and perform weighted calculation on the data feature parameter according to the dimension feature analysis value and the dynamic weight to obtain a dynamic storage analysis value; a data storage type label identification module configured to identify a data storage type label corresponding to the data file according to the dynamic storage analysis value; a target storage strategy acquisition module configured to match the data storage type label with a strategy in a preset storage strategy library to obtain a target storage strategy of the data file; a storage hierarchy determination module configured to determine a storage hierarchy of the data file according to the target storage strategy; a storage address identification module configured to identify a storage address of the data file in the storage hierarchy by using a preset transparent migration mechanism, and store the data file into the storage address.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the hierarchical storage method of the data file according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the hierarchical storage method of the data file according to any one of claims 1 to 7.

Images