A dynamic policy data management method and storage system

Abstract

The application provides a dynamic policy data management method and a storage system method, which comprises the following steps: analyzing a received file write request, and extracting file characteristic data of a to-be-written object in the file write request; traversing a plurality of candidate disks in a storage system, and obtaining static attribute data and real-time state data of each candidate disk; determining a static adaptation index of each candidate disk corresponding to the to-be-written object according to the file characteristic data and the static attribute data; comprehensively calculating a comprehensive adaptation score of each candidate disk by comprehensively combining the static adaptation index and the real-time state data; and sorting all the candidate disks based on the comprehensive adaptation score, and selecting a candidate disk with the highest comprehensive adaptation score as a target storage device to execute a write operation of the to-be-written object. The application realizes fine intelligent scheduling of data, balances physical adaptation and real-time load, and improves the performance and resource utilization rate of a heterogeneous storage system.

Description

Technical Field

[0001] This invention relates to the field of computer storage technology, and in particular to a dynamic strategy data management method and storage system. Background Technology

[0002] With the rapid development of information technology, union file systems or distributed storage systems have greatly simplified user operations and shielded the partitioning details of the underlying hardware by combining multiple physical disks or directories into a unified view. In modern data centers and enterprise storage architectures, in order to balance performance and cost, a hybrid disk (heterogeneous storage) environment consisting of solid-state drives (SSDs, NVMe) and hard disk drives (HDDs) is usually adopted.

[0003] However, most existing union file systems employ static storage strategies when handling data writes, such as random selection, mount order, round-robin, or simply based on remaining space. While this static strategy is simple to implement, it has significant drawbacks in heterogeneous storage environments: Lack of awareness of disk characteristics: Static strategies cannot effectively distinguish between high-performance disks (such as NVMe SSDs) and low-speed disks (such as HDDs). For example, a random strategy may write database files with extremely high IO performance requirements to mechanical hard drives, causing business delays; or write large amounts of cold data that are not frequently accessed to expensive solid-state drives, resulting in wasted resources.

[0004] Ignoring file characteristics and real-time status: Existing strategies based on remaining space often lead to more writes to larger capacity disks (usually mechanical disks), ignoring the file size, type, and the disk's current real-time load (such as speed and latency). In scenarios with frequent read / write operations, the performance advantages of high-speed disks cannot be utilized, nor can disks under high load be dynamically avoided. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a dynamic strategy data management method and storage system to realize intelligent and dynamic optimal matching of data among different types of disks.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A dynamic strategy data management method, the method comprising: Parse the received file write request and extract the file feature data of the object to be written from the file write request; Traverse multiple candidate disks in the storage system and obtain static attribute data and real-time status data for each candidate disk; Based on the file feature data and the static attribute data, determine the static adaptation index of each candidate disk corresponding to the object to be written; By combining the static adaptation metrics and the real-time status data, a comprehensive adaptation score is calculated for each candidate disk. Based on the comprehensive adaptation score, all candidate disks are sorted, and the candidate disk with the highest comprehensive adaptation score is selected as the target storage device to perform the write operation on the object to be written.

[0007] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows: A storage system includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the various steps of the aforementioned dynamic strategy data management method.

[0008] The beneficial effects of this invention are as follows: This application establishes a multi-dimensional adaptation mapping relationship between file requirements and disk characteristics by extracting file feature data of the object to be written and combining it with the static attributes and real-time status data of candidate disks. Using a multi-factor evaluation model to calculate a comprehensive adaptation score, the optimal storage device at any given moment can be dynamically selected based on the file's own characteristics (such as type and size) and the disk's current actual condition (such as media type and real-time load). This method effectively solves the problem that existing static strategies cannot perceive file differences and real-time disk status, achieving intelligent and accurate matching of data and storage resources in heterogeneous storage environments, thereby maximizing the overall performance of the storage system. Attached Figure Description

[0009] Figure 1 A flowchart illustrating the steps of a dynamic strategy data management method provided in this embodiment of the invention; Figure 2 This is a schematic diagram of a storage system provided in an embodiment of the present invention. Detailed Implementation

[0010] In related technologies, with the development of big data and cloud computing, storage systems typically include various types of storage media (such as high-speed solid-state drives (SSDs) and low-speed hard disk drives (HDDs). To improve data storage efficiency, it is usually necessary to select appropriate storage locations based on data characteristics. However, related technologies mainly rely on simple polling strategies or static allocation based solely on remaining disk capacity, lacking comprehensive consideration of file characteristics (such as size and type) and real-time disk status (such as load and latency). This extensive management approach can easily lead to large files crowding out high-speed small file channels or hot data concentrating on busy disks, resulting in a decline in overall storage system performance and an inability to effectively address performance fluctuations under mixed load scenarios.

[0011] To address the aforementioned problems, this application provides a dynamic strategy data management method and storage system. The following section details one such dynamic strategy data management method.

[0012] The dynamic strategy data management method in this application can be used for intelligent scheduling in heterogeneous storage environments. The storage system in this application can be a distributed architecture, a centralized architecture, a cloud-native architecture, or a hyper-converged architecture, such as the Hadoop Distributed File System (HDFS), an enterprise-grade network attached storage (NAS) array, a cloud data center object storage resource pool, or a high-performance computing (HPC) parallel file system.

[0013] The following describes in detail a dynamic strategy data management method of the present invention, with reference to the appendix. Figure 1 The method includes steps 110 to 150.

[0014] Step 110: Parse the received file write request and extract the file characteristic data of the object to be written from the file write request. The file characteristic data usually includes metadata information such as file size, file type (such as file extension), and creation time.

[0015] Step 120: Traverse multiple candidate disks in the storage system and obtain the static attribute data and real-time status data of each candidate disk. The static attribute data includes fixed parameters such as the disk's media type (e.g., solid-state drive SSD, hard disk drive HDD) and total capacity; the real-time status data includes dynamically changing parameters such as the current remaining capacity, read / write load, and I / O latency.

[0016] Step 130: Based on file feature data and static attribute data, determine the static adaptation index of each candidate disk corresponding to the object to be written. This step mainly evaluates the degree of matching between the inherent attributes of the file itself (such as size and type) and the physical characteristics of the disk (such as media type), without considering current load fluctuations.

[0017] Step 140: Combine static adaptation metrics with real-time status data to calculate the overall adaptation score for each candidate disk. In this step, the matching degree of physical attributes is combined with the current operating status of the system to obtain a quantified value that characterizes the suitability of the disk for handling the specific file write request at the current moment.

[0018] Step 150: Sort all candidate disks based on the overall adaptation score, and select the candidate disk with the highest overall adaptation score as the target storage device to perform the write operation on the object to be written. Finally, according to the sorting result, the system directs the data to the optimal node to complete the disk write operation.

[0019] This application constructs a closed-loop control process from feature extraction to decision execution through steps 110 to 150. This method differs from traditional single scheduling modes based solely on remaining capacity or random polling. Instead, it logically correlates data characteristics (file feature data) with storage device characteristics (static attributes and real-time status). By first calculating static adaptation metrics to quantify the matching degree between data and device physical attributes, and then combining this with real-time status data to assess the current system's load-bearing capacity, the final calculated comprehensive adaptation score objectively reflects the suitability of each disk in handling the write request at the current moment. This scheme realizes a shift from coarse-grained random writing to refined strategy writing, optimizing the allocation efficiency of storage resources.

[0020] In one embodiment of this application, step 140 includes steps 141 to 145.

[0021] Step 141: Based on the file size in the file feature data, calculate the size fit score for the media type of the current candidate disk using a preset file size fit algorithm. This score reflects the degree of fit between the file size and the disk media throughput characteristics.

[0022] Step 142: Based on the file type in the file feature data, retrieve the preset file type feature table to determine the type affinity score of the current candidate disk's media type. This score reflects the degree of matching between a specific file format (such as logs, videos, and documents) and the random / sequential read / write characteristics of the disk media.

[0023] Step 143: Use the size fit score and type affinity score as static fit indicators.

[0024] Step 144: Generate a performance availability score based on the real-time load metrics of the current candidate disks. This metric represents the current workload of the disks; the lower the load, the higher the score is typically.

[0025] Step 145: Weight and aggregate the size fit score, type affinity score, and performance availability score to generate a comprehensive fit score. The aggregation process uses mathematical operations to merge the scores of the three dimensions into a single scalar, facilitating subsequent ranking and comparison.

[0026] This embodiment establishes a three-dimensional evaluation model through steps 141 to 145. By decomposing the evaluation dimensions into size fit score (volume dimension), type affinity score (format dimension), and performance availability score (load dimension), the system can more comprehensively quantify the relationship between data and devices. Static indicators (size, type) ensure that data is stored long-term on media best suited to its physical characteristics, while dynamic indicators (performance availability) avoid congestion on hot disks. The weighted aggregation method allows the system to find a balance between physical fit and current performance bottlenecks, ensuring that write decisions conform to long-term storage planning while meeting immediate performance needs.

[0027] In one embodiment of this application, steps 135 to 137 are included before calculating the comprehensive adaptation score in step 140.

[0028] Step 135: Based on the file size and the remaining capacity of each candidate disk, calculate the estimated space utilization after the object to be written is written to each candidate disk. This step simulates the state after the write operation occurs and predicts the disk capacity load.

[0029] Step 136: Candidate disks whose estimated space usage exceeds a preset safety threshold are marked as unavailable. Candidate disks marked as unavailable will not participate in the comprehensive adaptation score calculation. This mechanism acts as a circuit breaker to prevent write failures or file system crashes due to space exhaustion.

[0030] Step 137: If the estimated space utilization of all candidate disks exceeds the safety threshold, remove the unavailable status mark and select the candidate disk with the largest remaining capacity as the target storage device.

[0031] This embodiment introduces a circuit breaker protection mechanism based on estimated space utilization through steps 135 to 137. Before performance optimization, risk nodes that may cause write failures or system instability due to insufficient space are preferentially eliminated. This pre-filtering strategy not only reduces the overhead of subsequent scoring calculations but, more importantly, ensures the security of the storage system. In the extreme case where all disks are underutilized, a fallback strategy forces the selection of the device with the largest remaining capacity, ensuring business continuity and avoiding data loss due to policy limitations.

[0032] In one embodiment of this application, the calculation logic of the size adaptation score in step 141 is described.

[0033] A mathematical model is used to calculate the size fit score S. S is defined as the size fit score, e as the natural constant, and x as an intermediate variable. The calculation formula follows a variant of the sigmoid function: ; Where S is the size adaptation score, e is the natural constant, and x is the intermediate variable; When the media type of the candidate disk is high-speed solid-state storage, the intermediate variables satisfy the following relationship: ; When the media type of the candidate disk is mechanical magnetic storage, the intermediate variable satisfies the following relationship: ; Where F is the file size in the file feature data; C is a preset offset constant used to adjust the baseline threshold for size evaluation (i.e., to control the logical upper and lower limits of file size determination); and k is a preset scaling factor used to perform dimensional normalization mapping on the physical size of the file (i.e., to reflect the weight ratio of the actual file size in the algorithm).

[0034] This embodiment achieves a precise mapping between file size and media characteristics by introducing a non-linear function (Sigmoid form) and media differentiation logic. For high-speed solid-state storage, the formula design ensures that small files (with a smaller F) receive a higher fit score, aligning with SSDs' strength in handling fragmented, small I / O operations. For mechanical and magnetic storage, an inversion operation ensures that large files receive a relatively higher fit trend (or lower scores for small files), consistent with HDDs' suitability for sequential read / write operations. This mathematical model-based scoring mechanism transforms the originally vague concept of size matching into precise numerical quantification, allowing different storage media to fully leverage their respective physical advantages.

[0035] In one embodiment of this application, before step 132, which retrieves a preset file type feature table based on the file type in the file feature data to determine the type affinity score of the media type of the current candidate disk, step 101 is included.

[0036] Step 101: Pre-configure the file type feature table. This table defines the access modes corresponding to different file extensions and preset affinity values ​​for the media types of candidate disks associated with those file extensions. For example, it defines log files as having a lower affinity value for HDDs and a higher affinity value for SSDs; and video files as having a higher affinity value for HDDs and a lower affinity value for SSDs, etc.

[0037] Step 142 includes steps 1421 to 1423.

[0038] Step 1421: Identify the file extension of the object to be written, and search for a matching item in the file type feature table based on the file extension.

[0039] Step 1422: If a match is found, read the preset affinity value as the type affinity score.

[0040] Step 1423: If no match is found, the preset default strategy value is used as the type affinity score.

[0041] This embodiment introduces a type-aware mechanism based on a lookup table through steps 101 and 1421 to 1423. File extensions often implicitly indicate data access patterns (e.g., log files are mostly append-only, database files are mostly random read / write, and archive files are mostly sequential write). Through a pre-defined file type characteristic table, the system makes this implicit business logic explicit as a preset affinity value. When a request arrives, a quick lookup of the table determines the affinity between the data and the medium, and a default policy value is provided as a fallback to ensure policy coverage. This design allows the storage system to direct data to the medium most suitable for its I / O mode based on the type characteristics of the business data.

[0042] In one embodiment of this application, step 144 includes steps 1441 to 1443.

[0043] Step 1441: Collect the current read / write rate and latency data of the candidate disks. This data reflects the current I / O throughput and response speed of the disks in real time.

[0044] Step 1442: Obtain the preset performance conversion ratio based on the media type of the candidate disks. Since the physical performance benchmarks of different media (such as high-speed solid-state storage and mechanical magnetic storage) differ significantly, this ratio is necessary for alignment.

[0045] Step 1443: Use a performance conversion ratio to uniformly convert the read / write rate data and latency data, and determine the performance availability score based on the conversion result. In the calculation logic of this step, the lower the value of the latency data, the higher the corresponding performance availability score; the higher the value of the read / write rate data, the higher the corresponding performance availability score.

[0046] This embodiment addresses and unifies the issue of dimensional differences in performance metrics across heterogeneous storage media through steps 1441 to 1443. By introducing a performance conversion ratio, it eliminates the differences in benchmark performance between different physical media (such as SSDs and HDDs), achieving normalization. It clarifies that low latency and high read / write speed metrics are positive indicators, uniformly mapping diverse physical performance data into dimensionless availability scores. This mechanism ensures that the system can fairly and objectively assess the real-time load pressure of each disk in a mixed-media environment, accurately identifying the node with the most timely response and the largest throughput margin.

[0047] In one embodiment of this application, step 145 includes steps 1451 to 1454.

[0048] Step 1451: Set the performance availability score for the first weight factor, the size fit score for the second weight factor, and the type affinity score for the third weight factor.

[0049] Step 1452: When the media types of the candidate disks are the same (e.g., an all-SSD cluster), increase the proportion of the first weighting factor. At this point, media differences are no longer relevant, and real-time load becomes the key differentiating factor.

[0050] Step 1453: When the candidate disks have different media types (e.g., a mixed SSD and HDD cluster), decrease the weighting of the first weighting factor and increase the weighting of the third weighting factor. In this case, the difference in physical characteristics of the media has a greater impact on performance, and type matching should be given priority.

[0051] Step 1454: Multiply the performance availability score by the first weighting factor to obtain the first weighted term; multiply the size fit score by the second weighting factor to obtain the second weighted term; multiply the type affinity score by the third weighting factor to obtain the third weighted term; calculate the sum of the first weighted term, the second weighted term, and the third weighted term to obtain the comprehensive fit score.

[0052] This embodiment implements a scenario-adaptive weight adjustment mechanism through steps 1451 to 1454. The system can perceive the current hardware environment structure (homogeneous or heterogeneous) and dynamically adjust the shift of the evaluation focus accordingly. When the media types of candidate disks are the same and the differences in physical attributes are small, the algorithm automatically prioritizes load balancing (high first weight factor); when the media types of candidate disks are different and the differences in physical attributes are large, the algorithm automatically prioritizes media selection (high third weight factor). This dynamic weight strategy avoids the rigidity caused by fixed weights and ensures that the optimal disk selection result can be output under different hardware architectures.

[0053] In one embodiment of this application, steps 161 to 163 are also included, which may be performed in step 140 to calculate the comprehensive adaptation score of each of the candidate disks; Step 161: Record file access logs within the historical time period and monitor the actual read / write response performance of historical files on different disk media.

[0054] Step 162: Analyze the distribution patterns of access frequency and file size for different file types.

[0055] Step 163: Based on the access frequency, distribution pattern, and actual read / write response performance, periodically adjust the weight factor or the values ​​in the file type feature table.

[0056] This embodiment constructs the system's adaptive evolution capability through steps 161 to 163. Steps 161 and 162 are responsible for extracting business characteristics and physical feedback from historical operational data, while step 163 feeds this information back to control parameters (weighting factors or feature tables). This closed-loop mechanism makes the data management method no longer static, but capable of self-calibrating as business models change and hardware environments age. By adjusting weights and parameters, the system can continuously eliminate the deviation between preset strategies and actual needs, maintaining long-term strategy effectiveness and performance stability.

[0057] In one embodiment of this application, based on steps 136 and 137 (selecting whether the disk has been marked as unavailable), steps 210-230 are further performed: Step 210: When parsing the received file write request and extracting file feature data, determine whether the file type in the file feature data belongs to the preset high-performance requirement file type. The high-performance requirement file type includes at least the database file type.

[0058] Step 220: If the determination result is a high-performance requirement file type, then perform the optimization operation: Ignoring the calculation results of the comprehensive adaptation score, the optimal disk is selected directly from the candidate disks whose media type is high-speed solid-state storage and which are not marked as unavailable, based on real-time status data.

[0059] Step 230: If the result is that it is not a high-performance requirement file type, then proceed to calculate the comprehensive adaptation score for each candidate disk.

[0060] This embodiment constructs a service quality (QoS) tiered guarantee mechanism based on business priority through steps 210 to 230. For critical business files such as databases, a streamlined process is provided, bypassing complex comprehensive scoring logic and directly locking in high-speed solid-state storage resources. Simultaneously, this step incorporates unavailability status judgment, ensuring that the optimization process is based on capacity security. This strategy guarantees both the ultimate performance experience for core businesses and the overall scheduling efficiency of ordinary businesses in a hybrid storage environment, achieving on-demand allocation and maximizing the value of storage resources.

[0061] The above-described dynamic strategy data management method is applied to a real-world scenario. Specifically, it includes the following steps 1 through 7.

[0062] Step 1: Obtain the disk type, which corresponds to obtaining the static attributes in step 120 above.

[0063] The system traverses all candidate disks in the storage pool, identifying their physical media type (e.g., SSD, NVMe, or HDD). Different disk types directly determine the baseline for subsequent parameter calculations. For example, high-performance disks (SSD / NVMe) offer low latency and high throughput, but have relatively small capacities, making them suitable for storing small files (such as documents and configuration files) and frequently accessed hot data; while mechanical disks (HDDs), although having higher latency, offer large capacities and low cost, making them suitable for storing large files that are read and written sequentially (such as videos and images). Obtaining the type is the foundation for establishing static adaptation mappings.

[0064] Step 2: Read the disk's real-time performance metrics, corresponding to the real-time status acquisition in step 120 above.

[0065] The system collects the I / O rate and response latency of each candidate disk in real time.

[0066] Generally, higher current speed and greater latency indicate a busier disk. Under the same adaptation conditions, the strategy should prioritize disks that are relatively idle (lower load). This metric will subsequently be normalized using a conversion ratio to generate a performance availability score.

[0067] Step 3: Obtain the remaining disk space, corresponding to the circuit breaker protection mechanism based on estimated space utilization in steps 135 to 137 above.

[0068] First, based on the size of the file to be written, the system estimates the space utilization of each disk after the new file is written. If a disk's capacity exceeds a safe threshold (e.g., 90%) after writing, it is marked as unusable and discarded. This is because disks generally experience significant performance degradation after writing more than a certain amount of data (e.g., 90%). If the capacity of all candidate disks is less than 90%, the remaining capacity is not included in the final calculation (i.e., it is not counted in the overall score formula). This is to prevent large-capacity HDDs from masking their performance disadvantages due to their space advantage. If all disks exceed 90% capacity after writing the file, performance optimization is no longer feasible, and the system will force the disk with the most remaining capacity to be written to.

[0069] Step 4: Obtain the size of the file to be operated on, corresponding to the file feature data extraction in step 110 above.

[0070] The calculation primarily involves taking the negative of the file size and then using a variant of the sigmoid function. Larger files are more compatible with mechanical hard drives (HDDs), while smaller files should be placed on high-speed hard drives (SSDs). The specific algorithm is as follows: Set F to file size (in MB), k to preset scaling factor (set to 0.7059 in this embodiment, this value can be adjusted), and C to offset constant (set to 6.0 in this embodiment).

[0071] The calculation formula for mechanical magnetic storage (HDD): ; 100; The calculation formula for high-speed solid-state storage (SSD): ; 100; The above calculations achieve a precise non-linear match between file size and media type.

[0072] Step 5: Obtain the type of the operation file, corresponding to the file feature data extraction in Step 110, the determination of type affinity score in Step 142, and the service quality grading guarantee mechanism based on business priority in Steps 210 to 230.

[0073] A comprehensive evaluation is performed using static attributes (file type) and dynamic behavior (access pattern). For database type files (such as .db, .sql), as long as it is determined that the file will be accessed by the database application, it must always be stored on the high-performance disk, regardless of how busy or slow the high-performance disk is. Otherwise, storing it on the mechanical disk will cause subsequent database access to be extremely slow. For non-mandatory types, the score is obtained by consulting the pre-defined file type profile table (extension_profiles). A configuration example is as follows: extension_profiles = { #Types of High IOPS Demand 'db':{'ssd': 100,'hdd': 0}, 'vmdk':{'ssd': 90,'hdd': 10}, # Common file types such as documents 'doc':{'ssd': 80, 'hdd': 20}, 'xls':{'ssd': 80,'hdd': 20}, 'ofd':{'ssd': 85,'hdd': 20}, # Sequential reading and writing of large files 'mp4':{'ssd': 10,'hdd': 90}, 'mkv':{'ssd': 10,'hdd': 95}, 'iso':{'ssd': 0,'hdd': 100}, #Other types 'zip':{'ssd': 30,'hdd': 90}, 'rar':{'ssd': 20,'hdd': 90}, # Default Policy 'default':{'ssd': 50,'hdd': 80} } To adapt to different scenarios, an adaptive learning mechanism is introduced, corresponding to the adaptive corrections in steps 161 to 163 above. Historical data training is used to assess the varying file access habits of different enterprises and departments. The system utilizes historical access data from client organizations for training. The training results can be used to adjust the affinity values ​​of the file type feature table in step 5, or to fix the specific parameters of the weighting factors in step 6. Adaptive feedback is provided based on the user access habits of the current device, dynamically adjusting the parameters of the file type feature table (extension_profiles) in step 5, making the strategy increasingly accurate over time.

[0074] Step 6: Calculate the weight of each disk by weighting the above factors, corresponding to the comprehensive adaptation score generated in Step 145 above. The specific algorithm is as follows: Overall score = Performance score × w1 + Size score × w2 + Type score × w3 Weight parameter settings: w1 (performance weight): Initial value set to 0.2.

[0075] w2 (size weight): Initial value set to 0.3.

[0076] w3 (type weight): Initial value is set to 0.5.

[0077] When candidate disk media types are the same (such as a full SSD pool), the primary w1 (performance weight) takes effect.

[0078] When the candidate disk media types are different, the disk performance differences are already very large (SSD far exceeds HDD). In this case, w1 (performance weight) should not play a significant role, and w2 (size weight) and w3 (type weight) should be relied upon for trait matching.

[0079] The specific weight values ​​will be adjusted based on the values ​​obtained from adaptive learning in step 5.

[0080] Step 7: Select the disk with the highest weight for file operations, which corresponds to the sorting of candidate disks and the execution of write operations in step 150 above.

[0081] The system sorts the overall scores of all candidate disks and selects the disk with the best score for file operations.

[0082] In summary, the data management method provided in this application is based on a logical construction that precisely couples data characteristics and media attributes. First, the method analyzes the file characteristics (size, type) of the object to be written and the dynamic and static attributes (media, load, capacity) of the candidate disks to establish the fundamental dimensions for decision-making. Second, it sets a capacity safety baseline by estimating space utilization to eliminate overflow risks. Subsequently, it executes a tiered strategy based on file type. For high-performance files, it selects the best option based on real-time status within a secure, high-speed media pool to ensure rapid response. For ordinary files, it calculates and sorts a comprehensive adaptation score through a non-linear size adaptation algorithm, type affinity lookup table, and load performance conversion. This entire process realizes a transformation from coarse allocation to multi-dimensional fine-grained scheduling, maximizing overall resource utilization and operational stability in a heterogeneous storage environment while ensuring system capacity security and core business performance.

[0083] Please refer to Figure 2The present invention also provides a storage system, including a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor executes the computer program to implement the various steps of the data management method of a dynamic strategy as described above.

[0084] The beneficial effects of the storage system of the present invention are the same as those of the method described above, and will not be repeated here.

[0085] The above are merely embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention's specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A data management method with dynamic strategies, characterized in that, The method includes: Parse the received file write request and extract the file feature data of the object to be written from the file write request; Traverse multiple candidate disks in the storage system and obtain static attribute data and real-time status data for each candidate disk; Based on the file feature data and the static attribute data, determine the static adaptation index of each candidate disk corresponding to the object to be written; By combining the static adaptation metrics and the real-time status data, a comprehensive adaptation score is calculated for each candidate disk. Based on the comprehensive adaptation score, all candidate disks are sorted, and the candidate disk with the highest comprehensive adaptation score is selected as the target storage device to perform the write operation on the object to be written.

2. The data management method for dynamic strategies according to claim 1, characterized in that, The calculation of the comprehensive fit score for each candidate disk includes: Based on the file size in the file feature data, a preset file size adaptation algorithm is used to calculate the size adaptation score of the media type of the current candidate disk; Based on the file type in the file feature data, a preset file type feature table is retrieved to determine the type affinity score of the media type of the current candidate disk; The size fit score and the type affinity score are used as the static fit index; Based on the real-time load metrics of the candidate disks, a performance availability score is generated; The size fit score, the type affinity score, and the performance availability score are weighted and aggregated to generate the comprehensive fit score.

3. The data management method for dynamic strategies according to claim 2, characterized in that, Before calculating the overall fit score for each of the candidate disks, the method further includes: Based on the file size and the remaining capacity of each candidate disk, calculate the estimated space utilization rate after the object to be written is written to each candidate disk; Candidate disks whose estimated space utilization exceeds a preset safety threshold are marked as unavailable. Candidate disks marked as unavailable are not included in the comprehensive adaptation score calculation. If the estimated space utilization of all candidate disks exceeds the security threshold, the unavailable status is removed, and the candidate disk with the largest remaining capacity is selected as the target storage device.

4. The data management method for dynamic strategies according to claim 2, characterized in that, The step of calculating the size adaptation score of the current candidate disk's media type using a preset file size adaptation algorithm based on the file size in the file feature data includes: ； Where S is the size adaptation score, e is the natural constant, and x is the intermediate variable; When the media type of the candidate disk is high-speed solid-state storage, the intermediate variables satisfy the following relationship: ； When the media type of the candidate disk is mechanical magnetic storage, the intermediate variable satisfies the following relationship: ； Where F is the file size in the file feature data, C is a preset offset constant, and k is a preset scaling factor.

5. The data management method for dynamic strategies according to claim 2, characterized in that, Before determining the type affinity score of the media type of the current candidate disk by retrieving a preset file type feature table based on the file type in the file feature data, the process includes: A pre-configured file type feature table is provided, which defines the access modes corresponding to different file extensions and the preset affinity values ​​of the media types of the candidate disks corresponding to the file extensions. The step of determining the media type affinity score of the current candidate disk by retrieving a preset file type feature table based on the file type includes: Identify the file extension of the object to be written, and search for a matching item in the file type feature table based on the file extension; If a match is found, the preset affinity value is read as the type affinity score; If no match is found, the preset default strategy value is used as the type affinity score.

6. The data management method for dynamic strategies according to claim 2, characterized in that, The step of generating a performance availability score based on the real-time load metrics of the candidate disks includes: Collect the current read / write speed and latency data of the candidate disks; Based on the media type of the candidate disks, a preset performance conversion ratio is obtained; The read / write rate data and latency data are uniformly converted using the performance conversion ratio, and the performance availability score is determined based on the conversion result. The lower the latency value, the higher the corresponding performance availability score; the higher the read / write rate data value, the higher the corresponding performance availability score.

7. The data management method for dynamic strategies according to claim 2, characterized in that, Also includes: The first weighting factor corresponds to the performance availability score, the second weighting factor corresponds to the size adaptation score, and the third weighting factor corresponds to the type affinity score. When the media types of the candidate disks are the same, the proportion of the first weighting factor is increased; When the media types of the candidate disks are different, the proportion of the first weighting factor is reduced and the proportion of the third weighting factor is increased. The performance availability score is multiplied by the first weighting factor to obtain the first weighted term, the size fit score is multiplied by the second weighting factor to obtain the second weighted term, and the type affinity score is multiplied by the third weighting factor to obtain the third weighted term. The sum of the first weighted term, the second weighted term, and the third weighted term is calculated to obtain the comprehensive adaptation score.

8. The data management method for dynamic strategies according to claim 7, characterized in that, Also includes: Record file access logs within a historical time period and analyze the distribution patterns of access frequency and file size for different file types; Monitor the actual read / write response performance of historical files on different disk media; Based on the access frequency, the distribution pattern, and the actual read / write response performance, the weighting factor or the value in the file type feature table is periodically adjusted.

9. The data management method for dynamic strategies according to claim 3, characterized in that, Also includes: When parsing the file feature data, it is determined whether the file type belongs to a preset high-performance requirement file type; If it is the high-performance requirement file type, the calculation result of the comprehensive adaptation score is ignored, and the optimal disk is selected based on the real-time status data among the candidate disks whose media type is high-speed solid-state storage and which are not marked as unavailable. If it is not the high-performance requirement file type, then perform the step of calculating the comprehensive fit score for each of the candidate disks; The high-performance requirement file types include at least the database file type.

10. A storage system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a dynamic strategy data management method according to any one of claims 1-9.

Images