Solid-state hard drive storage optimization method and system

Abstract

The present invention discloses a solid-state hard disk storage optimization method and system, which relates to the field of data storage technology. The method comprises: partitioning storage units into regions based on behavioral layer data and physical layer data of the solid-state hard disk using a topological coloring algorithm and a graph cut algorithm; detecting abnormal behavior of storage units in each region using an isolation forest algorithm to obtain abnormal storage units; obtaining incremental change data of the abnormal storage units; generating an optimization strategy set based on a neural network model using the feature data and incremental change data of the abnormal storage units as input, so that different regions receive different optimization treatments; and optimizing the storage of the solid-state hard disk after obtaining the optimal execution strategy based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal through multi-objective tuning. The present invention can apply different optimization strategies to the internal region, boundary region, and external region of the solid-state hard disk.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to a solid-state hard disk storage optimization method and system. Background Art

[0002] Solid-state drive (SSD) storage optimization methods generally refer to improving the read / write performance, durability, and efficiency of SSDs through intelligent algorithms or hardware architectures. Common SSD optimization methods include:

[0003] 1. Garbage Collection: SSDs use garbage collection to clean up no longer needed data blocks. Because SSDs use flash storage, when data is deleted, it isn't actually removed from the storage immediately. Instead, it's marked as recyclable. Garbage collection periodically cleans up this unused data to make room for new data, thereby improving performance.

[0004] 2. TRIM command: TRIM is a command issued by the operating system that notifies the SSD which data is no longer in use and can be safely erased. This helps improve the performance of the SSD, especially in scenarios where storage space is frequently written, because it reduces the accumulation of invalid data.

[0005] 3. Write Balance: Each memory cell in the flash memory chip has a limited number of write cycles. Write Balance technology uses an intelligent algorithm to distribute the write load across all memory cells, avoiding excessive wear in certain areas and extending the life of the SSD.

[0006] 4. Compression and encryption technology: SSDs use compression technology to reduce storage space usage or encryption to ensure data security. Compression technology can reduce the space occupied by data without affecting performance, thereby improving storage efficiency.

[0007] 5. Cache technology: To improve read and write speeds, many SSDs are equipped with high-speed cache (such as DRAM cache), making read and write operations faster. Intelligent algorithms optimize cache management to ensure that frequently accessed data can be read and written quickly.

[0008] However, existing SSD storage optimization methods still have the following shortcomings:

[0009] (1) Uneven performance of storage cells: Storage cells in SSDs often exhibit different performance characteristics at different locations and under different load conditions. Traditional methods fail to fully account for these differences, resulting in large fluctuations in storage cell performance.

[0010] (2) Difficulty in optimizing SSD storage units: The performance optimization of SSD storage units is often affected by uneven load, reduced lifespan, and data access speed issues. Traditional optimization methods have difficulty balancing the different needs of each storage area and are unable to achieve intelligent and dynamic adjustment and optimization.

[0011] (3) Multi-objective optimization problem: In the process of SSD storage optimization, it is usually necessary to balance multiple optimization objectives, such as performance, energy efficiency, load balancing, and durability. These objectives are sometimes contradictory, and a single optimization objective may lead to performance degradation in other aspects.

[0012] (4) Detection and processing of abnormal storage cells: Abnormal storage cells in SSDs may cause performance degradation, but traditional storage management methods find it difficult to effectively detect and process these abnormal areas, especially in large-scale storage systems.

[0013] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0014] In response to the problems in the related art, the present invention proposes a solid state drive storage optimization method and system to overcome the above technical problems existing in the existing related art.

[0015] To this end, the specific technical solutions adopted in the present invention are as follows:

[0016] According to one aspect of the present invention, a solid-state hard disk storage optimization method is provided, comprising:

[0017] S1. Based on the behavioral layer data and physical layer data of the solid-state drive, the storage unit is divided into regions using the topological coloring algorithm and the graph cut algorithm to obtain the internal region, boundary region, and external region, specifically including:

[0018] Based on the behavioral layer data and physical layer data of the solid-state drive, a topological relationship model between storage units is established; based on the topological relationship model between storage units, the adjacency relationship between storage units is identified;

[0019] Based on the adjacency relationship between storage units, the storage units are divided into different areas using the topological coloring function;

[0020] Optimize the storage unit area partitioning results by minimizing the cutting cost of the graph;

[0021] S2. Use the isolation forest algorithm to detect abnormal behavior of the storage units in each area to obtain abnormal storage units; obtain the incremental change data of the abnormal storage units;

[0022] S3. Based on the neural network model, the feature data and change increment data of the abnormal storage unit are used as input to generate an optimization strategy set so that different areas can receive different optimization treatments;

[0023] S4. Based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal, after obtaining the optimal execution strategy through multi-objective tuning, the storage of the solid-state drive is optimized.

[0024] Furthermore, a topological relationship model between storage units is established based on the behavioral layer data and the physical layer data of the solid-state drive; and identifying the adjacency relationship between storage units based on the topological relationship model between storage units includes:

[0025] Extract the physical location information of each storage unit and convert it into spatial coordinates. At the same time, extract the performance boundary data of each storage unit. Based on the physical location information and performance boundary data of the storage units, obtain the topological relationship model between the storage units.

[0026] Calculating the adjacency relationship between the storage units based on the physical location information of the storage units in the topological relationship model, wherein the distance between the storage units is determined by the Euclidean distance;

[0027] A distance threshold is obtained. When the distance between the storage units is less than the distance threshold, the two storage units are adjacent storage units.

[0028] Furthermore, based on the adjacency relationship between storage cells, the topological coloring function is used to divide the storage cells into different areas, including:

[0029] Construct a topological map of storage units based on the adjacency relationship between each storage unit and storage unit;

[0030] Based on the encirclement and connection rules of the storage cells, a topological coloring function is established:

[0031] ;

[0032] Where, V Represents a storage unit, A represents the internal area, B represents the boundary area, and C represents the external area;

[0033] Using the topological coloring function and combining the performance data and physical location data of the storage unit, the storage unit is preliminarily classified into internal areas, boundary areas and external areas;

[0034] Based on the preliminary classification results, the topology graph of the storage units is traversed starting from any unmarked storage unit using a depth-first search method;

[0035] During the traversal process, when all the adjacent nodes of a node in the topology graph of the storage unit are marked as the same region, the node is marked as the same region;

[0036] When some of the adjacent nodes of a node in the topology graph of the storage unit belong to different regions, the node is marked as a boundary region;

[0037] For a node that has no adjacent nodes or whose adjacent nodes are not marked, the corresponding node is marked as an external area.

[0038] Furthermore, by minimizing the graph cutting cost, the optimization of the storage unit area partition results includes:

[0039] Construct an objective function for evaluating the quality of storage unit region division and minimize the cutting cost between different regions. The objective function for evaluating the quality of storage unit region division is:

[0040] ;

[0041] Where, Cost (G) represents the objective function for evaluating the quality of storage unit region partitioning;

[0042] G represents a topological graph of storage units including storage units and adjacency relationships. E Represents the edge set in the topological graph;

[0043] w ij Represents a storage unit i and j The weight of the edge between them;

[0044] Used to represent storage units i and j Whether it is divided into different areas;

[0045] Construct an initial flow network based on the topological graph of the storage unit, determine the source nodes and sink nodes, and use the maximum flow algorithm to calculate the maximum flow from the source node to the sink node. Based on the maximum flow result, identify the minimum cut set of the graph;

[0046] The area division of the storage unit is adjusted according to the result of the minimum cut. If a storage unit belongs to the minimum cut, the area of the corresponding storage unit needs to be updated to ensure that the optimized area meets the minimization requirement of the objective function for evaluating the quality of the storage unit area division.

[0047] Furthermore, the isolation forest algorithm is used to detect abnormal behavior of the storage units in each region, and the abnormal storage units include:

[0048] By extracting the behavioral layer data features of the storage units in each region and inputting them into the isolation forest algorithm as training data, the behavioral layer data features include throughput, latency, and load;

[0049] Train the isolation forest model for each storage unit in each region and calculate the anomaly score for each storage unit; configure different anomaly score thresholds for different regions;

[0050] When the anomaly score of a storage unit exceeds the corresponding anomaly score threshold, it is marked as an abnormal storage unit;

[0051] Among them, when training the isolation forest model for the internal area, the behavioral layer data features of the moment-level time window are used; when training the isolation forest model for the boundary area, the behavioral layer data features of the minute-level time window are used; when training the isolation forest model for the external area, the storage unit behavior data under high load conditions is preferentially used.

[0052] Furthermore, obtaining the incremental change data of the abnormal storage unit includes:

[0053] Acquire behavioral layer data of abnormal storage units within a pre-set collection period;

[0054] Calculate the increment and change rate of behavioral layer data within the time window;

[0055] Based on the increment and change rate of behavior layer data, and statistical analysis of the increment and change rate, the key features in the change increment data are extracted.

[0056] Furthermore, based on the neural network model, the characteristic data and change increment data of the abnormal storage unit are used as input to generate an optimization strategy set so that different areas can receive different optimization treatments, including:

[0057] Determine the input layer, hidden layer, and output layer of the neural network model, and construct activation functions and loss functions;

[0058] The characteristic data and incremental change data of the abnormal storage unit are used as training data. For each sample of the abnormal storage unit, a corresponding optimization strategy label is generated; the neural network model is trained and parameters are adjusted;

[0059] The trained neural network model determines the region to which the storage unit belongs through the input feature data and change increment data, and generates the corresponding optimization strategy.

[0060] Furthermore, based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal, after obtaining the optimal execution strategy through multi-objective tuning, the storage of the solid-state drive is optimized, including:

[0061] Pre-set various optimization goals for the solid-state drive, including performance optimization goals, load balancing goals, energy efficiency goals, and durability goals, and set weights for each optimization goal;

[0062] By combining the objective functions of various optimization goals, a multi-objective optimization model is established. By simulating the impact of the optimization strategy set on the SSD, the performance of each objective function is evaluated, and corresponding optimization strategies are applied to different regions.

[0063] Monitor the execution effects of optimization strategies in different regions in real time and collect SSD performance indicators;

[0064] Evaluate the performance indicators of the optimized solid-state drives; based on the evaluation results, determine the optimal execution strategy for each region;

[0065] Utilize the optimal execution strategy for each area to optimize storage in different areas of the solid-state drive.

[0066] Furthermore, by utilizing the optimal execution strategy for each region, storage optimization is performed on different regions of the solid-state drive, including:

[0067] Implement optimal execution strategies in the inner, boundary, and outer areas of the SSD and monitor the performance and energy efficiency of each area;

[0068] Based on the performance and energy efficiency of each area obtained through real-time monitoring, the optimal execution strategy for each area is dynamically adjusted.

[0069] According to another aspect of the present invention, a solid-state hard disk storage optimization system is also provided, including a hard disk partitioning module, an abnormal unit determination and incremental data acquisition module, an optimization strategy generation module and a multi-objective tuning module, and the hard disk partitioning module, the abnormal unit determination and incremental data acquisition module, the optimization strategy generation module and the multi-objective tuning module are connected in sequence.

[0070] The hard disk partitioning module is used to divide the storage unit into regions based on the behavioral layer data and physical layer data of the solid-state hard disk, and use the topological coloring algorithm and the graph cut algorithm to obtain internal regions, boundary regions and external regions; the abnormal unit determination and incremental data acquisition module is used to use the isolation forest algorithm to detect abnormal behavior of the storage unit in each region to obtain abnormal storage units; and obtain the incremental data of the abnormal storage units; the optimization strategy generation module is used to generate an optimization strategy set based on the neural network model, using the characteristic data and incremental data of the abnormal storage unit as input, so that different regions can receive different optimization treatments; the multi-objective tuning module is used to optimize the storage of the solid-state hard disk after obtaining the optimal execution strategy based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal through multi-objective tuning.

[0071] The beneficial effects of the present invention are:

[0072] (1) The present invention can accurately and intelligently partition the performance of the storage units of the solid-state drive through comprehensive analysis of behavioral layer and physical layer data, and by using topological coloring algorithms and graph cut algorithms, so that storage units with similar performance characteristics can be divided into the same area, avoiding the performance fluctuation problem that may be caused by traditional simple partitioning methods. After the area division, storage resources can be reasonably allocated according to the specific performance and physical location of the storage unit, avoiding performance bottlenecks and resource waste caused by performance imbalance. In addition, the area division of the solid-state drive helps to give priority to storage units with stable performance for core data storage, thereby extending the life of the storage unit. In particular, in the boundary area and external area, the storage unit may be more likely to experience performance degradation. Reasonable division of these areas can effectively share the load and avoid overloading of a single storage unit.

[0073] (2) By applying different optimization strategies to the internal, boundary, and external areas of the SSD, the overall performance of the SSD can be improved, including higher throughput, lower latency, and more balanced load distribution. In particular, under high load conditions, by optimizing write order, data migration, and load balancing, the system's response speed and stability can be greatly improved.

[0074] (3) Based on the principle of balancing the optimization strategy of abnormal storage units with the overall optimization goal, multi-objective tuning is used to effectively balance the performance optimization and durability optimization of solid-state drives, thus avoiding the performance degradation or life shortening problems that may be caused by single-objective optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0076] Figure 1 is a flow chart of a solid-state hard disk storage optimization method according to an embodiment of the present invention;

[0077] Figure 2 This is a principle block diagram of a solid-state hard disk storage optimization system according to an embodiment of the present invention.

[0078] In the picture:

[0079] 1. Hard disk partitioning module; 2. Abnormal unit determination and incremental data acquisition module; 3. Optimization strategy generation module; 4. Multi-objective tuning module. DETAILED DESCRIPTION

[0080] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0081] According to an embodiment of the present invention, a solid state drive storage optimization method and system are provided.

[0082] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 According to one embodiment of the present invention, a method for optimizing solid-state hard disk storage is provided, comprising:

[0083] S1. Based on the behavioral layer data and physical layer data of the solid-state drive, the storage unit is divided into regions using the topological coloring algorithm and the graph cut algorithm to obtain the internal region, the boundary region and the external region.

[0084] Based on the behavioral layer data and physical layer data of the solid-state drive, the storage unit is divided into regions using the topological coloring algorithm and the graph cut algorithm, including:

[0085] A topological relationship model between storage units is established based on the behavioral layer data and physical layer data of the solid-state drive; and adjacency relationships between storage units are identified based on the topological relationship model between storage units.

[0086] According to the adjacency relationship between storage cells, the storage cells are divided into different regions using the topological coloring function.

[0087] The storage unit region division result is optimized by minimizing the cutting cost of the graph; the region type is marked for each storage unit, including internal region, boundary region and external region.

[0088] It's important to note that by using topological color and graph cut algorithms to partition storage units based on SSD behavioral and physical layer data, we can effectively identify and optimize storage unit layout. By building a topological relationship model and analyzing the adjacency relationships between storage units, we can precisely partition storage areas, improving load balancing and performance stability. Minimizing the graph cut cost further optimizes the partitioning.

[0089] In one embodiment, a topological relationship model between storage units is established based on behavioral layer data and physical layer data of the solid-state drive; and identifying adjacency relationships between storage units based on the topological relationship model between storage units includes:

[0090] The physical location information of each storage unit is extracted and converted into spatial coordinates. At the same time, the performance boundary data of each storage unit is extracted. Based on the physical location information and performance boundary data of the storage units, a topological relationship model between the storage units is obtained.

[0091] The adjacency relationship between the storage units is calculated according to the physical location information of the storage units in the topological relationship model, wherein the distance between the storage units is determined by the Euclidean distance.

[0092] A distance threshold is obtained. When the distance between the storage units is less than the distance threshold, the two storage units are adjacent storage units.

[0093] In one embodiment, dividing the storage cells into different regions using a topological coloring function according to the adjacency relationship between the storage cells includes:

[0094] According to the adjacency relationship between each storage unit and storage unit, a topological graph of the storage unit (i.e., a weighted undirected graph) is constructed.

[0095] Based on the encirclement and connection rules of the storage cells, a topological coloring function is established:

[0096] ;

[0097] Where, V Represents a storage unit, A represents the inner area, B represents the boundary area, and C represents the outer area.

[0098] The topological coloring function is used in combination with the performance data and physical location data of the storage units to preliminarily classify the storage units into internal areas, boundary areas and external areas.

[0099] Based on the preliminary classification results, the topology graph of the storage units is traversed starting from any unmarked storage unit using a depth-first search method.

[0100] During the traversal process, when all adjacent nodes of a node in the topology graph of the storage unit are marked as the same region, the node is marked as the same region.

[0101] When some of the adjacent nodes of a node in the topology graph of the storage unit belong to different areas, the node is marked as a boundary area.

[0102] For a node that has no adjacent nodes or whose adjacent nodes are not marked, the corresponding node is marked as an external area.

[0103] Among them, the topological shading function is based on the following principles:

[0104] Internal area (A): The area completely surrounded by other storage units.

[0105] Boundary region (B): A region that is partially surrounded by other storage cells but has direct connections to the outside.

[0106] External area (C): An area that is not connected to other storage units or is only indirectly connected.

[0107] In one embodiment, optimizing the storage unit region partitioning result by minimizing the graph cut cost includes:

[0108] Construct an objective function for evaluating the quality of storage unit region division and minimize the cutting cost between different regions. The objective function for evaluating the quality of storage unit region division is:

[0109] ;

[0110] Where, Cost (G) represents the objective function for evaluating the quality of the storage cell area division, which represents the cost of dividing the storage cell area. The goal is to minimize this cost and ensure that the storage cells in the same area have similar performance characteristics. In other words, the more reasonable the division, the lower the cost; G represents the topological graph of the storage cells including the storage cells and their adjacency relationships. E Represents the edge set in the topological graph; w ij Represents a storage unit i and j The weight of the edge between them is usually calculated based on their performance differences (such as I / O throughput, latency, etc.). The larger the weight, the more similar the performance between the storage units and the closer the adjacency relationship; Indicates the indicator function, used to indicate the storage unit i and j Is it divided into different areas, if the storage unit i and j is divided into different areas, the value is 1; if the storage unit i and j If they are divided into the same area, the value is 0.

[0111] An initial flow network is constructed according to the topological graph of the storage unit, the source nodes and sink nodes are determined, and the maximum flow from the source node to the sink node is calculated using the maximum flow algorithm. Based on the maximum flow result, the minimum cut set of the graph is identified.

[0112] The regional division of storage cells is adjusted according to the result of the minimum cut. If a storage cell belongs to the minimum cut, the region of the corresponding storage cell needs to be updated to ensure that the optimized region meets the minimization requirement of the objective function for evaluating the quality of the storage cell regional division. Special attention is paid to those storage cells located in the boundary area, because these cells may be adjacent to multiple different types of regions at the same time.

[0113] Specifically, in SSD storage optimization, the topology coloring algorithm divides storage cells into different regions (such as internal regions, boundary regions, and external regions) by identifying the adjacency relationships between them. This division helps optimize storage system performance, load balancing, and lifespan extension. The topology coloring function is used to assign nodes in a graph to different regions or classes based on their topological relationships. In the SSD storage optimization method, the topology coloring function divides storage cells into internal regions (A), boundary regions (B), and external regions (C) based on the adjacency relationships between each storage cell and other storage cells.

[0114] The graph cut algorithm is a technique for dividing a graph into multiple regions by cutting edges within the graph. In this application, the graph cut algorithm is used to optimize the regional division of storage cells and minimize the cutting cost between regions. The goal is to ensure that storage cells within the same region have similar performance characteristics to avoid performance imbalances. The minimum cut cost refers to the cost of dividing the region during the graph segmentation process. This cost is related to factors such as the adjacency relationship between storage cells and performance differences. By minimizing the cut cost, a more reasonable storage cell division is ensured, and the allocation of storage resources is optimized. The maximum flow algorithm is an algorithm used to calculate the maximum flow between source nodes and sink nodes in a network. The maximum flow algorithm is used to calculate the minimum cut of a graph. The maximum flow results can be used to identify the minimum cut set of the graph, thereby determining the storage cell regions that need to be adjusted. The purpose of this step is to optimize the storage cell regional division so that the performance characteristics of each region are as consistent as possible.

[0115] When using the Max Flow Min Cut algorithm, source nodes and sink nodes are identified, representing the different region categories to be distinguished. A maximum flow algorithm (such as the Ford-Fulkerson algorithm or Boykov's new algorithm) is used to calculate the maximum flow from the source node to the sink node, and a cut set of the graph is found that minimizes the net flow through the cut set. The region partitioning is optimized by minimizing the graph cut cost. The specific steps are as follows:

[0116] Initialize the flow network: Construct the initial flow network based on the weighted undirected graph G.

[0117] Maximum flow calculation: Use the maximum flow algorithm to calculate the maximum flow value.

[0118] Minimum cut identification: Based on the maximum flow results, identify the minimum cut set of the graph.

[0119] S2. Use the isolation forest algorithm to detect abnormal behavior of the storage units in each area to obtain abnormal storage units; obtain the incremental change data of the abnormal storage units.

[0120] In one embodiment, the isolation forest algorithm is used to detect abnormal behavior of storage units in each region, and the abnormal storage units include:

[0121] The behavioral layer data features of the storage units in each region are extracted and input into the isolation forest algorithm as training data. The behavioral layer data features include throughput, latency, and load.

[0122] The isolation forest model is trained for each storage unit in each region, and an anomaly score is calculated for each storage unit. Different anomaly score thresholds are configured for different regions. For example, a higher anomaly score threshold is set for the boundary area to capture more fluctuation behaviors.

[0123] When the anomaly score of a storage unit exceeds the corresponding anomaly score threshold, it is marked as an abnormal storage unit.

[0124] Among them, when training the isolation forest model for the internal area, the behavioral layer data features of the moment-level time window are used; when training the isolation forest model for the boundary area, the behavioral layer data features of the minute-level time window are used; when training the isolation forest model for the external area, the storage unit behavior data under high load conditions is preferentially used.

[0125] In one embodiment, obtaining the incremental change data of the abnormal storage unit includes:

[0126] During a preset collection period, behavioral layer data of abnormal storage units is obtained.

[0127] Calculate the increment and change rate of behavior layer data within the time window.

[0128] Based on the increment and change rate of the behavior layer data, and statistical analysis of the increment and change rate, the key features of the change increment data are extracted, including the maximum change value, the minimum change value, the average change and the standard deviation.

[0129] The isolation forest algorithm, a tree-based anomaly detection algorithm, isolates data samples by randomly selecting features and split points. Randomly splitting anomalies requires fewer splitting steps, resulting in shorter paths in the isolation forest algorithm.

[0130] Behavioral layer data: refers to various indicator data generated by the storage unit during its actual operation, such as:

[0131] Throughput: refers to the read and write speed or data transfer rate per unit time.

[0132] Latency: refers to the response time of a request or operation.

[0133] Load: refers to the resource consumption of a storage unit under workload (such as CPU, memory, and I / O load).

[0134] These feature data are used as input data in the Isolation Forest algorithm to help the model learn the normal behavior patterns of storage units and detect abnormal behavior.

[0135] Border regions are often at the edge of data migrations with significant load fluctuations. Therefore, detecting abnormal behavior requires high sensitivity to capture these fluctuations. The recommended threshold range is 0.7 to 0.9 (based on the anomaly score of the Isolation Forest algorithm). Anomalies are only flagged as abnormal when the anomaly score is significantly higher than the normal behavior pattern. Border regions experience greater volatility, so a higher threshold can prevent normal fluctuations from being misclassified as abnormalities.

[0136] The load in internal areas is typically more stable and less variable, so the anomaly scoring threshold doesn't need to be set too high. The recommended threshold range is 0.4 to 0.6. A too low threshold can cause normal fluctuations to be misclassified as anomalies, while a too high threshold can cause true anomalies to be overlooked. Therefore, a moderate threshold helps balance sensitivity with false positive rates.

[0137] External regions experience greater volatility due to high load or external factors, requiring a higher sensitivity for anomaly scoring. The recommended threshold range is 0.5 to 0.7, but it's preferable to train the model using behavioral data from high-load conditions. External regions may experience significant load fluctuations, so a lower threshold can capture potential anomalies in a timely manner, preventing fluctuations caused by high load from going undetected.

[0138] Threshold selection is adjusted based on:

[0139] Historical data: By analyzing behavioral patterns in different areas of historical data, you can set preliminary threshold ranges. Based on historical data such as throughput, latency, and load, you can assess which data changes are normal and which are abnormal.

[0140] Model training: The Isolation Forest model is trained on data from different regions to obtain more accurate anomaly scoring intervals. During this process, the threshold is dynamically adjusted to ensure that the model can accurately capture abnormal behavior without generating excessive false positives.

[0141] S3. Based on the neural network model, the feature data and change increment data of the abnormal storage unit are used as input to generate an optimization strategy set so that different areas can receive different optimization treatments.

[0142] In one embodiment, based on a neural network model, characteristic data and incremental change data of abnormal storage units are used as input to generate an optimization strategy set so that different areas receive different optimization treatments, including:

[0143] Determine the input layer, hidden layer, and output layer of the neural network model, and construct the activation function and loss function.

[0144] The characteristic data and change increment data of the abnormal storage unit are used as training data. For each sample of the abnormal storage unit, a corresponding optimization strategy label is generated; the neural network model is trained and the parameters are adjusted.

[0145] The trained neural network model uses the input feature data and change increment data to determine the area (internal area, boundary area, external area, etc.) to which the storage unit belongs, and generates a corresponding optimization strategy.

[0146] In the neural network model, the input layer processes the characteristic data and incremental change data of abnormal storage units. Characteristic data may include performance metrics of storage units (such as throughput, latency, and load), while incremental change data reflects the changes in these metrics within a specific time window (such as maximum change, minimum change, and standard deviation). This data is passed to the neural network as input features. The hidden layer performs nonlinear transformations on the input data, helping the model learn complex patterns. During training, the neural network continuously adjusts parameters (such as weights) to minimize the loss function. The output layer is responsible for generating a set of optimization strategies. Based on the model's training objectives, the output layer generates a set of strategies (such as load balancing, performance optimization, and fault recovery), and generates a corresponding optimization strategy based on the region and status of each storage unit.

[0147] S4. Based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal, after obtaining the optimal execution strategy through multi-objective tuning, the storage of the solid-state drive is optimized.

[0148] In one embodiment, based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal, after obtaining the optimal execution strategy through multi-objective tuning, the storage of the solid-state drive is optimized, including:

[0149] Various optimization goals of the solid-state drive are pre-set, including performance optimization goals, load balancing goals, energy efficiency goals, and durability goals, and weights are set for each optimization goal.

[0150] By merging the objective functions of various optimization goals, a multi-objective optimization model is established; by simulating the impact of the optimization strategy set on the solid-state drive, the performance of each objective function is evaluated, and corresponding optimization strategies are applied to different areas.

[0151] Monitor the execution effects of optimization strategies in different areas in real time and collect SSD performance indicators.

[0152] Evaluate the performance indicators of the optimized solid-state drive; based on the evaluation results, determine the optimal execution strategy for each area.

[0153] Utilize the optimal execution strategy for each area to optimize storage in different areas of the solid-state drive.

[0154] In one embodiment, optimizing storage in different areas of a solid-state drive using the optimal execution strategy for each area includes:

[0155] Implement optimal execution strategies in the inner, boundary, and outer areas of the SSD and monitor the performance and energy efficiency of each area.

[0156] Based on the performance and energy efficiency of each area obtained through real-time monitoring, the optimal execution strategy for each area is dynamically adjusted.

[0157] Among them, the multi-objective optimization process:

[0158] Simulating the impact of an optimization strategy set: Optimization strategies are generated using models such as neural networks. These strategies address specific issues in each region and propose optimization measures. During the simulation, different optimization strategies are applied to different regions and their impact on the overall SSD performance is evaluated. During the evaluation, an objective function is used to quantify the effectiveness of each strategy.

[0159] Monitoring and Evaluation: Real-time monitoring is an integral part of the optimization process. By monitoring SSD performance metrics (such as throughput, latency, load, and energy efficiency), we can continuously evaluate the effectiveness of each optimization strategy and dynamically adjust it based on real-time data.

[0160] For example, when monitoring detects slow performance improvement in a certain area (such as an internal area), the optimization strategy will be adjusted to increase attention to performance; if it is found that the energy efficiency of a certain area does not meet expectations, energy efficiency optimization measures will be strengthened.

[0161] By combining multi-objective optimization models (such as particle swarm optimization and genetic algorithms), real-time monitoring, and dynamic adjustments, we can implement the most appropriate optimization strategy for storage units in different regions. Each region's optimization strategy is tailored to its characteristics (such as load fluctuations, performance requirements, and energy efficiency needs), thereby improving the overall performance, load balancing, energy efficiency, and durability of the SSD.

[0162] like Figure 2As shown, according to another embodiment of the present invention, a solid-state hard disk storage optimization system is also provided, including a hard disk partitioning module 1, an abnormal unit determination and incremental data acquisition module 2, an optimization strategy generation module 3 and a multi-objective tuning module 4, and the hard disk partitioning module 1, the abnormal unit determination and incremental data acquisition module 2, the optimization strategy generation module 3 and the multi-objective tuning module 4 are kept connected in sequence.

[0163] The hard disk partitioning module 1 is used to divide the storage unit into regions based on the behavioral layer data and physical layer data of the solid-state hard disk, and use the topological coloring algorithm and the graph cut algorithm to obtain internal regions, boundary regions and external regions; the abnormal unit determination and incremental data acquisition module 2 is used to use the isolation forest algorithm to detect abnormal behavior of the storage unit in each region to obtain abnormal storage units; and obtain the incremental data of the abnormal storage units; the optimization strategy generation module 3 is used to generate an optimization strategy set based on the neural network model, using the feature data and incremental data of the abnormal storage unit as input, so that different regions can receive different optimization treatments; the multi-objective tuning module 4 is used to optimize the storage of the solid-state hard disk after obtaining the optimal execution strategy based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal through multi-objective tuning.

[0164] In order to facilitate understanding of the above technical solutions of the present invention, the working principle of the present invention in actual process is described in detail below.

[0165] A solid-state drive (SSD) contains 2,500 storage cells, each of which has different behavioral layer data (such as I / O performance, read and write load, etc.) and physical layer data (such as physical location, lifespan, performance boundary, etc.).

[0166] 1. Divide the storage unit into regions based on topological coloring algorithm and graph cut algorithm

[0167] 1. Extract physical location information and performance boundary data

[0168] Physical location: The location of each storage unit is represented by two-dimensional coordinates. Performance boundary data: including throughput, latency, and load.

[0169] Some of the extracted data are shown in Table 1.

[0170] Table 1 Extracted physical location information and performance boundary data

[0171]

[0172] 2. Build a topological relationship model and identify adjacency relationships

[0173] Euclidean distance, for example, the distance between storage unit 1 and storage unit 2 is 7.07. Since 7.07 < 10 (distance threshold), they are adjacent storage units.

[0174] 3. Use topological coloring function for preliminary classification

[0175] A topology map was constructed and traversed using a depth-first search (DFS) algorithm to mark the area types of storage units. Some of the results are shown in Table 2.

[0176] Table 2 Area types of storage units

[0177]

[0178] 4. Minimize the cut cost of the graph

[0179] The minimum cut set is calculated using the maximum flow algorithm and the region partitioning results are adjusted. Some of the final partitioning results are shown in Table 3.

[0180] Table 3 Final division results

[0181]

[0182] 2. Anomaly Detection Using Isolation Forest Algorithm

[0183] 1. Training Isolation Forest Model

[0184] For each region (inner region, boundary region, outer region), behavioral layer data features (throughput, latency, load) are extracted as training data.

[0185] 2. Anomaly Scoring and Threshold Setting

[0186] Anomaly scores are calculated for each storage unit, and anomaly score thresholds are set. Some results are shown in Table 4.

[0187] Table 4 Storage abnormality results

[0188]

[0189] 3. Generate Optimization Strategy Set Based on Neural Network

[0190] 1. Determine the neural network model structure

[0191] Build a neural network model, determine the input layer, hidden layer, and output layer, and set the activation function and loss function. For example, the input layer is the feature data and change increment data of the abnormal storage unit, and the output layer is the optimization strategy label.

[0192] 2. Training the neural network model

[0193] The characteristic data and change increment data of the abnormal storage unit are used as training data to generate the corresponding optimization strategy label. The trained neural network model can generate the corresponding optimization strategy based on the input data.

[0194] 4. Multi-objective Tuning and Optimal Execution Strategy

[0195] Pre-set various optimization goals and their weights for the solid-state drive:

[0196] Performance optimization target: weight 0.4.

[0197] Load balancing target: weight 0.3.

[0198] Energy efficiency target: weight 0.2.

[0199] Durability target: weight 0.1.

[0200] By combining the objective functions of each optimization goal, a multi-objective optimization model is established. The impact of the optimization strategy set on the SSD is simulated to evaluate the performance of each objective function. Specific optimization strategies for abnormal storage units are also evaluated to assess their impact on the overall optimization goal.

[0201] In order to ensure that the optimization strategy of abnormal storage units does not excessively affect the overall optimization goal, it is necessary to find a balance between the two. Specific methods include:

[0202] Introducing constraints: Set constraints for each optimization strategy to ensure that its impact on the overall optimization goal is within an acceptable range. Weighted comprehensive scoring: Calculate the impact of each optimization strategy on each optimization goal and give a comprehensive score based on the weights.

[0203] For example, a neural network model generates an optimization strategy for a specific area: load distribution and latency reduction. The results of this strategy are as follows: Throughput improvement: 5%. Load variation: -10%. Latency variation: -10%. Energy consumption variation: 0%. Lifespan variation: 0%.

[0204] According to the weight of each optimization goal, the comprehensive score is calculated: Since the comprehensive score is negative, it means that the strategy has a negative impact on the overall optimization goal. Therefore, the strategy needs to be adjusted until the optimal balance point is reached.

[0205] In summary, the present invention comprises:

[0206] (1) By constructing a topological relationship model between storage cells and using a topological coloring algorithm to divide the storage cells into internal, boundary, and external regions, the problem of uneven storage cell performance was initially resolved. The storage cells were partitioned based on their location and performance characteristics, ensuring that storage cells within the same region had similar performance characteristics. The region division was further optimized by minimizing the graph cut cost, ensuring that storage cells within the same region had as similar performance characteristics as possible, thereby reducing performance fluctuations.

[0207] (2) By presetting multiple optimization objectives for the SSD (such as performance, energy efficiency, load balancing, and durability) and assigning weights to each objective, a multi-objective optimization model was established. By simulating the impact of the optimization strategy set on the SSD, the performance of each objective function was evaluated, and corresponding optimization strategies were applied to different areas.

[0208] (3) The isolation forest algorithm is used to detect abnormal behavior of storage units in each region, identify abnormal storage units, and obtain their incremental change data. Based on the neural network model, the feature data and incremental change data of abnormal storage units are used as input to generate a specific optimization strategy to deal with abnormal storage units in a targeted manner.

[0209] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A solid state drive storage optimization method, characterized in that: include: S1. Based on the behavioral layer data and physical layer data of the solid-state drive, the storage unit is divided into regions using the topological coloring algorithm and the graph cut algorithm to obtain the internal region, boundary region, and external region, specifically including: Extracting the physical location information of each storage unit and converting it into spatial coordinates, and simultaneously extracting the performance boundary data of each storage unit; obtaining a topological relationship model between the storage units based on the physical location information and performance boundary data of the storage units; calculating the adjacency relationship between the storage units based on the physical location information of the storage units in the topological relationship model, wherein the distance between the storage units is determined by the Euclidean distance; obtaining a distance threshold; when the distance between the storage units is less than the distance threshold, the two storage units are adjacent storage units; Based on the adjacency relationship between storage cells, the topological coloring function is used to divide the storage cells into different regions. The storage cell region division result is optimized by minimizing the graph cutting cost. S2. Use the isolation forest algorithm to detect abnormal behavior of the storage units in each region to obtain abnormal storage units; obtain the incremental change data of the abnormal storage units, specifically including: The behavioral layer data features of the storage units in each region are extracted and input into the isolation forest algorithm as training data. The behavioral layer data features include throughput, latency, and load. The isolation forest model is trained for the storage units in each region, and an anomaly score is calculated for each storage unit. Different anomaly score thresholds are configured for different regions. When the anomaly score of a storage unit exceeds the corresponding anomaly score threshold, it is marked as an abnormal storage unit. When training the isolation forest model for the internal region, the behavioral layer data features of the moment-level time window are used; when training the isolation forest model for the boundary region, the behavioral layer data features of the minute-level time window are used. When training the isolation forest model for the external region, the behavioral data of the storage units under high load conditions are preferentially used. S3. Based on the neural network model, the feature data and incremental change data of the abnormal storage unit are used as input to generate an optimization strategy set so that different areas receive different optimization treatments. Specifically, the optimization strategy set includes: Determine the input layer, hidden layer, and output layer of the neural network model, and construct the activation function and loss function; use the feature data and incremental change data of the abnormal storage unit as training data, and generate a corresponding optimization strategy label for each abnormal storage unit sample; train the neural network model and adjust the parameters; the trained neural network model uses the input feature data and incremental change data to determine the region to which the storage unit belongs and generate a corresponding optimization strategy; S4. Based on the optimization strategy set and balancing the optimization strategy of the abnormal storage unit with the overall optimization goal, after obtaining the optimal execution strategy through multi-objective tuning, the storage of the solid-state drive is optimized, including: Predefine various optimization objectives of the solid-state drive, including performance optimization objectives, load balancing objectives, energy efficiency objectives and durability objectives, and set weights for each optimization objective; establish a multi-objective optimization model by merging the objective functions of various optimization objectives; evaluate the performance of each objective function by simulating the impact of the optimization strategy set on the solid-state drive, and apply corresponding optimization strategies to different areas; monitor the execution effects of the optimization strategies in different areas in real time, and collect solid-state drive performance indicators; evaluate the performance indicators of the optimized solid-state drive; determine the optimal execution strategy for each area based on the evaluation results; and use the optimal execution strategy for each area to optimize storage in different areas of the solid-state drive.

2. The solid state drive storage optimization method according to claim 1, wherein: The method of dividing the storage units into different areas by using a topological coloring function according to the adjacency relationship between the storage units includes: Construct a topological map of storage units based on the adjacency relationship between each storage unit and storage unit; Based on the encirclement and connection rules of the storage cells, a topological coloring function is established: f : V (A,B,C); Where, V Represents a storage unit, A represents the internal area, B represents the boundary area, and C represents the external area; Using the topological coloring function and combining the performance data and physical location data of the storage unit, the storage unit is preliminarily classified into internal areas, boundary areas and external areas; Based on the preliminary classification results, the topology graph of the storage units is traversed starting from any unmarked storage unit using a depth-first search method; During the traversal process, when all the adjacent nodes of a node in the topology graph of the storage unit are marked as the same region, the node is marked as the same region; When some of the adjacent nodes of a node in the topology graph of the storage unit belong to different regions, the node is marked as a boundary region; For a node that has no adjacent nodes or whose adjacent nodes are not marked, the corresponding node is marked as an external area.

3. The solid state drive storage optimization method according to claim 1, wherein: Optimizing the storage unit region partitioning result by minimizing the graph cutting cost includes: Construct an objective function for evaluating the quality of storage unit region division and minimize the cutting cost between different regions. The objective function for evaluating the quality of storage unit region division is: ； Where, Cost (G) represents the objective function for evaluating the quality of storage unit area partitioning; G represents a topological graph of storage units including storage units and adjacency relationships. E Represents the edge set in the topological graph; w ij Represents a storage unit i and j The weight of the edge between them; Used to represent storage units i and j Whether it is divided into different areas; Construct an initial flow network based on the topological graph of the storage unit, determine the source nodes and sink nodes, and use the maximum flow algorithm to calculate the maximum flow from the source node to the sink node. Based on the maximum flow result, identify the minimum cut set of the graph; The area division of the storage unit is adjusted according to the result of the minimum cut. If a storage unit belongs to the minimum cut, the area of the corresponding storage unit needs to be updated to ensure that the optimized area meets the minimization requirement of the objective function for evaluating the quality of the storage unit area division.

4. The method for optimizing solid-state hard disk storage according to claim 1, wherein: The step of obtaining the change increment data of the abnormal storage unit includes: Acquire behavioral layer data of abnormal storage units within a pre-set collection period; Calculate the increment and change rate of behavioral layer data within the time window; Based on the increment and change rate of behavior layer data, and statistical analysis of the increment and change rate, the key features in the change increment data are extracted.

5. The method for optimizing solid-state hard disk storage according to claim 1, wherein: The storage optimization of different areas of the solid-state drive using the optimal execution strategy of each area includes: Implement optimal execution strategies in the inner, boundary, and outer areas of the SSD and monitor the performance and energy efficiency of each area; Based on the performance and energy efficiency of each area obtained through real-time monitoring, the optimal execution strategy for each area is dynamically adjusted.

6. A solid-state hard disk storage optimization system, used to implement a solid-state hard disk storage optimization method according to any one of claims 1 to 5, characterized in that: It includes a hard disk partitioning module, an abnormal unit determination and incremental data acquisition module, an optimization strategy generation module and a multi-objective tuning module, and the hard disk partitioning module, the abnormal unit determination and incremental data acquisition module, the optimization strategy generation module and the multi-objective tuning module are connected in sequence; The hard disk partitioning module is used to divide the storage unit into regions based on the behavioral layer data and physical layer data of the solid state disk and using the topological coloring algorithm and the graph cut algorithm to obtain the internal region, the boundary region and the external region; The abnormal unit determination and incremental data acquisition module is used to use the isolation forest algorithm to detect abnormal behavior of the storage units in each area to obtain abnormal storage units; and obtain the incremental data of the abnormal storage units; The optimization strategy generation module is used to generate an optimization strategy set based on a neural network model, taking the characteristic data and the change increment data of the abnormal storage unit as input, so that different areas can receive different optimization treatments; The multi-objective tuning module is used to optimize the storage of the solid-state drive after obtaining the optimal execution strategy through multi-objective tuning based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage unit with the overall optimization goal.

Images