Solid state disk storage optimization method and system

Abstract

The invention discloses a solid state disk storage optimization method and system, and relates to the technical field of data storage.The method comprises the steps that based on behavior layer data and physical layer data of a solid state disk, a topological coloring algorithm and a graph cut algorithm are used for conducting regional division on a storage unit; performing abnormal behavior detection on the storage unit of each area by using an isolated forest algorithm to obtain an abnormal storage unit; acquiring change increment data of the abnormal storage unit; on the basis of a neural network model, taking the feature data and the change increment data of the abnormal storage unit as input, and generating an optimization strategy set, so that different regions are subjected to different optimization processing; and on the basis of the optimization strategy set and under the principle that the optimization strategy of the abnormal storage unit is balanced with the overall optimization target, the storage of the solid state disk is optimized after an optimal execution strategy is obtained through multi-target adjustment and optimization. Different optimization strategies can be applied to the internal area, the boundary area and the external area of the solid state disk.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and more specifically, to a method and system for optimizing solid-state drive (SSD) storage. Background Art

[0002] Methods for optimizing solid-state drive (SSD) storage generally refer to improving the read / write performance, durability, and efficiency of SSDs through intelligent algorithms or hardware architectures. Common SSD optimization methods include: 1. Garbage collection: SSDs use a garbage collection mechanism to clean up data blocks that are no longer needed. Since SSDs are based on flash memory storage, when data is deleted, it is not actually removed from storage immediately, but is marked as recyclable. Garbage collection periodically cleans up this useless data to free up space for writing new data, thereby improving performance.

[0003] 2. TRIM command: TRIM is a command issued by the operating system to notify the SSD which data is no longer in use and can be safely erased. This helps improve the performance of the SSD, especially in scenarios where the storage space is frequently written, as it reduces the accumulation of invalid data.

[0004] 3. Wear leveling: Each storage cell in a flash memory chip has a limited number of write cycles. Wear leveling techniques distribute the write load across all storage cells through intelligent algorithms to avoid excessive wear on certain areas, thereby extending the service life of the SSD.

[0005] 4. Compression and encryption technologies: SSDs utilize compression technologies to reduce the occupancy of storage space or use encryption to ensure data security. Compression technologies can reduce the space occupied by data without affecting performance, thereby improving storage efficiency.

[0006] 5. Caching technology: To improve read / write speeds, many SSDs are equipped with high-speed caches (such as DRAM caches), making read / write operations more rapid. Intelligent algorithms optimize cache management to ensure that frequently accessed data can be read and written quickly.

[0007] However, existing methods for optimizing solid-state drive storage still have the following deficiencies: (1) Uneven performance of storage cells: Storage cells in solid-state drives often exhibit different performance characteristics under different positions and load conditions. Traditional methods do not fully consider these differences, resulting in significant performance fluctuations in storage cells.

[0008] (2)Difficulty in optimizing SSD storage units: The performance optimization of SSD storage units is usually affected by uneven load, reduced lifespan, and data access speed issues. Traditional optimization methods are difficult to balance the different requirements of each storage area and cannot achieve intelligent and dynamic adjustment and optimization.

[0009] (3)Multi-objective optimization challenges: During the SSD storage optimization process, it is usually necessary to balance multiple optimization objectives, such as performance, energy efficiency, load balancing, and durability. These objectives are sometimes conflicting, and a single optimization objective may lead to a decline in performance in other aspects.

[0010] (4)Detection and handling of abnormal storage units: Abnormal storage units in SSDs may cause performance degradation, but traditional storage management methods are difficult to effectively detect and handle these abnormal areas, especially in large-scale storage systems.

[0011] Regarding the problems in the related technologies, no effective solutions have been proposed yet. Summary of the Invention

[0012] Regarding the problems in the related technologies, the present invention proposes a method and system for optimizing SSD storage to overcome the above-mentioned technical problems existing in the existing related technologies.

[0013] To this end, the specific technical solutions adopted by the present invention are as follows: According to one aspect of the present invention, there is provided a method for optimizing SSD storage, including: S1. Based on the behavioral layer data and physical layer data of the SSD, and using the topological coloring algorithm and graph cut algorithm to divide the storage units into regions, obtaining an internal region, a boundary region, and an external region, specifically including: Establish a topological relationship model between storage units according to the behavioral layer data and physical layer data of the SSD; identify the adjacency relationship between storage units according to the topological relationship model between storage units; Divide the storage units into different regions by using the topological coloring function according to the adjacency relationship between storage units; Optimize the storage unit region division result by minimizing the cutting cost of the graph; S2. Use the isolation forest algorithm to detect abnormal behaviors of the storage units in each region, obtaining abnormal storage units; obtain the change increment data of the abnormal storage units; S3. Based on the neural network model, use the feature data and change increment data of the abnormal storage units as inputs to generate an optimization strategy set, so that different regions are subject to different optimization processes; S4. Based on the optimized policy set, and under the principle of balancing the optimized policy of the abnormal storage unit with the overall optimization goal, through multi-objective tuning, after obtaining the optimal execution policy, optimize the storage of the solid-state drive.

[0014] Further, based on the behavioral layer data and physical layer data of the solid-state drive, establish a topological relationship model between storage units; according to the topological relationship model between storage units, identify the adjacency relationship between storage units, including: Extract the physical location information of each storage unit, convert it into spatial coordinates, and at the same time extract the performance boundary data of each storage unit; based on the physical location information and performance boundary data of the storage unit, obtain the topological relationship model between storage units; According to the physical location information of the storage units in the topological relationship model, calculate the adjacency relationship between storage units, where the distance between storage units is determined by the Euclidean distance; Obtain a distance threshold, and when the distance between storage units is less than the distance threshold, the two storage units are adjacent storage units.

[0015] Further, according to the adjacency relationship between storage units, use the topological coloring function to divide the storage units into different regions, including: According to each storage unit and the adjacency relationship of the storage unit, construct a topological graph of the storage unit; Based on the enclosure and connection rules of the storage unit, establish a topological coloring function: ; In the formula, V represents the storage unit, A represents the internal area, B represents the boundary area, and C represents the external area; Use the topological coloring function and combine the performance data and physical location data of the storage unit to preliminarily classify the storage unit, including the internal area, the boundary area and the external area; Based on the preliminary classification results, and using the depth-first search method, start traversing the topological graph of the storage unit from any unlabeled storage unit; During the traversal process, when all adjacent nodes of the node in the topological graph of the storage unit are marked as the same area, mark the node as the same area; When some adjacent nodes of the node in the topological graph of the storage unit belong to different areas, mark the node as the boundary area; For nodes without adjacent nodes or whose adjacent nodes are not marked, the corresponding nodes are marked as the external area.

[0016] Further, by minimizing the cutting cost of the graph, optimize the storage unit area division result, including: Construct an objective function for evaluating the quality of the storage unit area division, and evaluate the quality of the storage unit area division by minimizing the cutting cost between different regions. The objective function is as follows: ; In the formula, Cost (G) represents the objective function for evaluating the quality of the storage unit area division; G represents the topological graph of the storage units including the storage units and their adjacency relationships, E represents the edge set in the topological graph; w ij represents the storage unit i and j the weight of the edge between them; is used to represent whether the storage unit i and j are divided into different regions; Construct an initial flow network according to the topological graph of the storage units, determine the source node and the sink node, and use the maximum flow algorithm to calculate the maximum flow from the source node to the sink node. Based on the maximum flow result, identify the minimum cut set of the graph; Adjust the area division of the storage units according to the result of the minimum cut. If a storage unit belongs to a part of the minimum cut, the area of the corresponding storage unit needs to be updated to ensure that the optimized area meets the minimization requirement of the objective function for evaluating the quality of the storage unit area division.

[0017] Furthermore, use the Isolation Forest algorithm to detect abnormal behaviors of the storage units in each region. The abnormal storage units obtained include: By extracting the behavioral layer data features of the storage units in each region and using them as training data to input into the Isolation Forest algorithm, and the behavioral layer data features include throughput, latency, and load; Train the Isolation Forest model for the storage units in each region respectively, and calculate the anomaly score for each storage unit; configure different anomaly scoring thresholds for different regions; When the anomaly score of a storage unit exceeds the corresponding anomaly scoring threshold, it is marked as an abnormal storage unit; Among them, when training the Isolation Forest model for the internal region, use the behavioral layer data features with a time window at the moment level; when training the Isolation Forest model for the boundary region, use the behavioral layer data features with a time window at the minute level; when training the Isolation Forest model for the external region, preferentially use the storage unit behavior data under high load conditions.

[0018] Furthermore, obtain the change increment data of the abnormal storage units, including: Within a preset acquisition period, obtain the behavioral layer data of the abnormal storage units; Calculate the increment and change rate of the behavior layer data within the time window; Based on the increment and change rate of the behavior layer data, conduct statistical analysis on the increment and change rate, and extract the key features in the change increment data.

[0019] Furthermore, based on the neural network model, use the feature data and change increment data of the abnormal storage unit as inputs to generate an optimization strategy set, so that different regions can be optimized differently, including: Determine the input layer, hidden layer, and output layer of the neural network model, and construct the activation function and loss function; Use the feature data and change increment data of the abnormal storage unit as training data. For each sample of the abnormal storage unit, generate the corresponding optimization strategy label; train and adjust the parameters of the neural network model; The trained neural network model determines the region to which the storage unit belongs based on the input feature data and change increment data, and generates the corresponding optimization strategy.

[0020] Furthermore, based on the optimization strategy set, and under the principle of balancing the optimization strategy of the abnormal storage unit and the overall optimization goal, through multi-objective tuning, after obtaining the optimal execution strategy, optimize the storage of the solid-state drive, including: Preset each optimization goal of the solid-state drive, including performance optimization goal, load balancing goal, energy efficiency goal, and durability goal, and set weights for each optimization goal; By combining the objective functions of each optimization goal, establish a multi-objective optimization model; evaluate the performance of each objective function by simulating the impact of the optimization strategy set on the solid-state drive, and apply the corresponding optimization strategy for different regions; Monitor the execution effect of the optimization strategy for different regions in real time, and collect the performance indicators of the solid-state drive; Evaluate the performance indicators of the optimized solid-state drive; based on the evaluation results, determine the optimal execution strategy for each region; Use the optimal execution strategy for each region to optimize the storage of different regions of the solid-state drive respectively.

[0021] Furthermore, using the optimal execution strategy for each region to optimize the storage of different regions of the solid-state drive respectively, including: Implement the optimal execution strategy in the internal region, boundary region, and external region of the solid-state drive, and monitor the performance and energy efficiency of each region; Dynamically adjust the optimal execution strategy for each region according to the performance and energy efficiency of each region obtained from real-time monitoring.

[0022] According to another aspect of the present invention, there is also provided a solid-state drive storage optimization system, including a hard disk partitioning module, an abnormal unit determination and incremental data acquisition module, an optimization strategy generation module, and a multi-objective tuning module, and the hard disk partitioning module, the abnormal unit determination and incremental data acquisition module, the optimization strategy generation module, and the multi-objective tuning module are sequentially connected.

[0023] The hard disk partitioning module is used to perform regional division on storage units based on the behavioral layer data and physical layer data of the solid-state drive, and use the topological coloring algorithm and the graph cut algorithm to obtain an internal region, a boundary region, and an external region; the abnormal unit determination and incremental data acquisition module is used to use the isolation forest algorithm to detect abnormal behaviors of the storage units in each region to obtain abnormal storage units; obtain the change incremental data of the abnormal storage units; the optimization strategy generation module is used to, based on a neural network model, use the feature data and change incremental data of the abnormal storage units as inputs to generate an optimization strategy set, so that different regions are subject to different optimization processes; the multi-objective tuning module is used to, based on the optimization strategy set, and under the principle of balancing the optimization strategy of the abnormal storage units and the overall optimization goal, through multi-objective tuning, obtain the optimal execution strategy and then optimize the storage of the solid-state drive.

[0024] The beneficial effects of the present invention are as follows: (1) Through the comprehensive analysis of the behavioral layer and physical layer data, and using the topological coloring algorithm and the graph cut algorithm, the present invention can perform precise and intelligent performance division on the storage units of the solid-state drive, enabling storage units with similar performance characteristics to be divided into the same region, avoiding the performance fluctuation problems that may be caused by traditional simple division methods. After the regional division, it is possible to reasonably allocate storage resources according to the specific performance and physical location of the storage units, avoiding performance bottlenecks and resource waste caused by performance imbalance. In addition, the regional division of the solid-state drive helps to give priority to using storage units with stable performance for core data storage, thereby extending the lifespan of the storage units. Especially in the boundary region and the external region, the storage units may be more likely to experience performance degradation, and reasonable division of these regions can effectively share the load and avoid overloading of individual storage units.

[0025] (2) By applying different optimization strategies in the internal region, boundary region, and external region of the solid-state drive, the overall performance of the solid-state drive can be improved, including higher throughput, lower latency, and more balanced load distribution. Especially in high-load situations, by optimizing the write order, data migration, and load balancing, the response speed and stability of the system can be significantly improved.

[0026] (3) Under the principle of balancing the optimization strategy of abnormal storage units and the overall optimization goal, through multi-objective tuning, the performance optimization and durability optimization of the solid-state drive are effectively balanced, avoiding the problems of performance degradation or lifespan shortening that may be caused by single-objective optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0028] Figure 1 is a flowchart of a solid-state drive storage optimization method according to an embodiment of the present invention; Figure 2 is a schematic block diagram of a solid-state drive storage optimization system according to an embodiment of the present invention.

[0029] In the figure: 1. Hard disk partition module; 2. Abnormal unit determination and incremental data acquisition module; 3. Optimization strategy generation module; 4. Multi-objective tuning module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used to explain the operating principle of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these contents, those of ordinary skill in the art should be able to understand other possible implementation manners and the advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are usually used to represent similar components.

[0031] According to an embodiment of the present invention, a solid-state drive storage optimization method and system are provided.

[0032] Now, the present invention will be further described in conjunction with the drawings and specific implementation manners. As Figure 1 shown, according to an embodiment of the present invention, a solid-state drive storage optimization method is provided, including: S1. Based on the behavioral layer data and physical layer data of the solid-state drive, and using the topological coloring algorithm and graph cut algorithm to divide the storage units into internal regions, boundary regions, and external regions.

[0033] Dividing the storage units based on the behavioral layer data and physical layer data of the solid-state drive and using the topological coloring algorithm and graph cut algorithm includes: Establish a topological relationship model between storage units based on the behavioral layer data and physical layer data of the solid-state drive; identify the adjacency relationship between storage units according to the topological relationship model between storage units.

[0034] Divide the storage units into different regions by using the topological coloring function according to the adjacency relationship between storage units.

[0035] Optimize the storage unit area division result by minimizing the graph cut cost; mark the area type for each storage unit, including internal area, boundary area and external area.

[0036] It should be noted that by using the topological coloring algorithm and graph cut algorithm to perform area division on storage units based on the behavioral layer data and physical layer data of the solid-state drive, the layout of storage units can be effectively identified and optimized. By constructing a topological relationship model and analyzing the adjacency relationship between storage units, the storage area can be accurately divided, and the load balance and performance stability of storage units can be improved. Minimizing the graph cut cost further optimizes the area division.

[0037] In one embodiment, establishing a topological relationship model between storage units according to the behavioral layer data and physical layer data of the solid-state drive; identifying the adjacency relationship between storage units according to the topological relationship model between storage units includes: Extract the physical location information of each storage unit and convert it into spatial coordinates, and at the same time extract the performance boundary data of each storage unit; based on the physical location information and performance boundary data of the storage unit, obtain the topological relationship model between storage units.

[0038] Calculate the adjacency relationship between storage units according to the physical location information of storage units in the topological relationship model, wherein the distance between storage units is determined by the Euclidean distance.

[0039] Obtain a distance threshold, and when the distance between storage units is less than the distance threshold, the two storage units are adjacent storage units.

[0040] In one embodiment, dividing the storage units into different regions by using the topological coloring function according to the adjacency relationship between storage units includes: Construct a topological graph (i.e., a weighted undirected graph) of storage units according to each storage unit and the adjacency relationship of the storage unit.

[0041] Based on the enclosure and connection rules of storage units, establish a topological coloring function: ; In the formula, V represents a storage unit, A represents the internal area, B represents the boundary area, and C represents the external area.

[0042] Using a topological coloring function and combining the performance data and physical location data of storage units, the storage units are preliminarily classified into an internal area, a boundary area, and an external area.

[0043] Based on the preliminary classification results and using the depth-first search method, traverse the topological graph of storage units starting from any unlabeled storage unit.

[0044] During the traversal, when all adjacent nodes of a node in the topological graph of storage units are labeled as the same area, then label the node as the same area.

[0045] When some adjacent nodes of a node in the topological graph of storage units belong to different areas, then label the node as the boundary area.

[0046] For a node that has no adjacent nodes or all adjacent nodes are unlabeled, then label the corresponding node as the external area.

[0047] Among them, the topological coloring function is based on the following principles: Internal area (A): The area completely surrounded by other storage units.

[0048] Boundary area (B): The area partially surrounded by other storage units but having a direct connection to the outside.

[0049] External area (C): The area not connected to other storage units or having only an indirect connection.

[0050] In one embodiment, by minimizing the cut cost of the graph, optimizing the storage unit area division result includes: Construct an objective function for evaluating the storage unit area division quality, and to minimize the cut cost between different areas, the objective function for evaluating the storage unit area division quality is: ; In the formula, Cost (G) represents the objective function for evaluating the storage unit area division quality, representing the cost when dividing the storage unit area. The goal is to minimize this cost to ensure that storage units within the same area have similar performance characteristics. That is, the more reasonable the division, the smaller the cost; G represents the topological graph of storage units including storage units and their adjacency relationships, E represents the set of edges in the topological graph; w ij represents a storage unit i and j the weight of the edge between them, usually calculated based on their performance differences (such as I / O throughput, latency, etc.). The larger the weight, the more similar the performance of the storage units and the closer the adjacency relationship; Indicates an indicator function used to represent storage units i and j whether they are partitioned into different regions. If the storage units i and j are partitioned into different regions, the value is 1; if the storage units i and j are partitioned into the same region, the value is 0.

[0051] Construct an initial flow network based on the topology graph of storage units, determine the source node and the sink node, and use the maximum flow algorithm to calculate the maximum flow from the source node to the sink node. Based on the maximum flow result, identify the minimum cut set of the graph.

[0052] Adjust the region partition of storage units according to the result of the minimum cut. If a storage unit belongs to a part of the minimum cut, the region of the corresponding storage unit needs to be updated to ensure that the optimized region meets the minimization requirement of the objective function for evaluating the quality of the storage unit region partition; pay special attention to those storage units located in the boundary region because these units may be adjacent to multiple different types of regions at the same time.

[0053] Specifically, in solid-state drive storage optimization, the topological coloring algorithm divides these storage units into different regions (such as internal regions, boundary regions, and external regions) by identifying the adjacency relationships between storage units. This division helps to optimize the performance, load balancing, and lifespan extension of the storage system. The topological coloring function is used to assign the nodes in the graph to different regions or classes according to their topological relationships. In the solid-state drive storage optimization method, the topological coloring function divides the storage units into internal regions (A), boundary regions (B), and external regions (C) according to the adjacency relationships between each storage unit and other storage units.

[0054] The graph cut algorithm is a technique for dividing a graph into multiple regions by cutting the edges in the graph. In this application, the graph cut algorithm is used to optimize the region partition of storage units and minimize the cut cost between regions. The goal is to make the storage units within the same region have similar performance characteristics and avoid the problem of unbalanced performance. The minimum cut cost refers to the cost incurred when dividing the regions during the graph segmentation process. This cost is related to factors such as the adjacency relationships between storage units and performance differences. By minimizing the cut cost, ensure that the storage unit partition is more reasonable and optimize the allocation of storage resources. The maximum flow algorithm is an algorithm used to calculate the maximum flow between the source node and the sink node in a network. The maximum flow algorithm is used to calculate the minimum cut of a graph. Through the result of the maximum flow, the minimum cut set of the graph can be identified, thereby determining the storage unit regions that need to be adjusted. The purpose of this step is to optimize the storage unit region partition so that the performance characteristics of each region are as consistent as possible.

[0055] When using the maximum flow minimum cut algorithm, determine the source node and the sink node, which represent different regional categories to be distinguished. Use the maximum flow algorithm (such as the Ford-Fulkerson algorithm or Boykov's new algorithm) to calculate the maximum flow from the source node to the sink node, and find a cut set of the graph such that the net flow through this cut set is minimized. Optimize the regional division by minimizing the graph cutting cost. The specific steps are as follows: Initialize the flow network: Construct an initial flow network according to the weighted undirected graph G.

[0056] Calculate the maximum flow: Use the maximum flow algorithm to calculate the maximum flow value.

[0057] Identify the minimum cut: Based on the maximum flow result, identify the minimum cut set of the graph.

[0058] S2. Use the isolation forest algorithm to detect abnormal behaviors of the storage units in each region to obtain abnormal storage units; obtain the change increment data of the abnormal storage units.

[0059] In one embodiment, using the isolation forest algorithm to detect abnormal behaviors of the storage units in each region to obtain abnormal storage units includes: Extract the behavioral layer data features of the storage units in each region as training data and input them into the isolation forest algorithm, and the behavioral layer data features include throughput, latency, and load.

[0060] Train the isolation forest model for the storage units in each region respectively, and calculate the anomaly score for each storage unit; configure different anomaly scoring thresholds for different regions; for example, set a higher anomaly scoring threshold for the boundary region to capture more fluctuating behaviors.

[0061] When the anomaly score of the storage unit exceeds the corresponding anomaly scoring threshold, it is marked as an abnormal storage unit.

[0062] Among them, when training the isolation forest model for the internal region, use the behavioral layer data features of the time-level time window; when training the isolation forest model for the boundary region, use the behavioral layer data features of the minute-level time window; when training the isolation forest model for the external region, preferentially use the storage unit behavior data under high load.

[0063] In one embodiment, obtaining the change increment data of the abnormal storage units includes: Obtain the behavioral layer data of the abnormal storage units within a preset acquisition period.

[0064] Calculate the increment and change rate of the behavioral layer data within the time window.

[0065] According to the increment and change rate of the behavior layer data, and statistically analyze the increment and change rate, extract the key features in the changed increment data, including the maximum change value, the minimum change value, the average change amount, and the standard deviation.

[0066] For the above-mentioned Isolation Forest, it is a tree-based anomaly detection algorithm that isolates data samples by randomly selecting features and split points. Outliers require fewer splitting steps during random splitting, so their paths in the Isolation Forest algorithm are shorter.

[0067] Behavior layer data: refers to various types of indicator data generated by storage units during actual operation, such as: Throughput: refers to the read / write speed or data transfer rate per unit time.

[0068] Latency: refers to the response time of a request or operation.

[0069] Load: refers to the resource consumption of storage units under workload (such as CPU, memory, I / O load, etc.).

[0070] These feature data are used as input data in the Isolation Forest algorithm to help the model learn the normal behavior patterns of storage units and detect abnormal behaviors.

[0071] Boundary regions are usually at the edge of large load fluctuations and data migration. Therefore, the detection of abnormal behaviors requires a high level of sensitivity to capture these fluctuations. Recommended threshold range: 0.7 to 0.9 (based on the anomaly score of the Isolation Forest algorithm). Only when the anomaly score is significantly higher than the normal behavior pattern will it be marked as an anomaly. The boundary regions have large fluctuations, and a higher threshold can avoid misjudging normal fluctuations as anomalies.

[0072] The load in the internal region is usually relatively stable and changes less. Therefore, the anomaly score threshold does not need to be set too high. Recommended threshold range: 0.4 to 0.6. Too low a threshold may cause normal fluctuations to be misjudged as anomalies, while too high a threshold may cause real anomalies to be ignored. Therefore, a medium threshold can help balance sensitivity and false alarm rate.

[0073] External regions have large fluctuations due to high loads or external factors, and the sensitivity of anomaly scores needs to be higher. Recommended threshold range: 0.5 to 0.7, but it is preferred to use the behavior data under high load conditions to train the model. External regions may experience large load fluctuations, so a lower threshold can promptly capture potential abnormal behaviors and avoid fluctuations caused by high loads not being detected in a timely manner.

[0074] Basis for adjusting the threshold selection: Historical data: By analyzing the behavior patterns in different regions of historical data, preliminary threshold ranges can be set. Based on historical data such as throughput, latency, and load, evaluate which data changes are within the normal range and which are abnormal.

[0075] Model training: Train the data in different regions through the Isolation Forest model to obtain a more accurate anomaly scoring range. During this process, dynamically adjust the threshold to ensure that the model can accurately capture abnormal behaviors without generating too many false alarms.

[0076] S3. Based on the neural network model, use the feature data and change increment data of the abnormal storage units as inputs to generate an optimization policy set, so that different regions can be subject to different optimization processes.

[0077] In one embodiment, based on the neural network model, using the feature data and change increment data of the abnormal storage units as inputs to generate an optimization policy set, so that different regions can be subject to different optimization processes includes: Determine the input layer, hidden layer, and output layer of the neural network model, and construct the activation function and loss function.

[0078] Use the feature data and change increment data of the abnormal storage units as training data. For each sample of the abnormal storage unit, generate a corresponding optimization policy label; train and adjust the parameters of the neural network model.

[0079] The trained neural network model determines the region (internal region, boundary region, external region, etc.) to which the storage unit belongs through the input feature data and change increment data, and generates corresponding optimization policies.

[0080] In the neural network model, the input layer processes the feature data and change increment data of the abnormal storage units. The feature data may include the performance metrics of the storage unit (such as throughput, latency, load, etc.), while the change increment data reflects the change situation of the metrics within a certain time window (such as the maximum change value, minimum change value, standard deviation, etc.). These data are passed to the neural network as input features. The hidden layer is used to perform non-linear transformations on the input data to help the model learn complex patterns. During training, the neural network continuously adjusts the parameters (such as weights) to minimize the loss function. The task of the output layer is to generate an optimization policy set. According to the training objective of the model, the output layer will generate a set of policies (such as load balancing, performance optimization, fault repair, etc.), and generate corresponding optimization policies according to the region and status of each storage unit.

[0081] S4. Based on the optimization policy set, and under the principle of balancing the optimization policies of the abnormal storage units and the overall optimization goal, through multi-objective tuning, after obtaining the optimal execution policy, optimize the storage of the solid-state drive.

[0082] In one embodiment, based on the optimization policy set and under the principle of balancing the optimization policies of the abnormal storage unit with the overall optimization goal, after obtaining the optimal execution policy through multi-objective tuning, the optimization of the storage of the solid-state drive includes: Preset various optimization goals for the solid-state drive, including performance optimization goals, load balancing goals, energy efficiency goals, and durability goals, and set weights for each optimization goal.

[0083] By combining the objective functions of each optimization goal, a multi-objective optimization model is established; by simulating the impact of the optimization policy set on the solid-state drive, the performance of each objective function is evaluated, and corresponding optimization policies are applied to different regions.

[0084] Monitor the execution effects of the optimization policies for different regions in real time and collect the performance metrics of the solid-state drive.

[0085] Evaluate the performance metrics of the optimized solid-state drive; according to the evaluation results, determine the optimal execution policy for each region.

[0086] Use the optimal execution policies for each region to optimize the storage of different regions of the solid-state drive respectively.

[0087] In one embodiment, using the optimal execution policies for each region to optimize the storage of different regions of the solid-state drive respectively includes: Implement the optimal execution policy in the internal region, boundary region, and external region of the solid-state drive, and monitor the performance and energy efficiency of each region.

[0088] According to the performance and energy efficiency of each region obtained from real-time monitoring, dynamically adjust the optimal execution policy for each region.

[0089] Among them, the multi-objective optimization process: Simulate the impact of the optimization policy set: The optimization policy set is generated by models such as neural networks, and these policies propose optimization measures for specific problems in each region. During the simulation process, different optimization policies are applied to different regions, and the impact of these policies on the overall performance of the solid-state drive is evaluated. During the evaluation process, the objective function can be used to quantify the effect of each policy.

[0090] Monitoring and evaluation: Real-time monitoring is an essential part of the optimization process. By monitoring the performance metrics of the solid-state drive (such as throughput, latency, load, energy efficiency, etc.), the execution effects of each optimization policy can be continuously evaluated, and the policies can be dynamically adjusted according to real-time data.

[0091] For example, when it is monitored that the performance improvement of a certain area (such as the internal area) is slow, the optimization strategy will be adjusted to increase the attention to performance; if it is found that the energy efficiency of a certain area does not meet the expectation, the measures for energy efficiency optimization will be enhanced.

[0092] By combining multi-objective optimization models (such as particle swarm optimization, genetic algorithms, etc.), real-time monitoring, and dynamic adjustment, the most suitable optimization strategy can be executed for the storage units in different areas. The optimization strategy for each area is tailored according to its characteristics (such as load fluctuation, performance requirements, energy efficiency requirements, etc.), thereby improving the overall performance, load balancing, energy efficiency, and durability of the solid-state drive.

[0093] As Figure 2 shown, according to another embodiment of the present invention, a solid-state drive storage optimization system is further provided, including a hard disk partitioning module 1, an abnormal unit determination and incremental data acquisition module 2, an optimization strategy generation module 3, and a multi-objective tuning module 4, and the hard disk partitioning module 1, the abnormal unit determination and incremental data acquisition module 2, the optimization strategy generation module 3, and the multi-objective tuning module 4 are sequentially connected.

[0094] The hard disk partitioning module 1 is used to perform area partitioning on the storage units based on the behavioral layer data and physical layer data of the solid-state drive, and use the topological coloring algorithm and graph cut algorithm to obtain the internal area, boundary area, and external area; the abnormal unit determination and incremental data acquisition module 2 is used to detect the abnormal behavior of the storage units in each area by using the isolation forest algorithm to obtain the abnormal storage units; obtain the change incremental data of the abnormal storage units; the optimization strategy generation module 3 is used to generate an optimization strategy set by taking the feature data and change incremental data of the abnormal storage units as inputs based on the neural network model, so that different areas are subjected to different optimization processes; the multi-objective tuning module 4 is used to perform multi-objective tuning based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage units with the overall optimization goal, and after obtaining the optimal execution strategy, optimize the storage of the solid-state drive.

[0095] To facilitate the understanding of the above technical solution of the present invention, the working principle of the present invention in the actual process will be described in detail below.

[0096] A certain solid-state drive (SSD) contains 2500 storage units, and each storage unit has different behavioral layer data (such as I / O performance, read / write load, etc.) and physical layer data (such as physical location, lifespan, performance boundary, etc.).

[0097] I. Area partitioning of storage units based on topological coloring algorithm and graph cut algorithm 1. Extract physical location information and performance boundary data Physical location: The location of each storage unit is represented by two-dimensional coordinates. Performance boundary data: including throughput, latency, and load, etc.

[0098] The extracted partial data is shown in Table 1.

[0099] Table 1 Extracted physical location information and performance boundary data -- 2. Construct a topological relationship model and identify adjacency relationships Euclidean distance, for example, the distance between storage unit 1 and storage unit 2 is 7.07. Since 7.07 < 10 (distance threshold), they are adjacent storage units.

[0100] 3. Use the topological coloring function for preliminary classification Construct a topological graph and traverse the topological graph using the depth-first search method (DFS) to label the region types of storage units. Partial results are shown in Table 2.

[0101] Table 2 Region types of storage units 4. Minimize the cut cost of the graph Calculate the minimum cut set through the maximum flow algorithm and adjust the region division results. Partial final division results are shown in Table 3.

[0102] Table 3 Final division results II. Use the Isolation Forest algorithm for anomaly detection 1. Train the Isolation Forest model For each region (internal region, boundary region, external region), extract the behavioral layer data features (throughput, latency, load) as training data.

[0103] 2. Anomaly scoring and threshold setting Calculate the anomaly score for each storage unit and set the anomaly scoring threshold. Partial results are shown in Table 4.

[0104] Table 4 Storage anomaly results III. Generate an optimization strategy set based on a neural network 1. Determine the neural network model structure Construct a neural network model, determine the input layer, hidden layer, and output layer, and set the activation function and loss function. For example, the input layer is the feature data and change increment data of abnormal storage units, and the output layer is the optimization strategy label.

[0105] 2. Train the neural network model Using the characteristic data and change increment data of the abnormal storage unit as training data to generate corresponding optimization strategy labels, the trained neural network model can generate corresponding optimization strategies according to the input data.

[0106] IV. Multi-objective Tuning and Optimal Execution Strategy Preset each optimization objective and its weight of the solid-state drive: Performance optimization objective: weight 0.4.

[0107] Load balancing objective: weight 0.3.

[0108] Energy efficiency objective: weight 0.2.

[0109] Durability objective: weight 0.1.

[0110] By combining the objective functions of each optimization objective, a multi-objective optimization model is established. Simulate the impact of the optimization strategy set on the solid-state drive and evaluate the performance of each objective function. At the same time, for the specific optimization strategies of abnormal storage units, evaluate the impact of these strategies on the overall optimization objectives.

[0111] To ensure that the optimization strategy of the abnormal storage unit does not overly affect the overall optimization objective, a balance needs to be found between the two. Specific methods include: Introduce constraint conditions: Set constraint conditions for each optimization strategy to ensure that its impact on the overall optimization objective is within an acceptable range. Weighted comprehensive scoring: Calculate the impact of each optimization strategy on each optimization objective and conduct a comprehensive scoring in combination with the weights.

[0112] For example, the optimization strategies for a certain area generated based on the neural network model are: decentralized load and reduced latency. The implementation effects of the strategies are as follows: Throughput improvement: 5%. Load change: -10%. Latency change: -10%. Energy consumption change: 0%. Life change: 0%.

[0113] Calculate the comprehensive score according to the weights of each optimization objective: ... Since the comprehensive score is negative, it indicates that this strategy has a negative impact on the overall optimization objective. Therefore, this strategy needs to be adjusted until the best balance point is reached.

[0114] In summary, the present invention includes: (1)By constructing a topological relationship model between storage units and using a topological coloring algorithm to divide the storage units into internal regions, boundary regions, and external regions, the problem of unbalanced performance of storage units is initially solved. The storage units are partitioned according to their positions and performance characteristics, so that the storage units within the same region have similar performance characteristics. The region division is further optimized by minimizing the graph cut cost to ensure that the storage units within the same region have as similar performance characteristics as possible, thereby reducing performance fluctuations.

[0115] (2)By presetting multiple optimization goals for the solid-state drive (such as performance, energy efficiency, load balancing, and durability) and setting weights for each goal, a multi-objective optimization model is established. By simulating the impact of the optimization strategy set on the solid-state drive, the performance of each objective function is evaluated, and corresponding optimization strategies are applied to different regions.

[0116] (3)The Isolation Forest algorithm is used to detect abnormal behaviors of the storage units in each region, identify the abnormal storage units, and obtain their change increment data. Based on the neural network model, the feature data and change increment data of the abnormal storage units are used as inputs to generate specific optimization strategies to specifically handle the abnormal storage units.

[0117] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for optimizing solid-state drive storage, characterized in that, Including: S1. Based on the behavioral layer data and physical layer data of the solid-state drive, and using the topological coloring algorithm and graph cut algorithm to divide the storage units into regions, obtaining an internal region, a boundary region, and an external region, specifically including: Establishing a topological relationship model between storage units according to the behavioral layer data and physical layer data of the solid-state drive; identifying the adjacency relationship between storage units according to the topological relationship model between storage units; Dividing the storage units into different regions by using the topological coloring function according to the adjacency relationship between storage units; Optimizing the storage unit region division result by minimizing the cutting cost of the graph; S2. Using the isolation forest algorithm to detect abnormal behaviors of the storage units in each region, obtaining abnormal storage units; acquiring the change increment data of the abnormal storage units; S3. Based on the neural network model, taking the feature data and change increment data of the abnormal storage units as inputs, generating an optimization strategy set to enable different regions to obtain different optimization processes; S4. Based on the optimization strategy set, and under the principle of balancing the optimization strategy of the abnormal storage units and the overall optimization goal, through multi-objective tuning, obtaining the optimal execution strategy and then optimizing the storage of the solid-state drive.

2. The method for optimizing solid-state drive storage according to claim 1, characterized in that, The establishing a topological relationship model between storage units according to the behavioral layer data and physical layer data of the solid-state drive; The identifying the adjacency relationship between storage units according to the topological relationship model between storage units includes: Extracting the physical location information of each storage unit and converting it into spatial coordinates, and at the same time extracting the performance boundary data of each storage unit; obtaining the topological relationship model between storage units based on the physical location information and performance boundary data of the storage units; Calculating the adjacency relationship between storage units according to the physical location information of the storage units in the topological relationship model, where the distance between storage units is determined by the Euclidean distance; Obtaining a distance threshold, and when the distance between storage units is less than the distance threshold, the two storage units are adjacent storage units.

3. The method for optimizing solid-state drive storage according to claim 1, characterized in that, The dividing the storage units into different regions by using the topological coloring function according to the adjacency relationship between storage units includes: Constructing a topological graph of the storage units according to each storage unit and the adjacency relationship of the storage units; Establishing a topological coloring function based on the enclosure and connection rules of the storage units; ； In the formula, V represents a storage unit, A represents an internal area, B represents a boundary area, and C represents an external area; Using the topological coloring function and combining the performance data and physical location data of the storage units to preliminarily classify the storage units, including an internal region, a boundary region, and an external region; Based on the preliminary classification result, and using the depth-first search method, starting from any unlabeled storage unit to traverse the topological graph of the storage units; During the traversal process, when all adjacent nodes of the node in the topological graph of the storage unit are marked as the same region, the node is marked as the same region; When some adjacent nodes of the node in the topological graph of the storage unit belong to different regions, the node is marked as the boundary region; For nodes without adjacent nodes or whose adjacent nodes are not marked, the corresponding nodes are marked as the external region.

4. The method for optimizing solid-state drive storage according to claim 1, characterized in that, The optimizing the storage unit region division result by minimizing the cutting cost of the graph includes: Construct an objective function for evaluating the quality of the storage unit area partition, and evaluate the quality of the storage unit area partition by minimizing the cutting cost between different areas. The objective function is as follows: ； In the formula, Cost (G) represents the objective function for evaluating the quality of the storage unit area division; G represents a topological graph of memory cells including memory cells and adjacency relationships, E representing the set of edges in the topological graph; w ij Indicates the storage unit i and j the weight of the edge between; for representing a storage cell i and j whether it is divided into different regions; Construct an initial flow network according to the topology graph of the storage unit, determine the source node and the sink node, and use the maximum flow algorithm to calculate the maximum flow from the source node to the sink node. Based on the maximum flow result, identify the minimum cut set of the graph; Adjust the area partition of the storage unit according to the result of the minimum cut. If a storage unit belongs to a part of the minimum cut, the area of the corresponding storage unit needs to be updated to ensure that the optimized area meets the minimization requirement of the objective function for evaluating the quality of the storage unit area partition.

5. The method for optimizing solid-state drive storage according to claim 1, characterized in that, The abnormal behavior detection of the storage units in each area by using the isolation forest algorithm to obtain abnormal storage units includes: Extract the behavioral layer data features of the storage units in each area as training data input to the isolation forest algorithm, and the behavioral layer data features include throughput, latency, and load; Train the isolation forest model for the storage units in each area respectively, and calculate the abnormal score for each storage unit; configure different abnormal scoring thresholds for different areas; When the abnormal score of a storage unit exceeds the corresponding abnormal scoring threshold, it is marked as an abnormal storage unit; Among them, when training the isolation forest model for the internal area, use the behavioral layer data features of the time-level time window; when training the isolation forest model for the boundary area, use the behavioral layer data features of the minute-level time window; when training the isolation forest model for the external area, give priority to using the storage unit behavior data under high load.

6. A method for optimizing solid - state drive storage according to claim 1, characterized in that, The acquisition of the change increment data of the abnormal storage units includes: Obtain the behavioral layer data of the abnormal storage units within a preset acquisition period; Calculate the increment and change rate of the behavioral layer data within the time window; According to the increment and change rate of the behavioral layer data, and conduct statistical analysis on the increment and change rate, extract the key features in the change increment data.

7. A method for optimizing solid - state drive storage according to claim 1, characterized in that, Based on the neural network model, use the feature data and change increment data of the abnormal storage units as inputs to generate an optimization strategy set, so that different areas are subject to different optimization processes, including: Determine the input layer, hidden layer, and output layer of the neural network model, and construct the activation function and loss function; Use the feature data and change increment data of the abnormal storage units as training data. For each sample of the abnormal storage unit, generate the corresponding optimization strategy label; train and adjust the parameters of the neural network model; The trained neural network model determines the area to which the storage unit belongs through the input feature data and change increment data, and generates the corresponding optimization strategy.

8. A method for optimizing solid - state drive storage according to claim 1, characterized in that, Based on the optimization strategy set, and under the principle of balancing the optimization strategy of the abnormal storage unit and the overall optimization goal, through multi-objective tuning, after obtaining the optimal execution strategy, optimize the storage of the solid-state drive, including: Preset each optimization goal of the solid-state drive, including performance optimization goal, load balancing goal, energy efficiency goal, and durability goal, and set weights for each optimization goal; A multi-objective optimization model is established by combining the objective functions of each optimization objective; the performance of each objective function is evaluated by simulating the impact of the optimization strategy set on the solid-state drive, and corresponding optimization strategies are applied to different regions; The execution effects of the optimization strategies for different regions are monitored in real time, and the performance metrics of the solid-state drive are collected; The performance metrics of the optimized solid-state drive are evaluated; based on the evaluation results, the optimal execution strategies for each region are determined; The storage of different regions of the solid-state drive is optimized respectively by using the optimal execution strategies of each region.

9. A method for optimizing solid - state drive storage according to claim 8, characterized in that, The optimizing the storage of different regions of the solid-state drive respectively by using the optimal execution strategies of each region includes: Implement the optimal execution strategy in the internal region, boundary region and external region of the solid-state drive, and monitor the performance and energy efficiency of each region; Dynamically adjust the optimal execution strategy of each region according to the performance and energy efficiency of each region obtained by real-time monitoring.

10. A solid - state drive storage optimization system for implementing the method for optimizing solid - state drive storage according to any one of claims 1 - 9, characterized in that, It includes a hard disk partitioning module, an abnormal unit determination and incremental data acquisition module, an optimization strategy generation module and a multi-objective tuning module, and the hard disk partitioning module, the abnormal unit determination and incremental data acquisition module, the optimization strategy generation module and the multi-objective tuning module are sequentially connected; The hard disk partitioning module is used to divide the storage units into internal regions, boundary regions and external regions based on the behavioral layer data and physical layer data of the solid-state drive, and by using the topological coloring algorithm and the graph cut algorithm; The abnormal unit determination and incremental data acquisition module is used to detect the abnormal behavior of the storage units in each region by using the isolation forest algorithm to obtain the abnormal storage units; obtain the change incremental data of the abnormal storage units; The optimization strategy generation module is used to generate an optimization strategy set by taking the feature data and change incremental data of the abnormal storage units as inputs based on a neural network model, so that different regions are subject to different optimization processes; The multi-objective tuning module is used to optimize the storage of the solid-state drive after obtaining the optimal execution strategy through multi-objective tuning based on the optimization strategy set and under the principle of balancing the optimization strategy of the abnormal storage units and the overall optimization objective.

Images