Intelligent capacity expansion method and system for enterprise-level solid state disk

Abstract

The invention relates to the technical field of intelligent capacity expansion, in particular to an intelligent capacity expansion method and system for an enterprise-level solid state disk, and the method comprises the following steps: an I / O controller detects a newly accessed enterprise-level solid state disk and sets a total write-in amount threshold value; according to the method, the total write-in quantity threshold is set after the enterprise-level solid state disk is accessed, and the dynamic delay portrait is constructed in combination with the real-time delay data, so that the performance state of the target hard disk is precisely mastered in the capacity expansion process, and the logic data block and the check code are synchronously embedded in the data write-in stage; and an erasure code check block is synchronously generated in combination with a write-in voucher and written into a far-end address, so that a main and auxiliary path parallel write-in mechanism is realized. And comparing data consistency by reading main and auxiliary addresses, constructing an inconsistent item set, obtaining a corresponding physical region error log by relying on an NVMe interface, extracting an abnormal sector, and mapping to form an unstable physical region coordinate, so that the system has the capability of actively avoiding an unstable region.

Description

Technical Field

[0001] The present invention relates to the field of intelligent capacity expansion technology, and in particular to an intelligent capacity expansion method and system for enterprise-level solid-state drives. Background Art

[0002] The field of intelligent capacity expansion technology involves the dynamic identification, capacity assessment, and automated expansion management of hardware resources in computing storage systems. The goal is to enable the storage system to automatically complete storage media mounting, capacity allocation, and write scheduling optimization based on current load status, capacity utilization, response performance, and other dimensions without interrupting business operations.

[0003] Existing technologies mainly rely on single-point detection and static rule judgment of storage device status, and lack a resource identification mechanism based on parallel comparison of multiple parameters, which can easily cause performance fluctuations of newly connected storage media to not be discovered in time. For example, during high-frequency writing, if the target hard disk delay increases or local abnormal sectors appear, it is difficult for the system to predict hardware unstable areas during the expansion phase, which can easily lead to write failures or redundant copy overwriting anomalies. Failure to perform differentiated scheduling for requests of different priorities can easily lead to low-priority tasks preempting high-performance channel resources, increasing the write pressure on the already heavily loaded hard disks, and lowering the overall system response efficiency and redundant data consistency level. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an intelligent expansion method and system for enterprise-level solid-state hard drives.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for intelligent capacity expansion of an enterprise-class solid-state drive, comprising the following steps: The I / O controller detects the newly connected enterprise-class SSD and sets a total write threshold. The I / O controller continuously obtains the real-time latency value of the enterprise-class SSD and creates a dynamic latency profile for the new drive. Based on the dynamic latency profile of the new disk, when the I / O controller receives a write request, it writes the logical data block and the check code to the target address, obtains a write certificate for the target address, and based on the write certificate for the target address, generates an erasure code check block for the logical data block and writes it to the remote address, thereby establishing a redundant data write pair. Based on the redundant data write pair, read the data of the target address and the remote address and perform a consistency check to obtain a read-back check result; based on the data inconsistency items in the read-back check result, extract the physical area information in the error log through the NVMe interface to establish the unstable physical area coordinates; The host bus adapter parses the received I / O request, extracts the application priority, generates a request priority identifier, calculates the matching degree with the dynamic delay profile of the new disk based on the request priority identifier, and avoids the unstable physical area coordinates to generate a target drive letter routing decision.

[0006] Preferably, the steps for obtaining the dynamic delay profile of the new disk are: The I / O controller monitors the physical connection port of the SSD, confirms that the voltage and current signals on the port are stable, identifies the device identifier and feature code sent by the port, parses the data, obtains confirmation that the connected device is an enterprise-class SSD, and generates an enterprise-class SSD access identifier; Based on the enterprise-class solid-state drive access identifier, the I / O controller calls the manufacturer's firmware interface to read the maximum lifespan parameter preset by the solid-state drive, parses the theoretical write volume value supported by the maximum lifespan parameter, sets the total write volume threshold allowed by the solid-state drive, and generates a total write volume threshold; Based on the total write volume threshold, the I / O controller continuously calls the internal latency monitoring register of the enterprise-level solid-state drive through the NVMe interface in real time, reads the latency response time of each operation, arranges the latency response time in time series and stores it to generate a new dynamic latency profile of the disk.

[0007] Preferably, the steps for obtaining the target address writing credential are: After receiving the write request, the I / O controller parses the logical data block in the write request, calculates the check code corresponding to the logical data block, completes data splicing of the logical data block and the check code, and generates a logical data block and check code splicing unit; Based on the logic data block and the check code splicing unit, calling the new disk dynamic delay profile, matching the real-time delay values ​​corresponding to the logic data block and the check code splicing unit, screening the available target address with the lowest real-time delay value, and forming a screening result of the available target addresses; Based on the screening result of the available target addresses, the logic data block and the check code splicing unit are written into the target address through the PCIe bus protocol. After the target address is written, the unique certificate identifier of the writing is obtained to form a target address writing certificate.

[0008] Preferably, the steps of acquiring the redundant data write pair are: Based on the target address write voucher, the I / O controller extracts the logical data block information recorded in the target address write voucher, and performs byte-by-byte redundancy check calculation on the logical data block information according to the erasure code operation rules to obtain an erasure code check block; Parsing the network interface address of the remote storage device according to the erasure code check block, and verifying the address validity and connection status of the remote storage device to obtain an available remote address; Based on the available remote address, the I / O controller sends the erasure code check block to the remote storage device through the data transmission interface, and after the remote storage device confirms the write, establishes a one-to-one mapping relationship between the logical data block information and the erasure code check block to form a redundant data write pair.

[0009] Preferably, the steps of obtaining the read-back verification result are: Call the target address and remote address recorded in the redundant data write pair, read the bit-level original values ​​in the corresponding storage units from the NVMe channel respectively, and map the target address data block and the remote address data block one-to-one according to the bit arrangement order to form a matching set of the target address bit sequence and the remote address bit sequence; Calculating a weighted burst error sensitivity index based on a matching set of the target address bit sequence and the remote address bit sequence; According to the weighted burst error sensitivity index, it is determined whether it is zero or below the tolerance threshold. If the condition is met, the data consistency is determined to meet the requirements. If not, it is marked that there is a risk of continuous structural error and a readback verification result is generated.

[0010] Preferably, the steps of obtaining the coordinates of the unstable physical area are: Based on the data inconsistency items in the read-back verification result, the target address bit sequence corresponding to the data inconsistency item is parsed, the logical position of the bit where the data inconsistency occurs in the target address bit sequence is recorded bit by bit, and all the logical positions are deduplicated and sorted to form a set of data inconsistency logical positions; Based on the set of inconsistent logical locations, the NVMe interface is called to query the physical area number and sector address within the physical area corresponding to each logical location from the internal error log of the enterprise-level solid-state drive one by one, and the position coordinates of each logical location mapped to the physical area are recorded to form a physical area information mapping table; Based on the physical area information mapping table, the physical area numbers and corresponding sector addresses where data inconsistency occurs repeatedly are identified and marked as unstable, and the physical coordinate information corresponding to the marked positions is stored in sequence to generate unstable physical area coordinates.

[0011] Preferably, the steps for obtaining the request priority identifier are: The host bus adapter receives the I / O request, decomposes the message header and payload of the I / O request segment by segment, extracts the application identification field and task type field contained therein, and caches and records them in the order of receipt to form an I / O request parsing record; According to the I / O request parsing record, calling a pre-set application priority mapping table, determining the application priority level corresponding to each I / O request by comparing the application identification field and the task type field with the importance levels of various tasks recorded in the mapping table, and forming an application priority level sequence; Based on the application priority level sequence, each I / O request is assigned a request priority code corresponding to the application priority level one by one, and the request priority code is associated and bound with the corresponding I / O request to generate a request priority identifier.

[0012] Preferably, the steps of obtaining the target drive letter routing decision are: After receiving the request priority identifier, the host bus adapter sequentially calls the new disk dynamic latency profile from all enterprise-class solid-state drives, extracts the average response time, I / O queue depth, and idle time ratio within a recent fixed period, and performs maximum and minimum normalization on the original parameters to generate normalized response time, normalized queue depth, and normalized idle time ratio; Calculate the priority-sensitive cost index for each enterprise-class SSD based on normalized response time, normalized queue depth, and normalized idle percentage; Based on the priority-sensitive cost index of all enterprise-class solid-state drives, hard drives that do not contain unstable physical area coordinates are screened out, and the hard drive with the smallest priority-sensitive cost index is selected from the remaining hard drives. The device identifier is converted into a system drive letter to generate a target drive letter routing decision.

[0013] The present invention also provides an intelligent capacity expansion system, comprising: Detection module: The I / O controller detects newly connected enterprise-class SSDs and sets a total write threshold. The I / O controller continuously obtains the real-time latency value of the enterprise-class SSD and creates a dynamic latency profile for the new drive. Write module: Based on the dynamic latency profile of the new disk, when the I / O controller receives a write request, it writes the logical data block and the check code to the target address, obtains the target address write certificate, and based on the target address write certificate, generates an erasure code check block for the logical data block and writes it to the remote address, thus establishing a redundant data write pair. Verification module: Based on the redundant data write pair, read the data of the target address and the remote address and perform consistency verification, obtain the read-back verification result, and based on the data inconsistency items in the read-back verification result, extract the physical area information in the error log through the NVMe interface to establish the unstable physical area coordinates; Scheduling module: The host bus adapter parses the received I / O requests, extracts the application priority, generates a request priority identifier, calculates the matching degree with the dynamic delay profile of the new disk based on the request priority identifier, and avoids the unstable physical area coordinates to generate the target drive letter routing decision.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are: The present invention sets a total write threshold after accessing an enterprise-level solid-state hard drive, and builds a dynamic delay profile in combination with real-time delay data to ensure that the expansion process has a precise grasp of the target hard drive performance status, and then synchronously embeds logical data blocks and check code processing in the data writing stage, and synchronously generates an erasure code check block in combination with the write voucher and writes it to the remote address, thereby realizing a parallel write mechanism for the primary and secondary paths. Then, by reading the primary and secondary addresses to compare the data consistency, a set of inconsistent items is constructed, and the corresponding physical area error log is obtained based on the NVMe interface, abnormal sectors are extracted and mapped to form unstable physical area coordinates, so that the system has the ability to actively avoid unstable areas. In further processing, the application priority content in the I / O request is parsed, and the priority-sensitive cost index is calculated according to the priority characteristics and the delay profile, and an asymmetric adaptation decision-making mechanism is established between the write request and the hard drive performance status. On the basis of avoiding unstable physical areas, the allocation and pointing of the hard drive path is completed preferentially. Data integrity is ensured through a comprehensive parallel mechanism of erasure correction and verification, and adaptive evolution of resource configuration is achieved through real-time profiling and dynamic adaptation parameters. At the same time, the overall fault tolerance and task distribution efficiency are improved, the probability of request redirection and data recovery pressure are reduced, and the expansion process has the ability to provide continuous write guarantee and dynamic scheduling optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0017] See also Figure 1 The present invention provides a technical solution, a method for intelligent expansion of enterprise-level solid-state hard drives, comprising the following steps: The I / O controller detects the newly connected enterprise-class SSD and sets a total write threshold. The I / O controller continuously obtains the real-time latency value of the enterprise-class SSD and creates a dynamic latency profile for the new drive. Based on the dynamic latency profile of the new disk, when the I / O controller receives a write request, it writes the logical data block and the checksum to the target address, obtains the write certificate for the target address, and then generates an erasure code check block for the logical data block based on the write certificate and writes it to the remote address, thus establishing a redundant data write pair. Based on the redundant data write pair, the data at the target address and the remote address are read and consistency checked to obtain the read-back verification result. Based on the data inconsistencies in the read-back verification result, the physical area information in the error log is extracted through the NVMe interface to establish the coordinates of the unstable physical area. The host bus adapter parses the received I / O requests, extracts the application priority, generates a request priority identifier, and calculates the matching degree with the dynamic latency profile of the new disk based on the request priority identifier, while avoiding unstable physical area coordinates and generating the target drive letter routing decision.

[0018] The steps to obtain the dynamic latency profile of a new disk are as follows: The I / O controller monitors the physical connection port of the SSD, confirms that the voltage and current signals on the port are stable, identifies the device identifier and feature code sent by the port, parses the data, obtains confirmation that the connected device is an enterprise-class SSD, and generates an enterprise-class SSD access identifier; Based on the enterprise-level SSD access identifier, the I / O controller calls the manufacturer's firmware interface to read the SSD's preset maximum lifespan parameters, parses the theoretical write volume value supported in the maximum lifespan parameters, sets the SSD's allowed total write volume threshold, and generates a total write volume threshold. Based on the total write volume threshold, the I / O controller continuously calls the internal latency monitoring register of the enterprise-level solid-state drive through the NVMe interface in real time, reads the latency response time of each operation, arranges the latency response time in time series, and stores it to generate a new dynamic latency profile of the drive.

[0019] Specifically, the I / O controller monitors the physical connection port of the solid-state drive, specifically by continuously polling the physical layer status of the NVMe or SATA interface. After detecting the physical connection handshake signal, it starts to monitor the voltage value of the specified power pin. For example, for the NVMeM.2 interface, it monitors the 3.3V power rail to confirm that it is stable within the range of ±5% of the rated voltage, that is, between 3.135V and 3.465V within a specified time, such as 500 milliseconds. At the same time, the current consumption of the drive during the initialization phase is monitored through an integrated current sensing circuit or an external current sensor to ensure that it quickly falls back to a stable standby voltage after a short startup peak. The upper limit of the standby current is set according to the preset enterprise-level solid-state drive standby power consumption specification, for example, not more than 0.8A@3.3V. If the voltage and current are maintained within their respective stability standards in a continuous sampling, for example, 10 times (sampling interval 10 milliseconds), the port signal is determined to be stable. Subsequently, the I / O controller uses the port protocol, for example, for NVMe devices, by sending the IdentifyController command in NVMeAdminCommand, reading the returned IdentifyController data structure, and extracting the device identifier such as the manufacturer I The I / O controller compares the extracted VID, DID and feature code with the preset enterprise-level solid-state drive feature database, which is pre-configured based on the firmware version, the flags of specific enterprise-level features (such as atomic writes and virtualization enhancements) in the supported optional NVM command set, the reported endurance indicators (such as non-zero valid values ​​of TBW or DWPD), and the supported queue depth (for example, the number of submission queue and completion queue entries are both greater than or equal to 1024). The database is compiled from product specifications and industry standards, and contains a list of known enterprise-class SSD VIDs / DIDs and the valid ranges of their key characteristic parameters. For example, a database entry might be {VID: 0x8086, DID_Range: [0xF1A0-0xF1AF], Min_Reported_TBW: 1000TB, Required_ONCS_Flags: 0b00101100}. If the queried information completely matches any enterprise-class SSD record in the database, the connected device is confirmed to be an enterprise-class SSD, and an enterprise-class SSD access identifier is generated.

[0020] After confirming the identity of the newly connected hard drive based on the enterprise-level SSD access identifier, the I / O controller calls the manufacturer's firmware interface to read the maximum life parameters preset for the SSD. This process usually involves sending specific NVMeAdmin commands to the SSD, such as the standard GetLogPage command with SMART / HealthInformationLog (LogIdentifier02h) or a manufacturer-specific LogPage, or executing the manufacturer's customized VendorSpecificCommand (VSC). The opcode and parameters of this command are found from the preset manufacturer instruction mapping table based on the vendor ID (VID) obtained during the identification phase. For example, a manufacturer may specify that VSC0xC1 is used with subcommand 0x01 to return a detailed life parameter structure. From the returned data structure, the I / O controller parses the preset maximum life parameters, which are expressed as a rated value of the total number of bytes written, or the daily drive write volume, warranty years, and drive capacity. If DWPD is obtained, for example, the DWPD is 3, the warranty period is 5 years, and the drive capacity is 3.2TB, the I / O controller calculates its theoretical write volume value. The calculation method is: Theoretical write volume , substitute the numerical value into Then, set the total write volume threshold allowed by the solid-state drive. This threshold is usually set as a percentage of the theoretical write volume value, such as 85%. The setting of (for example, 0.85) is based on the risk tolerance and early warning requirements for hard drive life management in the enterprise operation and maintenance strategy. It aims to provide sufficient buffer time for data migration and hard drive replacement. Its setting refers to the common practice of preventive maintenance of enterprise-level storage devices in the industry, as well as considerations of specific business continuity requirements. For example, for critical business systems that require 7x24 hours high availability, this percentage may be set lower, such as 75%, to increase the sensitivity of early warnings. For non-real-time critical businesses such as disaster recovery and backup, it may be set slightly higher, such as 90%, to maximize the hard drive life. In this example, if 85% is used, the total write volume threshold is calculated as The calculated 14892TB is the total write threshold generated.

[0021] Based on the total write volume threshold, the I / O controller begins to continuously monitor the performance of newly connected enterprise-class SSDs. Specifically, the I / O controller obtains the latency response time of each I / O operation processed by the enterprise-class SSD in real time through the NVMe interface. For each I / O request (such as a read or write command) initiated by the host, the I / O controller records a high-precision timestamp before dispatching the command to the NVMe submission queue of the SSD. (For example, using a system clock with nanosecond precision), when the I / O operation is completed, the corresponding completion queue item is written back to the NVMe completion queue by the solid-state drive, and when the I / O controller processes the CQE, a high-precision timestamp is recorded again. , then the I / O controller calculates the delayed response time of this operation as The delay value is recorded in microseconds (µs). This process is performed on all I / O operations (including read and write operations) directed to the new disk that flow through the I / O controller, and each delay response time obtained is together with the point in time at which it occurred (e.g., using These data pairs are appended in chronological order to a data structure dedicated to the new drive with first-in, first-out (FIFO) characteristics, such as a fixed-size ring buffer or a time series database table configured with a rolling window strategy. The capacity of this storage structure is set based on the system's requirements for the precision and historical traceability of the latency profile. For example, it can be set to store latency records for the most recent 1 million I / O operations, or for all I / O operations in the past 24 hours. For example, if the average IOPS is 5000, storing 1 million records is equivalent to approximately 200 seconds of detailed data. This setting aims to balance memory consumption and the effectiveness of latency pattern analysis, ensuring the continuous accumulation of sufficient data points to reflect the dynamic behavior of the drive under different loads. Ultimately, this time-series-arranged and stored (timestamp, latency) data set constitutes the dynamic latency profile of the newly connected enterprise-class SSD.

[0022] The steps to obtain the target address writing credentials are: After receiving the write request, the I / O controller parses the logical data block in the write request, calculates the check code corresponding to the logical data block, completes data splicing of the logical data block and the check code, and generates a logical data block and check code splicing unit; Based on the logical data block and check code splicing unit, the new disk dynamic delay profile is called, the real-time delay value corresponding to the logical data block and the check code splicing unit is matched, and the available target address with the lowest real-time delay value is selected to form the screening result of the available target address; Based on the screening results of the available target addresses, the logic data block and the check code splicing unit are written to the target address through the PCIe bus protocol. After the target address is written, the unique certificate identifier of the writing is obtained to form the target address writing certificate.

[0023] Specifically, after receiving a write request, the I / O controller first parses the logical data block to be written from the request (for example, a submission queue item of an NVMe I / O command), which includes extracting the target logical block address (LBA), the length of the write data, and the data content itself. Then, for the extracted pure logical data block content, a preset cyclic redundancy check algorithm is used, such as CRC32C (whose generating polynomial is 0x1EDC6F41, which is widely used in the network and storage fields to ensure data integrity), to calculate a 32-bit check code. The calculation process traverses each byte of the logical data block and appends the calculated check code to the end of the logical data block without performing any byte order conversion or encoding, thereby completing the direct data splicing of the logical data block and the check code. For example, if the logical data block size is 4096 bytes, a 4100-byte (4096-byte data + 4-byte CRC32C) unit is formed after splicing, generating a logical data block and check code splicing unit.

[0024] Based on the logical data block and check code splicing unit, the I / O controller calls the previously established dynamic delay profile of the new disk that stores the historical operation records of the newly connected solid-state drive. First, a group of candidate write areas are selected from the list of free physical areas of the new disk maintained internally. For example, 5 consecutive free logical block address ranges with a size equal to the current erase block size are selected as candidate target addresses. For each candidate target address range, the dynamic delay profile of the new disk is retrieved within a recent preset time window (for example, the time window length within the past 5 seconds). The delay response time is extracted based on the historical write operation records of all starting LBAs falling within the candidate target address range within a period of 5 seconds, which is based on empirical observations and can better reflect the delay changes under the current load. If the valid historical write records within a candidate target address range are less than a minimum sample number (for example, less than 3, the minimum sample number (The value is set to 3 to avoid inaccurate latency assessments due to insufficient data). A higher penalty value, such as 10 milliseconds, is assigned to the candidate region's real-time latency to reduce its probability of selection. For candidate regions with sufficient samples, the average latency response time of all valid historical write operations within the time window is calculated. This average value serves as the candidate region's real-time latency. The real-time latency values ​​of all candidate regions are then compared against a "maximum tolerable write latency threshold." This threshold, for example, is set to 800 microseconds. This threshold is determined by taking 1.2 times the 99.9th percentile latency commitment under sustained write load, as per the typical performance specifications of enterprise-class SSDs. If a candidate region's real-time latency exceeds this 800 microsecond threshold, it is removed from the candidate list. Finally, the starting logical block address of the candidate region with the lowest real-time latency value, which has been calculated and has been retained, is selected to form the screening result for available target addresses.

[0025] Based on the target logical block address determined in the screening results of the available target addresses, the I / O controller encapsulates the previously generated logical data block and check code splicing unit into a standard NVMe write command through the PCIe bus protocol. The command specifies the target namespace ID, the starting LBA (that is, the screening results of the available target address), the number of logical blocks (calculated based on the total length of the logical data block and check code splicing unit and the logical block size of the SSD. For example, if the splicing unit is 4100 bytes and the SSD logical block size is 512 bytes, 9 logical blocks need to be written, and the last logical block is partially filled), and a physical region pointer list (PRPList) or hash table (SGL) pointing to the data buffer containing the logical data block and check code splicing unit. The command is sent to the corresponding submission queue of the enterprise-class solid-state drive through the PCIe link. After an enterprise-class SSD performs a write operation, it returns a completion queue item (CQE) to the host's completion queue through the PCIe link. The I / O controller polls or receives this CQE through an interrupt and checks the status field. If the status is success, it means that the target address is written successfully. At this time, the I / O controller obtains and records a unique write credential identifier containing multiple elements. This identifier specifically includes: the target starting logical block address of the successful write, the internal transaction ID of this write operation (a unique serial number generated by the I / O controller for each I / O operation, such as a 64-bit incrementing integer), the precise timestamp when the write is completed (nanosecond level), the original CRC32C checksum of the actual written logical data block (excluding the checksum part), and the length (bytes) of the written data block. These elements are combined to form the target address write credential.

[0026] The steps for obtaining redundant data write pairs are: Based on the target address write voucher, the I / O controller extracts the logical data block information recorded in the target address write voucher, performs byte-by-byte redundancy check calculation on the logical data block information according to the erasure code operation rules, and obtains the erasure code check block; Parse the network interface address of the remote storage device based on the erasure code check block, verify the address validity and connection status of the remote storage device, and obtain an available remote address; Based on the available remote address, the I / O controller sends the erasure code check block to the remote storage device through the data transmission interface. After the remote storage device confirms the write, it establishes a one-to-one mapping relationship between the logical data block information and the erasure code check block to form a redundant data write pair.

[0027] Specifically, based on the target address write voucher, the I / O controller first extracts the content of the "actual written logical data block" and the "length of the written data block" from the voucher. The logical data block information here is the original user data that does not contain the CRC check code when it was previously written to the SSD. For example, if the original write request data is 4096 bytes, the 4096 bytes of data are extracted. Subsequently, the I / O controller uses the preset erasure code operation rules, such as the Reed-Solomon (k, m) encoding scheme, where Represents the number of data blocks, Represents the number of check blocks. For example, if the system is configured as Reed-Solomon (4, 2), it means that 2 check blocks will be generated for every 4 data blocks, which can tolerate the loss or damage of any 2 blocks. For the extracted logical data block information, if its size is not enough to directly form an erasure code unit (i.e. The size of the data block is first filled or aggregated with other logical data blocks until the size requirement of the coding unit is met. Then, the information of this (or these) logical data blocks is regarded as Data symbols of equal size (usually in bytes) are used to perform redundancy check calculations byte by byte through polynomial operations on the Galois field. The specific calculation process involves multiplication of the generator matrix and the data vector, thereby obtaining For example, if a 4096-byte logical data block is divided into four data symbols of 1024 bytes, two 1024-byte erasure code check blocks will be generated. These two 1024-byte check blocks are spliced ​​together to form a 2048-byte erasure code check block.

[0028] Based on the calculated erasure code check block, the I / O controller needs to select a suitable remote storage device for storage. First, the I / O controller queries available devices from a pre-configured "remote storage device list". This list contains each remote storage device's unique identifier, network interface address (such as IP address and port number, or iSCSI Qualified Name (IQN)), storage type (such as a local NVMe SSD on another server or a dedicated network attached storage (NAS)), and current health status and load information. For example, the list may contain a record such as {Device ID: "Remote_SSD_01", IP: "192.168. 1.100", Port: 3260, Protocol: "iSCSI", Health: "OK", Load_Avg_5min: 0.65}, the I / O controller selects one or more candidate remote storage devices from them. The selection strategy can be based on polling, lowest load, or data affinity, etc. Then, for the selected candidate remote storage device, the I / O controller uses its network interface address, for example, the 3260 port of the IP address "192.168.1.100", uses a standard network detection command such as ping or tries to establish a short-lived TCP connection (for example, sending a SYN packet and waiting for a SYN-ACK response, with a timeout of 500 milliseconds). The timeout It is based on the statistical value of the average round-trip time RTT of the intranet. For example, if the average RTT is 50 milliseconds, it is set to 10 times the RTT to tolerate network jitter) to verify its network reachability. If the ping response is successful or the TCP connection is successfully established and a response is received within the specified time, the address is considered valid and the device is online. Then, the I / O controller will send a status query command specific to the storage protocol (for example, for an iSCSI target, send a NOP-Out command and check the status of the NOP-In response) to confirm that the storage service itself is in normal operation and has sufficient available capacity to receive new parity blocks. The criterion for judging the available capacity is that the remaining space reported by the remote storage device must be greater than 1.5 times the size of the erasure code parity block to be written. For example, if the parity block is 2048 bytes, the remote end requires at least 3072 bytes of free space. This 1.5x coefficient This is to reserve a certain amount of space margin to avoid write failures due to critical conditions. Its setting is based on empirical values, taking into account the file system metadata overhead and the space competition that may be caused by concurrent writes. Only remote storage device addresses that meet the conditions of network accessibility, normal service and sufficient capacity are confirmed as available remote addresses.

[0029] Based on the available remote address, the I / O controller uses the TCP / IP protocol stack through the selected data transmission interface, such as the Ethernet interface, to encapsulate the previously generated erasure code check block in the payload of the appropriate network storage protocol (such as iSCSI, NVMe-oF or a custom RPC call) and send it to the remote storage device pointed to by the available remote address. After receiving the data, the remote storage device performs the write operation and returns the write result (success or failure, and the storage credential after success, such as the remote LBA or file handle) to the I / O controller. The I / O controller waits for the write confirmation from the remote storage device. A timeout is set for this waiting process, for example, 5 seconds. This timeout The timeout is set based on the expected remote write latency plus three standard deviations of the statistical mean of network transmission latency to cover most normal fluctuations. For example, if the average remote write latency is 100 milliseconds, the network RTT is 50 milliseconds, and their combined standard deviation is 500 milliseconds, then the timeout can be set to (100+50)+3*500=1650 milliseconds, rounded up to 5 seconds. If a successful write confirmation is received within the timeout, the I / O controller establishes a one-to-one mapping between the original logical data block information (for example, its starting LBA and length on the local SSD, and the previously recorded internal transaction ID) and the erasure code parity block information just successfully written to the remote storage (for example, the DeviceID of the remote storage device, the remote storage location identifier, and the CRC32C checksum of the parity block). This mapping is stored in a persistent metadata index, such as a key-value store system. The key can be the transaction ID of the local logical data block, and the value contains the local storage location, remote storage location, and parity block metadata, forming a redundant data write pair.

[0030] The steps to obtain the readback verification results are: Call the target address and remote address recorded in the redundant data write pair, read the bit-level original values ​​in the corresponding storage units from the NVMe channel respectively, and map the target address data block and the remote address data block one-to-one according to the bit arrangement order to form a matching set of the target address bit sequence and the remote address bit sequence; Based on the matching set of the target address bit sequence and the remote address bit sequence, the weighted burst error sensitivity index is calculated using the following formula: ; in, is the weighted burst error sensitivity index, For the The target address bit value, For the Remote address bit value, For the The importance weight corresponding to each bit is is the continuous error penalty factor, The total number of bits, the default setting ; Based on the weighted burst error sensitivity index, determine whether it is zero or below the tolerance threshold. If the conditions are met, the data consistency is determined to meet the requirements. If not, it is marked that there is a risk of continuous structural errors and a readback verification result is generated.

[0031] Specifically, the target address and remote address recorded in the redundant data write pair are called. Specifically, the I / O controller initiates a read operation based on the starting logical block address (LBA) and data length on the local SSD contained in the redundant data write pair, and the storage location identifier of the erasure code check block recorded on the remote storage device. For the target address, an NVMeRead command is sent to the local enterprise-level solid-state drive through the NVMe channel, specifying the starting LBA and the number of logical blocks to be read to obtain the bit-level content of the original logical data block stored at the target address. For the remote address, a corresponding read command is sent to the remote storage device according to its storage protocol (such as iSCSI or NVMe-oF) to obtain the bit-level original value of the erasure code check block previously stored. Both read operations retrieve data in raw binary form without any formatting. After the conversion or error correction attempt is performed, the target address data block (for example, a 4096-byte logical data) and the remote address data block (for example, a 2048-byte erasure code check block) are obtained. If the (4, 2) RS code is used and the original data is 4096 bytes, the corresponding data segment may be 8192 bytes and the check segment is 4096 bytes. Alternatively, the remote data block and the target data block may be the same size for direct comparison. According to the bit arrangement order of the two, the first bit of the target address data block is mapped to the first bit of the remote address data block, the second bit is mapped to the second bit, and so on, until all bits of the shorter data block are mapped. If the two are of different lengths, only the common length is compared, and a one-to-one mapping is performed to form a matching set of the target address bit sequence and the remote address bit sequence.

[0032] formula: The benefit of the formula is that the weighted burst error sensitivity index not only considers the importance of a single bit difference, but also adds a penalty weight specifically for common burst errors (consecutive multiple bit errors) in storage systems, thereby more accurately assessing the severity of data inconsistency. Parameter This allows bits at different locations in the data to be assigned different levels of importance. For example, errors in key metadata bits can lead to more serious consequences than errors in ordinary data bits. The parameter The introduction of makes the formula more sensitive to continuity errors, which is important for early detection of structural problems that may lead to large-scale data corruption. By comprehensively considering the differences and importance of each bit and the continuity of the error and normalizing them, an indicator that can intuitively reflect the risk of data inconsistency is obtained; is the total number of bits, and the acquisition steps are: determine it according to the actual size of the "target address data block" read from the target address, if the target address data block size is Bytes, then For example, if the target address data block size is 512 bytes, then Bit; For the The target address bit value is obtained by sequentially extracting the first bits of logical value (0 or 1), for example, for an 8-bit sequence ,but ; For the The remote address bit value is obtained by sequentially extracting the first remote address bit value from the remote address bit sequence formed. The logical value of the bits (0 or 1) must be the same as the target address bit sequence or in the order of For example, for an 8-bit sequence ,but ; For the The importance weight corresponding to each bit is obtained as follows: the system presets a "bit importance configuration table", which assigns different ranges of bits to the key areas (such as file header, metadata area, user data area) according to the criticality of different areas in the data block. Assign corresponding weight values. These weight values ​​are set by the system designer based on experience and analysis of the importance of the data structure. For example, for a 512-byte (4096-bit) data block, bytes 0-31 (bits 0-255) can be set as header information, and their weights are Set to 3.0, bytes 32-63 (bits 256-511) are key metadata, weight Set to 2.0, the remaining bytes 64-511 (bits 512-4095) are normal data, weight Set to 1.0. These values ​​(3.0, 2.0, 1.0) reflect the relative impact of different data area damage on the system. For example, header information damage is three times more harmful than ordinary data damage. is the continuous error penalty factor, which is obtained as follows: This parameter is determined based on the statistical analysis of the historical error patterns of enterprise-class solid-state drives. The number of single-bit errors and continuous bit errors (burst errors) that occur in a specific model of hard drive within a certain operating cycle (for example, 10,000 hours) and their length distribution are counted. If the statistics show that burst errors account for a high proportion (for example, more than 15% of the total error events) and the damage caused by continuous errors is significantly greater than the same number of isolated errors, then set , its specific value, such as 1.5, is obtained through calibration experiments, that is, injecting different types of errors and observing The change of the index, combined with the evaluation of the actual impact of the error on the system, selects the one that can effectively amplify the impact of the sudden error Value, for example, setting The penalty for consecutive errors is 1.5 times that of isolated errors. Default settings , this is to deal with the formula hour Boundary conditions of items; Calculation process: set up Bit, Target address bit sequence , Remote address bit sequence , Set the importance weight of the first 4 bits to , the importance weight of the last 4 bits is ,Right now , Set the continuous error penalty factor , set up , First calculate , Then calculate the difference of each bit : ; ; ; ; ; ; ; ; So, the difference sequence , Then calculate the sum ,in : for , for , for , for , for , for , for , for , The summation result is .

[0033] so, .

[0034] The results show that the calculated weighted burst error sensitivity index is 0.25. This value is not zero, indicating that there is a difference between the target address bit sequence and the remote address bit sequence. The index combines the importance of the position of the error bit and whether it is a continuous error. The larger the value, the higher the risk or severity of data inconsistency. A value of 0 indicates complete consistency.

[0035] According to the weighted burst error sensitivity index , which is 0.25 obtained in the previous calculation step, the I / O controller makes a judgment and first judges Is it zero? In this case, 0.25 is not zero. Then judge Is it lower than the preset "data consistency tolerance threshold"? The setting of this tolerance threshold is based on the data consistency requirements of the application scenario and historical data analysis. For example, for highly sensitive financial transaction data, the tolerance threshold may be set very low, close to 0, such as 0.001. This value means that even the slightest, lowest-weighted single-bit difference may be considered unacceptable. For some unstructured data that allows a certain degree of data distortion (such as non-key frames of video streams), the tolerance threshold may be set higher, such as 0.05. The process of setting this value usually includes: analyzing different The actual impact of the value on the business is collected in different In the case of system failure or data error at the lowest level, an acceptable level is determined through risk assessment model. Upper limit, for example, by analyzing 1000 historical data verification events, it is found that When the tolerance is lower than 0.01, no data inconsistency events that have a substantial impact on the business have occurred. When it is higher than 0.01, there is a 5% probability of perceptible business anomalies. Therefore, the data consistency tolerance threshold is set to 0.01. If the calculated (0.25 in this example) is zero or lower than the tolerance threshold of 0.01, the data consistency is determined to meet the requirements. If this condition is not met (for example, 0.25>0.01 in this example), the corresponding data block is marked as having a potential risk of continuity structural error, and a judgment result is generated (for example, "inconsistent, high risk, ”) and the readback verification results of related diagnostic information (such as the inconsistent bit position, weight, and continuity).

[0036] The steps to obtain the coordinates of the unstable physical area are: Based on the data inconsistency items in the read-back verification results, the target address bit sequence corresponding to the data inconsistency item is parsed, the logical position of the bit where the data inconsistency occurs in the target address bit sequence is recorded bit by bit, and all the logical positions are deduplicated and sorted to form a set of data inconsistency logical positions; Based on the set of inconsistent logical locations, the NVMe interface is called to query the internal error log of the enterprise-level solid-state drive for the physical area number and sector address corresponding to each logical location. The coordinates of each logical location mapped to the physical area are recorded to form a physical area information mapping table. Based on the physical area information mapping table, the physical area numbers and corresponding sector addresses where data inconsistency occurs repeatedly are identified and marked as unstable. The physical coordinate information corresponding to the marked position is stored in sequence to generate unstable physical area coordinates.

[0037] Specifically, based on the data inconsistency items in the readback verification results, such as the data obtained in the previous step and its associated bit difference sequence The I / O controller first extracts the target address bit sequence corresponding to the target data block marked as inconsistent from the read-back verification result. and remote address bit sequence , and then compare each bit one by one, and record all Bit index of , these indexes are the logical positions of the bits where the data inconsistency occurs within the data block. and , the inconsistent logical position is bit 3 (index 3, because ) and position 6 (index 6, because ), if the same data block generates duplicate logical position records due to multiple verifications or verification of multiple copies, these recorded logical positions are deduplicated. For example, if the record is [3, 6, 3], it becomes [3, 6] after deduplication. Then the deduplicated logical position list is sorted in ascending order. For example, [6, 3] becomes [3, 6] after sorting. This processed logical position list is the data inconsistent logical position set.

[0038] Based on the set of inconsistent logical locations of data, such as [3, 6], which contains the logical offsets of the inconsistent bits in the data block, the I / O controller calls the NVMe interface to send a specific AdminCommand to the enterprise-level SSD, such as the GetLogPage command in conjunction with the ErrorLogPage or a more detailed manufacturer-defined error diagnosis LogPage, to query the physical storage information related to these logical locations. Since the NVMe protocol itself does not directly provide direct queries from logical bit offsets to physical addresses, this process is usually indirect: first, the logical bit position is converted into its logical block address (LBA) and the offset within the block. For example, if the data block size is 512 bytes (4096 bits), logical positions 3 and 6 both fall within the first LBA of the data block. Then, the error log entry related to the LBA is queried. The error log entry may contain the physical block information associated with the most recent one or more read / write operations failed on the LBA (such as the physical block address P The I / O controller parses these log entries, extracts the physical area number (e.g., a unique physical block identifier composed of a combination of channel, die, plane, block, etc.) and the specific sector address (or a finer-grained physical page address) within the physical area to which each inconsistent logical location (or its LBA) is mapped, and records the mapping relationship between these logical locations and physical locations. For example, logical location 3 (belonging to LBAX) may be mapped to sector address 0x0010 within physical area number 0x1A2B3C4D. Logical location 6 (also belonging to LBAX) belongs to the same logical block and is also mapped to sector address 0x0010 within the same physical area number 0x1A2B3C4D (e.g., the sector is the smallest addressing unit, or the error log is only recorded at the sector level). All such queried mapping relationships are summarized to form a physical area information mapping table.

[0039] Based on the physical region information mapping table, for example, including records such as {(LBA_X, Bit_Offset_3)->(Physical_Region_ID_0x1A2B3C4D, Sector_0x0010)} and {(LBA_Y, Bit_Offset_1027)->(Physical_Region_ID_0x5E6F7A8B, Sector_0x0088)} and {(LBA_Z, Bit_Offset_2050)->(Physical_Region_ID_0x1A2B3C4D, Sector_0x0010)}, the I / O controller analyzes the records in the table and counts each unique combination of "physical region number and corresponding sector address" (for example, (0x1A 2B3C4D, 0x0010)) appears, set an "unstable area determination threshold", which defines the minimum number of data inconsistencies at the same physical location within a certain observation period (for example, the past 24 hours or the last 1000 verification events). If the number exceeds this number, the physical location is considered unstable. The basis for setting this threshold is a comprehensive consideration of the mean time between failures (MTBF) of the hard disk, the typical failure mode of a specific SSD model, and the business's requirements for data reliability. For example, for an enterprise-class SSD with an MTBF of 2 million hours, if the observation period is set to the last 100 verifications, if a physical location has 2 or more data inconsistencies in 100 verifications (that is, a threshold of 2), it is considered to have significantly deviated from the normal random error rate and there may be potential physical media defects. The error distribution characteristics of a large number of hard drives of the same model under normal operating conditions can be statistically analyzed, and the frequency corresponding to the tail percentile of the error frequency distribution can be added by 1 to identify abnormally high-frequency error locations. For example, if 99.9% of the physical locations have fewer than 2 errors in 100 checks, the threshold can be set to 2. If the physical area (0x1A2B3C4D, 0x0010) appears twice in the mapping table (corresponding to inconsistency between LBA_X and LBA_Z), the physical area is marked as unstable. All physical area numbers and corresponding sector addresses marked as unstable are stored in the order of their appearance in the physical area information mapping table or in the natural order of the physical addresses to form a list or set. This list is the unstable physical area coordinates.

[0040] The steps to obtain the request priority identifier are: The host bus adapter receives the I / O request, decomposes the message header and payload of the I / O request segment by segment, extracts the application identification field and task type field contained therein, and caches and records them in the order of receipt to form an I / O request parsing record; According to the I / O request parsing record, the pre-set application priority mapping table is called. By comparing the application identification field and task type field with the importance level of each task recorded in the mapping table, the application priority level corresponding to each I / O request is determined to form an application priority level sequence; Based on the application priority level sequence, each I / O request is assigned a request priority code corresponding to the application priority level one by one, and the request priority code is associated and bound with the corresponding I / O request to generate a request priority identifier.

[0041] Specifically, the host bus adapter receives an I / O request from an upper-layer application or operating system. The request is usually encapsulated in a standardized format. For example, for the NVMe protocol, it is a submission queue item. The HBA parses the received SQE and first checks the fixed fields in the SQE header, such as the command operation code (Opcode, such as read, write), namespace ID (NSID), etc. Then, if the SQE contains a pointer to metadata or a specific extension field, the HBA will further parse these optional payload parts. The key is to extract the "application identification field" and "task type field" from specific fields of the SQE or through an agreement with the operating system kernel (for example, the control block accompanying the I / O request contains additional information). The application identification field can be a unique The task type field describes the specific business meaning or importance of the I / O request, such as "Online Transaction Processing (OLTP)", "Batch Data Analysis (OLAP)", "Metadata Operation", "Log Writing", "Background Backup", and other enumeration values. The HBA extracts the application ID and task type fields, along with the I / O request reception timestamp (accurate to microseconds) and an internally generated unique request ID, and temporarily stores them as a structured record in a first-in-first-out (FIFO) memory buffer to form an I / O request resolution record.

[0042] Based on the application identification field and task type field extracted from the I / O request parsing record, the host bus adapter calls a pre-set "application priority mapping table" loaded into the HBA firmware or driver memory. This mapping table is a two-dimensional lookup table or a set of rules. Its construction is based on the results of evaluating and quantifying the importance of various application systems deployed within the enterprise. The evaluation process is usually jointly participated by system administrators and business managers, and is determined based on factors such as the application's business impact level, service level agreement requirements (such as maximum allowed latency, minimum IOPS guarantee), and data sensitivity. For example, the mapping table may contain the following entries: {Application ID: "CRM_System_PID_Range_1000_1050", Task Type: "OLTP_Write", Importance Level: 5 (Highest)}; {Application ID: "Reporting_Service_Hash_XYZ", Task Type: "OLAP_Read", Importance Level: 3 (Medium)}; {Application ID: "Ba ckup_Agent_Enum_0x03", Task Type: "Sequential_Write", Importance Level: 1 (Lowest)}, where Importance Level is a numerical value, such as 1 to 5, with a larger value representing a higher priority. The HBA matches the Application Identifier field and Task Type field in the current I / O request parsing record against the entries in the mapping table one by one. When a fully matched entry is found, the Importance Level defined in that entry is used as the application priority level of this I / O request. If there is no exact match in the Application Identifier field or Task Type field in the table, a preset default priority level, such as Level 2 (lower), can be used. Alternatively, a wildcard rule can be used to match. For example, all I / O requests with "DB_APP_ID" that do not explicitly specify a task type are assigned Importance Level 4 by default. Through this process, each I / O request parsing record is assigned a clear application priority level. All these levels are arranged in the order in which the I / O requests are received to form an application priority level sequence.

[0043] Based on the application priority level sequence, for example, the sequence [5, 3, 5, 1, 4] is obtained, the host bus adapter looks up the corresponding "request priority code" for each application priority level in the sequence (for example, the first one is 5). The request priority code is a standardized or internal encoding representation of the application priority level, which is usually designed to be more suitable for hardware processing or queue scheduling. For example, the importance levels from 1 to 5 can be directly mapped to numerical codes (for example, level 5 corresponds to code 0x05, level 1 corresponds to code 0x01), or mapped to a specific hardware priority queue index. This mapping relationship is pre-defined. For example, a simple direct mapping is: {level 1->code 0x01 (lowest priority)}, {level 2->code 0x02}, {level 3->code 0x 03}, {level 4->code 0x04}, {level 5->code 0x05 (highest priority)}, the HBA takes out the level values ​​in the application priority level sequence one by one, converts them into request priority codes through this mapping relationship, and then associates and binds the generated request priority code with the original I / O request (usually identified by its unique request ID inside the HBA). This binding can be achieved by adding a priority code field in the internal descriptor of the I / O request, or storing the request ID and the corresponding priority code in a dedicated priority scheduling data structure, such as a priority-sorted request queue or hash table. After completing this operation, each pending I / O request carries a clear request priority identifier that can be used for subsequent routing decisions and resource scheduling.

[0044] The steps to obtain the target drive routing decision are: After receiving the request priority identifier, the host bus adapter sequentially calls the new drive dynamic latency profile from all enterprise-class SSDs, extracting the average response time, I / O queue depth, and idle time ratio within the most recent fixed period for each block. The original parameters are then normalized to their maximum and minimum values ​​to generate normalized response time, normalized queue depth, and normalized idle time ratio. Based on the normalized response time, normalized queue depth, and normalized idle ratio, the priority-sensitive cost index of each enterprise-class SSD is calculated using the following formula: ; in, For the The priority-sensitive cost index of enterprise-class solid-state drives, The normalized value of the request priority identifier. is the normalized response time, indicating the The response rate of an enterprise-class solid-state drive within a set time window, is the normalized queue depth, indicating the The current queue level of enterprise-class solid-state drives, is the normalized idle time ratio, indicating the The percentage of time that an enterprise-class solid-state drive is idle, is a constant used to balance the impact of idleness on the cost index; Based on the priority-sensitive cost index of all enterprise-class solid-state drives, hard drives that do not contain unstable physical area coordinates are screened out, and the hard drive with the smallest priority-sensitive cost index is selected from the remaining hard drives. The device identifier is converted into a system drive letter to generate a target drive letter routing decision.

[0045] Specifically, after the host bus adapter receives the request priority identifier, for example, the request priority code is 0x05, the host bus adapter will traverse the list of all connected enterprise-class solid-state drives it manages. For each solid-state drive in the list (hereinafter referred to as the "target hard drive"), it first calls the new drive dynamic latency profile corresponding to the target hard drive, which is created and continuously updated in the previous step, and extracts the dynamic latency profile within the most recent fixed period (for example, the last 5 seconds, the period) from the profile. The setting refers to the typical change rate of system load characteristics and I / O behavior. 5 seconds is an empirical balance point between ensuring data freshness and avoiding too frequent fluctuations). The specific extracted parameters include: average response time, that is, the average of the delayed response time of all completed I / O operations within the 5-second period, for example, 250 microseconds is calculated; I / O queue depth, that is, the number of pending commands in the NVMe submission queue of the target hard disk at the end of the 5-second period, for example, recorded as 32; idle time ratio, that is, the total time during which the target hard disk is in a state of no I / O operation (that is, the queue is empty and no command is executed) within the 5-second period, divided by the total period length (5 seconds) to obtain the percentage, for example, 15% is calculated. Subsequently, the host bus adapter performs maximum and minimum normalization processing on these original parameters (250 microseconds, 32, 15%). This normalization is performed on similar parameters of all currently managed hard disks, that is, finding the maximum value of the parameter among all hard disks. and minimum value , and then set the parameter value for each hard disk Apply the formula (like ,but Set to 0 or 0.5), so that all parameter values ​​are mapped to the range of 0 to 1. For example, if the average response time of all hard disks ranges from 100 microseconds to 500 microseconds, then 250 microseconds is normalized to , and finally generate the normalized response time, normalized queue depth, and normalized idle ratio of the target hard disk.

[0046] formula: The benefit of the formula is that it dynamically combines the priority of the I / O request and the real-time performance status of the target SSD (response time, queue depth, idleness) to calculate a comprehensive cost index through the exponential function and As a weight, high-priority requests are more sensitive to response time, while low-priority requests are more sensitive to queue depth, so that the request characteristics and hard disk load can be more intelligently matched when selecting the target disk. The parameter The introduction of balancing considerations for disk idleness avoids situations where a disk with poor performance is selected because it is idle, or a disk with excellent performance is ignored because it is constantly busy. This design enables routing decisions to better strike a balance between load balancing and performance satisfaction, improving overall I / O service quality. The normalized value of the request priority identifier is obtained as follows: first, obtain the request priority code generated in the previous step, for example, the code range is 0x01 to 0x05, representing the priority from low to high, map these codes to a numerical range, for example, directly use its decimal value 1 to 5, and then normalize this value and map it to the interval of 0 to 1. If the original priority code is higher, it means the priority is higher. For example, for a request priority code of 0x04 (decimal 4), the minimum code is 0x01 (decimal 1), and the maximum code is 0x05 (decimal 5). ; For the The normalized response time of an enterprise-class solid-state drive is obtained by: directly using The normalized response time value calculated for each solid-state drive is between 0 and 1. A smaller value indicates a faster response. For example, for hard drive k, its normalized response time is 0.375. For the The normalized queue depth of an enterprise-class solid-state drive is obtained by: directly using The normalized queue depth value calculated for each SSD is between 0 and 1. A larger value indicates a deeper queue and a busier queue. For example, for hard drive k, its normalized queue depth is 0.6. For the The normalized idle time ratio of enterprise-level solid-state drives is obtained by: directly using The normalized idle time ratio calculated for each solid-state drive is between 0 and 1. A larger value indicates a more idle drive. For example, for drive k, its normalized idle time ratio is 0.2. is a constant used to balance the impact of idleness on the cost index. The steps to obtain it are: The setting is a tuning parameter whose value is determined by the system's preference for load balancing and performance response. It is usually adjusted through a series of benchmark tests and simulation experiments to observe different The overall I / O performance of the system (such as average throughput, average latency, and satisfaction rate of high-priority requests) changes under different values, and a value that enables the system to achieve the desired goal is selected. For example, if you want to make more active use of idle hard disks, you can increase If you are more concerned about avoiding sacrificing the performance of the already loaded hard disk in pursuit of idleness, then reduce , an empirical initial value can be set to 0.5, which means that the impact of idleness is roughly equivalent to the impact of response time and queue depth. For example, after testing, it was found that in the current system When , the delay jitter of high priority tasks is minimized, while the overall throughput is kept at a high level, so ; Calculation process: No. An enterprise-class solid-state drive with the following parameters: ; ; ; ; ; Calculate the exponential term: ; ; Calculate the idleness penalty: ; Substitute into the formula to calculate : ; ; ; ; This result shows that the The priority-sensitive cost index of an enterprise-class solid-state drive for this I / O request with a priority of 0.75 is 1.804275. The smaller the index, the more suitable the hard drive is for processing the current I / O request with a specific priority. The index comprehensively considers the hard drive's response speed, current load level, and idle status.

[0047] Based on the priority sensitive cost index of all enterprise-class solid-state drives, for example, the calculated cost of hard drive A is: , hard disk B , Hard Drive C The host bus adapter will first check for each hard disk from a pre-maintained "unstable physical area coordinate list" to see if it contains any known physical area coordinates marked as unstable in the previous step. The list is generated and dynamically updated by the previous "unstable physical area coordinate acquisition step". If a hard disk's device identifier (such as its WWN or serial number) appears in the hard disk associated with the list, and the current logical block address (LBA) range to be written may fall into the marked unstable physical area of ​​the hard disk through a certain mapping relationship (such as based on LBA to PBA conversion logic. Although the precise real-time LBA to PBA mapping is usually managed internally by the SSD, the HBA layer can be judged based on historical data or rough interval correspondence), then the hard disk will be excluded from the candidate list. For example, if hard disk A is detected to have a known unstable physical area that is at risk of overlapping with the target LBA range of the current I / O request, hard disk A is excluded, leaving hard disk B and hard disk C. The host bus adapter selects a priority-sensitive cost index from these remaining hard disks that have passed the screening (in this case, hard disk B and hard disk C). The smallest hard drive is used as a target, and the comparison and , the cost index of hard disk B is lower, so hard disk B is selected. Finally, the device identifier of the selected hard disk B (for example, its device node name at the operating system level, such as / dev / nvme0n1, or its hardware path information) is converted into a system drive letter recognizable at the application or file system level (for example, if the hard disk is mounted as the D: drive or the root directory of a file system). This conversion process relies on the device management and file system mounting information provided by the operating system, and ultimately forms a specific target drive letter routing decision, such as "route the I / O request to / dev / nvme1n1 (corresponding to the system drive letter E:)".

[0048] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for intelligent expansion of enterprise-level solid-state hard drives, characterized in that: The following steps are involved: The I / O controller detects the newly connected enterprise-class SSD and sets a total write threshold. The I / O controller continuously obtains the real-time latency value of the enterprise-class SSD and creates a dynamic latency profile for the new drive. Based on the dynamic latency profile of the new disk, when the I / O controller receives a write request, it writes the logical data block and the check code to the target address, obtains a write certificate for the target address, and based on the write certificate for the target address, generates an erasure code check block for the logical data block and writes it to the remote address, thereby establishing a redundant data write pair. Based on the redundant data write pair, read the data of the target address and the remote address and perform a consistency check to obtain a read-back check result; based on the data inconsistency items in the read-back check result, extract the physical area information in the error log through the NVMe interface to establish the unstable physical area coordinates; The host bus adapter parses the received I / O request, extracts the application priority, generates a request priority identifier, calculates the matching degree with the dynamic delay profile of the new disk based on the request priority identifier, and avoids the unstable physical area coordinates to generate a target drive letter routing decision.

2. The intelligent capacity expansion method for enterprise-level solid-state drives according to claim 1, wherein: The steps for obtaining the dynamic delay profile of the new disk are as follows: The I / O controller monitors the physical connection port of the SSD, confirms that the voltage and current signals on the port are stable, identifies the device identifier and feature code sent by the port, parses the data, obtains confirmation that the connected device is an enterprise-class SSD, and generates an enterprise-class SSD access identifier; Based on the enterprise-class solid-state drive access identifier, the I / O controller calls the manufacturer's firmware interface to read the maximum lifespan parameter preset by the solid-state drive, parses the theoretical write volume value supported by the maximum lifespan parameter, sets the total write volume threshold allowed by the solid-state drive, and generates a total write volume threshold; Based on the total write volume threshold, the I / O controller continuously calls the internal latency monitoring register of the enterprise-level solid-state drive through the NVMe interface in real time, reads the latency response time of each operation, arranges the latency response time in time series and stores it to generate a new dynamic latency profile of the disk.

3. The intelligent expansion method for enterprise-level solid-state drives according to claim 1, wherein: The steps for obtaining the target address writing credential are as follows: After receiving the write request, the I / O controller parses the logical data block in the write request, calculates the check code corresponding to the logical data block, completes data splicing of the logical data block and the check code, and generates a logical data block and check code splicing unit; Based on the logic data block and the check code splicing unit, calling the new disk dynamic delay profile, matching the real-time delay values ​​corresponding to the logic data block and the check code splicing unit, screening the available target address with the lowest real-time delay value, and forming a screening result of the available target addresses; Based on the screening result of the available target addresses, the logic data block and the check code splicing unit are written into the target address through the PCIe bus protocol. After the target address is written, the unique certificate identifier of the writing is obtained to form a target address writing certificate.

4. The intelligent capacity expansion method for enterprise-level solid-state drives according to claim 1, wherein: The steps of obtaining the redundant data write pair are: Based on the target address write voucher, the I / O controller extracts the logical data block information recorded in the target address write voucher, and performs byte-by-byte redundancy check calculation on the logical data block information according to the erasure code operation rules to obtain an erasure code check block; Parsing the network interface address of the remote storage device according to the erasure code check block, and verifying the address validity and connection status of the remote storage device to obtain an available remote address; Based on the available remote address, the I / O controller sends the erasure code check block to the remote storage device through the data transmission interface, and after the remote storage device confirms the write, establishes a one-to-one mapping relationship between the logical data block information and the erasure code check block to form a redundant data write pair.

5. The intelligent capacity expansion method for enterprise-level solid-state drives according to claim 1, wherein: The steps for obtaining the readback verification result are: Call the target address and remote address recorded in the redundant data write pair, read the bit-level original values ​​in the corresponding storage units from the NVMe channel respectively, and map the target address data block and the remote address data block one-to-one according to the bit arrangement order to form a matching set of the target address bit sequence and the remote address bit sequence; Calculating a weighted burst error sensitivity index based on a matching set of the target address bit sequence and the remote address bit sequence; According to the weighted burst error sensitivity index, it is determined whether it is zero or below the tolerance threshold. If the condition is met, the data consistency is determined to meet the requirements. If not, it is marked that there is a risk of continuous structural error and a readback verification result is generated.

6. The intelligent capacity expansion method for enterprise-level solid-state drives according to claim 1, wherein: The steps for obtaining the coordinates of the unstable physical area are: Based on the data inconsistency items in the read-back verification result, the target address bit sequence corresponding to the data inconsistency item is parsed, the logical position of the bit where the data inconsistency occurs in the target address bit sequence is recorded bit by bit, and all the logical positions are deduplicated and sorted to form a set of data inconsistency logical positions; Based on the set of inconsistent logical locations, the NVMe interface is called to query the physical area number and sector address within the physical area corresponding to each logical location from the internal error log of the enterprise-level solid-state drive one by one, and the position coordinates of each logical location mapped to the physical area are recorded to form a physical area information mapping table; Based on the physical area information mapping table, the physical area numbers and corresponding sector addresses where data inconsistency occurs repeatedly are identified and marked as unstable, and the physical coordinate information corresponding to the marked positions is stored in sequence to generate unstable physical area coordinates.

7. The intelligent capacity expansion method for enterprise-level solid-state drives according to claim 1, wherein: The steps for obtaining the request priority identifier are: The host bus adapter receives the I / O request, decomposes the message header and payload of the I / O request segment by segment, extracts the application identification field and task type field contained therein, and caches and records them in the order of receipt to form an I / O request parsing record; According to the I / O request parsing record, calling a pre-set application priority mapping table, determining the application priority level corresponding to each I / O request by comparing the application identification field and the task type field with the importance levels of various tasks recorded in the mapping table, and forming an application priority level sequence; Based on the application priority level sequence, each I / O request is assigned a request priority code corresponding to the application priority level one by one, and the request priority code is associated and bound with the corresponding I / O request to generate a request priority identifier.

8. The intelligent capacity expansion method for enterprise-level solid-state drives according to claim 1, wherein: The steps for obtaining the target drive letter routing decision are as follows: After receiving the request priority identifier, the host bus adapter sequentially calls the new disk dynamic latency profile from all enterprise-class solid-state drives, extracts the average response time, I / O queue depth, and idle time ratio within a recent fixed period, and performs maximum and minimum normalization on the original parameters to generate normalized response time, normalized queue depth, and normalized idle time ratio; Calculate the priority-sensitive cost index for each enterprise-class SSD based on normalized response time, normalized queue depth, and normalized idle percentage; Based on the priority-sensitive cost index of all enterprise-class solid-state drives, hard drives that do not contain unstable physical area coordinates are screened out, and the hard drive with the smallest priority-sensitive cost index is selected from the remaining hard drives. The device identifier is converted into a system drive letter to generate a target drive letter routing decision.

9. The intelligent capacity expansion system of the enterprise-level solid-state drive intelligent capacity expansion method according to any one of claims 1 to 8, characterized in that: include: Detection module: The I / O controller detects newly connected enterprise-class SSDs and sets a total write threshold. The I / O controller continuously obtains the real-time latency value of the enterprise-class SSD and creates a dynamic latency profile for the new drive. Write module: Based on the dynamic latency profile of the new disk, when the I / O controller receives a write request, it writes the logical data block and the check code to the target address, obtains the target address write certificate, and based on the target address write certificate, generates an erasure code check block for the logical data block and writes it to the remote address, thus establishing a redundant data write pair. Verification module: Based on the redundant data write pair, read the data of the target address and the remote address and perform consistency verification, obtain the read-back verification result, and based on the data inconsistency items in the read-back verification result, extract the physical area information in the error log through the NVMe interface to establish the unstable physical area coordinates; Scheduling module: The host bus adapter parses the received I / O requests, extracts the application priority, generates a request priority identifier, calculates the matching degree with the dynamic delay profile of the new disk based on the request priority identifier, and avoids the unstable physical area coordinates to generate the target drive letter routing decision.

Images