Data processing method and device, electronic equipment and storage medium

Abstract

This application provides a data processing method, apparatus, electronic device, and storage medium; applied to a host bus adapter (HBA), the method comprising: acquiring device information of multiple storage devices connected to the HBA; configuring at least two non-volatile memory standard NVMe sets corresponding to the storage space of the multiple storage devices based on the device information; wherein different NVMe sets correspond to different independent disk redundant array RAID levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels; receiving a first control command sent by a host; the first control command is used to control the HBA to create a target namespace of a target RAID level; creating a target namespace for the NVMe sets based on the first control command; the target namespace is generated in the storage space of the storage devices; and, in response to receiving a data processing request from the host, performing data processing on the data processing request based on the target namespace. This improves resource utilization and data processing flexibility.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Current host bus adapters (HBAs) typically employ a fixed RAID configuration, meaning each hard drive array supports only a single RAID level. This design has significant limitations. First, the same group of physical storage devices cannot simultaneously meet the data protection needs of different applications on the host side, resulting in low resource utilization. Second, when application requirements on the host side change, the entire hard drive array needs to be reconfigured, causing service interruptions. Therefore, current data processing methods suffer from low resource utilization and poor data processing flexibility of host bus adapters. Summary of the Invention

[0003] This application provides a data processing method, apparatus, electronic device, and storage medium to solve one or more problems existing in the related art.

[0004] This application provides a data processing method applied to a host bus adapter (HBA). The method includes: acquiring device information of multiple storage devices connected to the HBA; configuring at least two non-volatile memory standard NVMe sets corresponding to the storage spaces of the multiple storage devices based on the device information; wherein different NVMe sets correspond to different independent disk redundant array (RAID) levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels; receiving a first control command sent by a host; the first control command is used to control the HBA to create a target namespace for a target RAID level; creating a target namespace for the NVMe sets based on the first control command; the target namespace is generated in the storage space of the storage devices; and processing the data request based on the target namespace in response to receiving a data processing request from the host.

[0005] According to one embodiment of this application, before receiving the first control command sent by the host, the method further includes: in response to receiving the query command sent by the host, sending an NVMe set attribute entry corresponding to the HBA to the host, so that the host determines the target NVMe set and generates the corresponding first control command based on the NVMe set attribute entry; the NVMe set attribute entry contains RAID level information corresponding to each NVMe set.

[0006] According to one embodiment of this application, the step of processing the data processing request based on the target namespace includes: obtaining RAID level information of the NVMe set corresponding to the target namespace; splitting the data processing request based on the RAID level information to obtain at least one data transmission instruction and at least one data verification instruction; controlling the storage device corresponding to the NVMe set based on the data transmission instruction, and performing verification calculation of the RAID level corresponding to the NVMe set based on the data verification instruction to obtain the data processing result.

[0007] According to one embodiment of this application, obtaining device information of multiple storage devices connected to the HBA includes: identifying the multiple storage devices connected to the HBA during HBA initialization to obtain the device information; the device information includes at least the number of storage devices, the storage device capacity, and the device interface type.

[0008] According to one embodiment of this application, configuring at least two NVMe sets corresponding to the storage space of the plurality of storage devices based on the device information includes: obtaining set configuration information; the configuration information characterizes the RAID level included in the HBA and the number of NVMe sets created; creating a plurality of NVMe sets based on the configuration information and the device information, each NVMe set corresponding to a RAID level; and establishing an attribute information table for each NVMe set, the attribute information table including at least the identification information, performance data, capacity information, and RAID level information of the NVMe set.

[0009] According to one embodiment of this application, creating a target namespace for the NVMe set based on the first control instruction includes: parsing the first control instruction to obtain the corresponding target RAID level and target NVMe set; allocating the target namespace in the storage space of the storage device based on the NVMe namespace management command; and establishing a mapping relationship between the target namespace and the target NVMe set so that the target namespace can perform data processing for the target RAID level.

[0010] According to one embodiment of this application, the method further includes: encoding the RAID level information and storing it in the setting field of the NVMe set attribute entry.

[0011] This application also provides a data processing apparatus, comprising: an acquisition module for acquiring device information of multiple storage devices connected to an HBA; a configuration module for configuring at least two non-volatile memory standard NVMe sets corresponding to the storage spaces of the multiple storage devices based on the device information; wherein different NVMe sets correspond to different independent disk redundant array RAID levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels; a receiving module for receiving a first control command sent by a host; the first control command is used to control the HBA to create a target namespace of a target RAID level; a creation module for creating a target namespace for the NVMe sets based on the first control command; the target namespace is generated in the storage space of the storage device; and a processing module for processing data based on the target namespace in response to receiving a data processing request from the host.

[0012] This application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of the above-described embodiments.

[0013] This application also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described above.

[0014] The method described in this application embodiment, applied to a host bus adapter (HBA), includes: acquiring device information of multiple storage devices connected to the HBA; configuring at least two non-volatile memory standard NVMe sets corresponding to the storage spaces of the multiple storage devices based on the device information; wherein different NVMe sets correspond to different independent disk redundant array (RAID) levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels; receiving a first control command sent by a host; the first control command is used to control the HBA to create a target namespace for the target RAID level; creating a target namespace for the NVMe sets based on the first control command; the target namespace is generated in the storage space of the storage devices; and, in response to receiving a data processing request from the host, performing data processing on the data processing request based on the target namespace. This improves resource utilization and data processing flexibility.

[0015] It should be understood that the teachings of this application are not required to achieve all the beneficial effects described above, but rather that a specific technical solution can achieve a specific technical effect, and other embodiments of this application can also achieve beneficial effects not mentioned above. Attached Figure Description

[0016] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which:

[0017] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0018] Figure 1 This application illustrates a schematic diagram of the data processing method provided in an embodiment. Figure 1 ; Figure 2 This application illustrates a schematic diagram of the data processing method provided in an embodiment. Figure 2 ; Figure 3 This application illustrates an application scenario of the data processing method provided in the embodiments of this application. Figure 1 ; Figure 4 This application illustrates an application scenario of the data processing method provided in the embodiments of this application. Figure 2 ; Figure 5 An optional schematic diagram of the data processing apparatus provided in an embodiment of this application is shown; Figure 6 A schematic diagram of the composition structure of the electronic device provided in the embodiments of this application is shown. Detailed Implementation

[0019] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0024] PCIe: A high-speed serial computer expansion bus standard, short for Peripheral Component Interconnect Express. It adopts an efficient point-to-point connection protocol and features high speed, low latency, and high bandwidth. It supports hot-swapping and is widely used in various high-performance components and peripherals in modern computer systems. It is the core technology for connecting modern high-performance hardware.

[0025] NVMe: NVMe (Non-Volatile Memory Express) is a communication protocol based on the PCIe (PCI-Express) interface, designed specifically for SSDs (Solid State Drives) to provide high-speed, low-latency storage access. NVMe interface communication is implemented based on a paired Submission Queue (SQ) and Completion Queue (CQ) mechanism. The upstream host software places commands into the Submission Queue (SQ). The controller places the completion result into the corresponding Completion Queue (CQ). The SQ and CQ queues are allocated in the upstream host's memory.

[0026] RAID stands for Redundant Array of Independent Disks, a data storage technology that combines multiple independent physical hard drives in a specific way to form a logical storage unit. This improves the data read and write speed of the storage system, enhances data redundancy, and improves fault tolerance. Different RAID levels can achieve different performance and security goals.

[0027] SATA (Serial Advanced Technology Attachment) is an interface standard for connecting computers and storage devices. It uses serial transmission, supports hot-swapping, and compared to the previous Parallel ATA interface, SATA offers higher data transfer rates, better signal integrity, and better compatibility. It has become one of the mainstream interface standards for modern computer storage devices and is widely used in personal computers and servers.

[0028] The processing flow of the data processing method provided in the embodiments of this application will be described. See [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram of the data processing method provided in the embodiments of this application. Figure 1 , will combine Figure 1 Steps S101-S105 are explained below.

[0029] Step S101: Obtain device information of multiple storage devices connected to the HBA.

[0030] In some embodiments, a Host Bus Adapter (HBA) can connect the host to the storage device, expanding and converting the internal bus interface (such as PCIe) of the host server system into an external storage device interface (such as SATA). Specifically, the HBA connects upstream to the host via a PCIe bus, supports the NVMe standard protocol, and appears as a standard NVMe storage device on the host. Downstream, it connects to the storage device via a high-speed interface, realizing the conversion from the NVMe protocol to various popular high-speed interface protocols, as well as various levels of RAID calculation functions. The storage device may include a solid-state drive (SSD) or a hard disk drive (HDD). This application embodiment does not limit the specific storage device. Device information may include parameters such as the number of storage devices, storage device capacity, and device interface type.

[0031] Step S102: Based on the device information, configure at least two non-volatile memory standard NVMe sets corresponding to the storage space of multiple storage devices; wherein, different NVMe sets correspond to different independent disk redundant array RAID levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels.

[0032] In some embodiments, storage space may include physical blocks within a storage device that can be used for data storage. An NVMe set may be a resource partitioning unit within the NVMe controller of an HBA. An NVMe set can logically group physical storage resources to achieve resource management and performance isolation. Each NVMe set contains one or more namespaces and unallocated reserved space. RAID levels may specifically include RAID 0, RAID 5, RAID 6, RAID 1, and RAID 10. This application does not limit the RAID level corresponding to each NVMe set.

[0033] Step S103: Receive the first control command sent by the host; the first control command is used to control the HBA to create the target namespace of the target RAID level.

[0034] In some embodiments, the host side may include a computer system or server that sends storage management instructions and I / O (input / output) requests. The first control instruction may include an NVMe namespace management command. The target namespace created by the first control instruction can be assigned to a corresponding NVMe set for management.

[0035] Step S104: Based on the first control command, create a target namespace for the NVMe set; the target namespace is generated in the storage space of the storage device.

[0036] Step S105: In response to receiving a data processing request from the host, perform data processing on the data processing request based on the target namespace.

[0037] In some embodiments, the target namespace may be a logical storage unit allocated within the storage space of a storage device. The target namespace possesses data protection capabilities at the RAID level of its corresponding NVMe set. Data processing requests may include: I / O requests sent from the host to the HBA. Data processing may include: splitting the data processing request according to the RAID level and executing corresponding data transfer instructions and data verification instructions.

[0038] As an example, after starting up, the HBA first scans the multiple connected storage devices to obtain information such as the number of storage devices, their capacity, and interface types. Based on the preset configuration, it creates at least two logical NVMe sets on the storage space of the connected storage devices. Each NVMe set corresponds to a different RAID level (such as RAID5 and RAID10), and the storage space of the same storage device is divided among multiple NVMe sets belonging to different RAID levels for management. When the host needs to create a namespace, it first sends a query command to obtain the NVMe set attribute entries supported by the HBA. The HBA returns the NVMe set attribute entries containing RAID level information to the host. After parsing, the host selects the target RAID level that meets its data protection requirements and sends the first control command to the HBA. Upon receiving the first control command, the HBA creates the target namespace on the storage space managed by the NVMe set of the corresponding RAID level. Subsequently, when the host sends a data processing request to the target namespace, the HBA obtains the RAID level information of the NVMe set corresponding to the target namespace, splits the data processing request into data transmission commands and data verification commands, controls the storage devices to perform data read and write operations respectively, and performs the corresponding RAID verification calculations, ultimately completing the data processing.

[0039] The technical effects of the method in this application embodiment include: Enhance the flexibility of storage systems: By extending the namespace management commands of the NVMe protocol, storage spaces with different RAID protection levels can be created on different namespaces to meet diverse storage needs.

[0040] Enhance the reliability of the storage system: Based on the importance of the application and performance requirements, you can flexibly select the appropriate RAID level (such as RAID 0, RAID 5, RAID 6, RAID 1, RAID 10) to ensure the high reliability of critical data, while enhancing data redundancy and fault tolerance.

[0041] Excellent compatibility: The HBA is compatible with mainstream HDD and SSD hard drive devices on the market and supports a variety of high-speed interfaces (such as SATA, SAS, PCIe), ensuring the system's wide applicability.

[0042] This application combines the logical isolation capability of NVMe namespaces with RAID data protection technology, achieving for the first time at the HBA level the configuration of multiple RAID levels by namespace. Specifically, this application includes implementation methods for supporting multiple RAID levels, extended NVMe set management commands, dynamic RAID level configuration and logical mapping, system architecture design, and intelligent I / O request processing mechanisms.

[0043] The design integrates multiple RAID levels with NVMe namespaces: It extends RAID level features into the NVMe protocol standard namespace management mechanism, enabling different namespaces to be configured with different data protection levels. By modifying the reserved fields of the NVMe set (bytes 07:04 in Table-1), a RAID level identifier is defined (e.g., 0x8001 represents RAID 1). This allows the same group of physical storage devices to simultaneously support multiple RAID protection strategies, meeting the differentiated performance and reliability requirements of various applications. Furthermore, by extending compatibility with existing protocols through reserved fields, it supports differentiated data protection needs, reduces host-side adaptation costs, and improves resource utilization.

[0044] Dynamic RAID Level Configuration and Logical Mapping: The HBA system software dynamically creates NVMe sets with different RAID levels and supports logical address mapping between namespaces and physical hard drives. RAID configuration information is exposed through NVMe management commands (such as Identify NVM SetList), allowing the host to select the appropriate namespace protection level. This enables flexible pooling and dynamic allocation of storage resources. The host can discover and select the optimal RAID level storage resources in real time based on business needs, improving storage resource utilization and management efficiency, and avoiding the low resource utilization and poor data processing flexibility problems caused by traditional fixed RAID configurations.

[0045] System architecture design: Supports HBA hardware architecture for multiple RAID level NVMe sets (such as the collaborative design of NVMe controller, RAID processor unit, and cache unit). Through the collaborative architecture of NVMe controller, RAID processor unit and high-speed cache unit, the integration of NVMe protocol processing and RAID calculation is realized, reducing protocol conversion latency, improving I / O processing throughput, ensuring high-performance data protection capabilities in multi-RAID level scenarios, and improving the flexibility of data processing.

[0046] Intelligent I / O request processing mechanism: Based on the RAID attributes of the target namespace, the host NVMe I / O request (R... host Split into multiple hard disk operations (R) disk ) and verification calculation (C raid It achieves RAID-aware fine-grained I / O scheduling, dynamically selecting the optimal data splitting and verification strategy based on the RAID level of the namespace, maximizing parallel I / O efficiency while ensuring data redundancy and reliability, and improving the overall storage system performance.

[0047] In some embodiments, the processing flow of the data processing method is illustrated. Figure 2 ,like Figure 2 As shown, the data processing method may also include: Step S2: In response to receiving the query command sent by the host, send the NVMe set attribute entry corresponding to the HBA to the host.

[0048] As an example, a query command may include: the NVMe Set List command defined by the NVMe protocol. The query command can be used to retrieve the NVMe sets supported by the HBA and their RAID level information. NVMe set attribute entries may include: a data structure containing the NVMe set ID (identification information), performance data, capacity, and RAID level information. RAID level information may include: redundancy protection level identifiers such as RAID 0, RAID 5, RAID 6, RAID 1, and RAID 10. For applications with high reliability and integrity requirements, a higher data protection level should be selected; for applications with lower requirements, a lower data protection level should be selected. Typically, RAID data protection levels from low to high are RAID 0, RAID 5, RAID 6, RAID 1, and RAID 10. The host can determine the target NVMe set based on the NVMe set attribute entries and generate the corresponding first control command. The target NVMe set may include: the NVMe set selected by the host based on the RAID level information.

[0049] In some embodiments, the data processing method further includes: encoding RAID level information and storing it in the setting field of an NVMe set attribute entry.

[0050] As an example, an extension is implemented based on the NVMe set attribute entries defined in the NVMe standard protocol, adding RAID level information, as shown in Table 1 below. The original reserved fields (bytes 4 to 7) of the NVMe set attribute entries are used to store the corresponding RAID level information.

[0051] Table 1 NVMe Set Attribute Entries

[0052] For example, if an NVMe set corresponds to RAID 6, then bytes 4 to 7 of the NVMe set attribute entry will store the RAID level information 0x8006.

[0053] Step S3: Receive the first control command sent by the host.

[0054] As an example, the host first sends a query command to the HBA to obtain storage resource configuration information. After receiving the query command, the HBA sends NVMe set attribute entries to the host. The NVMe set attribute entries include the RAID level information corresponding to each NVMe set. After parsing the NVMe set attribute entries, the host selects the RAID6 level NVMe set as the target NVMe set according to business requirements, and then generates the first control command and sends it to the HBA.

[0055] In some embodiments, step S105, which involves processing the data processing request based on the target namespace, may include: obtaining the RAID level information of the NVMe set corresponding to the target namespace; splitting the data processing request based on the RAID level information to obtain at least one data transmission instruction and at least one data verification instruction; controlling the storage device corresponding to the NVMe set based on the data transmission instruction, and performing verification calculations for the RAID level corresponding to the NVMe set based on the data verification instruction to obtain the data processing result.

[0056] In this embodiment, data transfer instructions may include read / write operation commands for the storage device connected to the HBA. These data transfer instructions can be used to control the storage device to perform actual data access. Data verification instructions may include RAID parity calculation instructions.

[0057] As an example, the HBA receives an NVMe standard I / O request R sent by the host. host Analyze R hostThe command word contains the namespace ID, logical block address (LBA), and user data information. According to R... host The RAID level of the NVMe set corresponding to the target namespace will be R host Break it down into one or more I / O operations R for SSDs disk And RAID parity calculation operation C raid The system schedules the SSD controller to complete SSD I / O operations and performs RAID checksum calculations. For example, taking RAID 5 as an example, a typical RAID 5 NVMe set... host Write requests typically need to be split into two R requests. disk Read operation request, 2 Rs disk Write operation request and one RAID parity calculation instruction. Based on 2 R disk Read operation request, 2 Rs disk Write operation requests control the SSD and perform RAID 5 level parity calculations based on RAID parity calculation instructions to obtain the data processing results. This completes the NVMe standard I / O request. host After data operations, an NVMe response command word is generated, which sends the NVMe standard I / O request R. host The data processing results are sent to the host.

[0058] In some embodiments, obtaining device information of multiple storage devices connected to the HBA in step S101 may include: identifying multiple storage devices connected to the HBA during HBA initialization to obtain device information; the device information includes at least the number of storage devices, storage device capacity, and device interface type.

[0059] In this embodiment, HBA initialization can be a hardware self-test process executed after the HBA starts. The number of storage devices can include the total number of hard drives connected to the HBA. The storage device capacity can include the total amount of data that each hard drive can store. The device interface type can include interface standards such as SATA, SAS, or PCIe.

[0060] As an example, after HBA starts, it enters the HBA initialization phase. First, it scans the connected storage devices to obtain device information, identifies 4 SSDs, and records the number of storage devices. It then reads the storage capacity (e.g., 1TB) and device interface type (e.g., PCIe 4.0 x4) of each SSD to obtain device information.

[0061] In some embodiments, step S102, configuring at least two NVMe sets corresponding to the storage space of multiple storage devices based on device information, may include: obtaining set configuration information; the configuration information characterizes the RAID level included in the HBA and the number of NVMe sets created; creating multiple NVMe sets based on the configuration information and device information, each NVMe set corresponding to a RAID level; and establishing an attribute information table for each NVMe set, the attribute information table including at least the identification information, performance data, capacity information, and RAID level information of the NVMe set.

[0062] As an example, the configuration information can be derived from a dynamic configuration file or a pre-defined program. First, the configuration information is obtained from the pre-defined dynamic configuration file. The configuration information indicates that the HBA includes three RAID levels: RAID5, 6, and 10, and that the number of NVMe sets created is 3. Based on the configuration information and the acquired device information (4 1TB SSDs), HBA creates multiple NVMe sets on the storage space of the 4 SSDs. Specifically, 100GB of space is allocated in each SSD to create a RAID5 level NVMe set (SetID=1), 200GB of space is allocated in each SSD to create a RAID6 level NVMe set (SetID=2), and 300GB of space is allocated in each SSD to create a RAID10 level NVMe set (SetID=3), ensuring that each NVMe set corresponds to a RAID level. Finally, HBA establishes an attribute information table for each NVMe set. The attribute information table includes at least the NVMe set's identification information (i.e., SetID=1 / 2 / 3), performance data, capacity information (400GB / 800GB / 1200GB), remaining unallocated space, and RAID level information (i.e., 5 / 6 / 10).

[0063] In some embodiments, step S104, which involves creating a target namespace for the NVMe set based on the first control command, may include: parsing the first control command to obtain the corresponding target RAID level and target NVMe set; allocating the target namespace in the storage space of the storage device based on the NVMe namespace management command; and establishing a mapping relationship between the target namespace and the target NVMe set so that the target namespace can perform data processing at the target RAID level.

[0064] As an example, after the first control command sent by the host reaches the HBA, the HBA first parses the first control command and obtains the target RAID level as RAID6 and the target NVMe set identifier as Set2. Based on the NVMe namespace management command, the HBA allocates 100GB of capacity in the storage space of the storage device corresponding to Set2 as the target namespace. A mapping relationship is established between the target namespace and Set2, ensuring that all subsequent I / O requests to the target namespace will perform double parity calculations according to the RAID6 level of Set2.

[0065] refer to Figure 3 Application scenarios of the data processing method provided in the embodiments of this application Figure 1 It is used in NVMe host bus adapters with multiple RAID levels.

[0066] Component 201: Host Bus Adapter (HBA) connects to the host via the PCIe bus upstream, fully supports the NVMe protocol, and appears as a standard NVMe storage device on the host side; downstream, it connects to the storage device through the device interface type, realizing the conversion from the NVMe protocol to various popular high-speed interface protocols, as well as the calculation function of RAID level independent disk redundant arrays at various levels.

[0067] Component 202: NVMe Controller. Adhering to the NVMe specification and implementing its defined standard functions, it receives management commands or I / O requests from the host through the commit queue defined in the NVMe protocol specification. It temporarily stores these request data in the HBA's internal cache and then hands them over to the HBA system software for decomposition and scheduling. After the management command or I / O operation is completed, it sends the execution result back to the host through the completion queue (CQ), thus fully fulfilling the interaction process specified in the NVMe protocol specification.

[0068] Component 203: The HBA processor, serving as the system software for the entire HBA, runs the HBA system software and performs the following main functions: Device Information Acquisition: Responsible for identifying and initializing storage devices of various interface types downstream, creating NVMe sets with different RAID levels, and establishing and maintaining attribute information tables for each NVMe set. These attribute information tables include the NVMe set's identifier, performance data, physical capacity, capacity information, and any additional RAID level information as defined by the NVMe standard, providing a reference for subsequent target namespace creation on the host side. During HBA system software initialization, three NVMe sets are created, each corresponding to RAID levels 10, 6, and 5, respectively. According to the NVMe standard protocol definition, an NVMe set is a resource partitioning unit within the NVMe system, logically grouping physical storage resources to achieve resource management and performance isolation. Each NVMe set corresponds to one or more target namespaces.

[0069] Management Command Processing: This function handles various NVMe management commands defined by the NVMe protocol from the host. These commands may include an NVMe set list identification command, used to retrieve detailed attribute information for all NVMe sets within an NVMe device. It extends the NVMe set attribute entries defined in the standard NVMe protocol by adding RAID level information, storing this information in the original reserved fields (bytes 4 to 7) of the NVMe set attribute entries. Based on the RAID level protection information in the NVMe set attribute entries and its own application requirements, the host application selects to create a target namespace within the storage space of the storage device managed by the corresponding NVMe set. This created target namespace will possess the RAID level data protection functionality for the corresponding NVMe set.

[0070] I / O Request Processing: Responsible for decomposing, scheduling, and transforming NVMe standard I / O requests received by the NVMe controller, parsing the target namespace ID, logical block address, and user data information in the command word. Based on the RAID level of the NVMe set corresponding to the target namespace, the request is split into one or more data transfer instructions and data verification instructions for downstream storage devices. The disk controller is then scheduled to complete the disk I / O operation, and RAID parity calculations are performed. For example, a typical write request for a RAID5 NVMe set usually needs to be split into two data verification instructions (read old data, read old parity), two data transfer instructions (write new data, write new parity), and one RAID parity calculation.

[0071] Response word generation: After completing the data operation of the NVMe standard I / O request, an NVMe response command word is generated, and the processing result of the I / O request is fed back to the CQ queue on the host through the NVMe controller.

[0072] Component 204: Cache unit, which temporarily stores the command words of NVMe standard I / O requests, user data of I / O requests, and command words for downstream storage devices after splitting, for the HBA system software to schedule and process.

[0073] Component 205: Hard disk controller, which converts the data transmission instructions scheduled by the HBA system software into transactions that conform to the interface type of the downstream device, and efficiently completes the data read and write operations of the storage device.

[0074] Component 206: Storage device, storing user data and RAID check data in I / O requests.

[0075] refer to Figure 4 Application scenarios of the data processing method provided in the embodiments of this application Figure 2 It is applied to the processing flow of NVMe namespace creation and I / O operations.

[0076] Step 301: After the HBA starts, the HBA system software performs initialization operations, scans and identifies storage devices, including the number of storage devices, storage device capacity, device interface type and other information, to provide basic data for subsequent RAID level configuration and management.

[0077] Step 302: The HBA's system software creates one or more NVMe sets with different independent RAID levels according to a specific configuration. The capacity information of each NVMe set can vary depending on the specific configuration. The specific configuration can originate from a dynamic configuration file or a pre-defined program. This configuration flexibility allows the HBA to adapt to the needs of different users and provide personalized storage solutions.

[0078] Step 303: The host sends out a command to identify the NVMe set list as defined by the standard protocol through the NVMe management command queue, querying the list of NVMe sets supported by the HBA and their NVMe set attribute entries. The host obtains the RAID level information supported by the HBA through the NVMe set list identification command, providing a basis for subsequent target namespace creation.

[0079] Step 304: The HBA system software returns the list of non-volatile memory standard NVMe sets and NVMe set attribute entries established in step 302. The NVMe set attribute entries contain RAID level information, which indicates the RAID data protection level of the current NVMe set.

[0080] Step 305: The host selects the target NVMe set corresponding to the required RAID level based on application business needs, and creates the target namespace using the NVMe namespace management commands defined by the standard protocol. For applications with high reliability and integrity requirements, a higher data protection level is selected; for applications with lower requirements, a lower data protection level is selected. Typically, RAID data protection levels from low to high are RAID 0, RAID 5, RAID 6, RAID 1, and RAID 10. Various types of tiered protection mechanisms provide users with flexible choices, ensuring a balance between data security and performance.

[0081] Step 306: For the target namespace created for a specific application, the host sends a data processing request to perform data read or write operations. This data processing request includes NVMe standard I / O operation requests.

[0082] Step 307: Upon receiving the data processing request, the HBA system software, based on the RAID level information of the NVMe set corresponding to the target namespace, breaks down the data processing request into one or more data transfer instructions and data verification instructions for the storage device. It then schedules the hard disk controller to complete the I / O operations for the storage device and performs RAID verification calculations through the software. After completing all RAID-related operations, an acknowledgment word is returned via the CQ queue. At this point, the I / O data processing for the target namespace with RAID data protection is complete.

[0083] The exemplary structure of the software modules included in the data processing apparatus 90 provided in the embodiments of this application will be further described below. In some embodiments, such as Figure 5 As shown, the data processing device 90 may include: The acquisition module 901 is used to acquire device information of multiple storage devices connected to the HBA; Configuration module 902 is used to configure at least two non-volatile memory standard NVMe sets corresponding to the storage space of multiple storage devices based on device information; wherein, different NVMe sets correspond to different independent disk redundant array RAID levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels; Receiver module 903 is used to receive the first control command sent by the host; the first control command is used to control the HBA to create the target namespace of the target RAID level; Module 904 is created to create a target namespace for the NVMe assembly based on the first control instruction; the target namespace is generated in the storage space of the storage device. The processing module 905 is used to respond to a data processing request received from the host and perform data processing on the data processing request based on the target namespace.

[0084] In some embodiments, the data processing apparatus further includes a sending module, which is configured to: in response to receiving a query command sent by the host, send an NVMe set attribute entry corresponding to the HBA to the host, so that the host can determine the target NVMe set and generate a corresponding first control command based on the NVMe set attribute entry; the NVMe set attribute entry contains RAID level information corresponding to each NVMe set.

[0085] In some embodiments, the processing module 905 is configured to: obtain RAID level information of the NVMe set corresponding to the target namespace; split the data processing request based on the RAID level information to obtain at least one data transmission instruction and at least one data verification instruction; control the storage device corresponding to the NVMe set based on the data transmission instruction, and perform verification calculation of the RAID level corresponding to the NVMe set based on the data verification instruction to obtain the data processing result.

[0086] In some embodiments, the acquisition module 901 is used to: identify multiple storage devices connected to the HBA during HBA initialization and obtain device information; the device information includes at least the number of storage devices, the storage device capacity, and the device interface type.

[0087] In some embodiments, the configuration module 902 is configured to: obtain set configuration information; the configuration information represents the RAID level and the number of NVMe sets created by the HBA; create multiple NVMe sets based on the configuration information and device information, each NVMe set corresponding to a RAID level; and establish an attribute information table for each NVMe set, the attribute information table including at least the identification information, performance data, capacity information and RAID level information of the NVMe set.

[0088] In some embodiments, the creation module 904 is configured to: parse the first control instruction to obtain the corresponding target RAID level and target NVMe set; allocate the target namespace in the storage space of the storage device based on the NVMe namespace management command; and establish a mapping relationship between the target namespace and the target NVMe set so that the target namespace can perform data processing at the target RAID level.

[0089] In some embodiments, RAID level information is encoded and stored in the setting field of an NVMe set attribute entry.

[0090] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For any technical details not covered in the data processing apparatus provided in this application embodiment, please refer to... Figures 1 to 4 The meaning is understood in accordance with the description of any of the accompanying drawings.

[0091] According to embodiments of this application, this application also provides an electronic device and a non-transitory computer-readable storage medium.

[0092] Figure 5 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0093] like Figure 5 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0094] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0095] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0096] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0097] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0098] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0099] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0100] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0101] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0102] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, Applied to a host bus adapter (HBA), the method includes: Obtain device information for multiple storage devices connected to the HBA; Based on the device information, configure at least two non-volatile memory standard NVMe sets corresponding to the storage space of the multiple storage devices; wherein, different NVMe sets correspond to different independent disk redundant array RAID levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels; Receive the first control command sent by the host; the first control command is used to control the HBA to create the target namespace of the target RAID level; Based on the first control command, a target namespace is created for the NVMe set; the target namespace is generated in the storage space of the storage device. In response to receiving a data processing request from the host, data processing is performed on the data processing request based on the target namespace.

2. The method according to claim 1, characterized in that, Before receiving the first control command sent by the host terminal, the method further includes: In response to receiving a query command from the host, the system sends the NVMe set attribute entry corresponding to the HBA to the host, so that the host can determine the target NVMe set and generate the corresponding first control command based on the NVMe set attribute entry. The NVMe set attribute entries contain RAID level information corresponding to each NVMe set.

3. The method according to claim 1, characterized in that, The data processing of the data processing request based on the target namespace includes: Obtain the RAID level information of the NVMe set corresponding to the target namespace; Based on the RAID level information, the data processing request is split to obtain at least one data transmission instruction and at least one data verification instruction; Based on the data transmission instructions, the storage device corresponding to the NVMe set is controlled, and the verification calculation of the RAID level corresponding to the NVMe set is performed based on the data verification instructions to obtain the data processing result.

4. The method according to claim 1, characterized in that, The process of obtaining device information for multiple storage devices connected to the HBA includes: During HBA initialization, multiple storage devices connected to the HBA are identified to obtain device information; The device information includes at least the number of storage devices, the storage capacity, and the device interface type.

5. The method according to claim 1, characterized in that, The step of configuring at least two NVMe sets corresponding to the storage spaces of the multiple storage devices based on the device information includes: Obtain the configured information; the configuration information represents the RAID level included in the HBA and the number of NVMe sets created; Based on the configuration information and the device information, multiple NVMe sets are created, each NVMe set corresponding to a RAID level; An attribute information table is established for each NVMe set. The attribute information table includes at least the identification information, performance data, capacity information, and RAID level information of the NVMe set.

6. The method according to claim 1, characterized in that, The step of creating a target namespace for the NVMe set based on the first control command includes: Parse the first control command to obtain the corresponding target RAID level and target NVMe set; Based on NVMe namespace management commands, allocate a target namespace in the storage space of the storage device; Establish a mapping relationship between the target namespace and the target NVMe set so that the target namespace can perform data processing at the target RAID level.

7. The method according to claim 2, characterized in that, The method further includes: The RAID level information is encoded and stored in the setting field of the NVMe set attribute entry.

8. A data processing apparatus, characterized in that, include: The acquisition module is used to obtain device information of multiple storage devices connected to the HBA. The configuration module is used to configure at least two non-volatile memory standard NVMe sets corresponding to the storage space of the multiple storage devices based on the device information; wherein, different NVMe sets correspond to different independent disk redundant array RAID levels, and the storage space of the same storage device can be configured as multiple NVMe sets belonging to different RAID levels; The receiving module is used to receive the first control command sent by the host; the first control command is used to control the HBA to create the target namespace of the target RAID level; A creation module is configured to create a target namespace for the NVMe set based on the first control command; the target namespace is generated in the storage space of the storage device. The processing module is configured to respond to a data processing request received from the host and process the data based on the target namespace.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images