Data reading and writing method, system, device, computing device and storage medium

Abstract

The embodiments of this specification provide a data read and write method and system, apparatus, computing device and storage medium, wherein the data read and write method includes: receiving a data read and write request sent by a user state through an asynchronous I / O layer, and storing the data read and write request in a request queue; sending the data read and write request to a driver layer when the asynchronous I / O layer determines that the data read and write request is a pass-through type; parsing the data read and write request through the driver layer, and sending the request information of the data read and write request to a target processing end device for read and write processing. Receiving a data read and write request sent by a user state through an asynchronous I / O layer and sending it to the driver layer after determining that it is a pass-through type reduces context switching and data copying overhead, improves the processing efficiency of I / O operations, parses the request at the driver layer and sends it to the processing end device for read and write processing, optimizes the scheduling of I / O tasks, reduces disk addressing time and I / O latency, and improves overall performance and system stability.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a method for reading and writing data. Background Art

[0002] In recent years, with the development of virtualization technology and high-performance storage devices, Virtual Transport Network (Virtio) has achieved efficient communication between virtual machines and physical devices. It has the characteristics of low latency and high performance and is widely used in device virtualization network systems.

[0003] The read and write operations (I / O operations) of virtual machines are one of the key factors in system performance. At present, in order to realize the communication between virtual machines and physical devices, read and write operations are usually performed through the Virtual File System (VFS) and the Block Layer. These intermediate layers are responsible for managing and scheduling read and write requests to ensure the correct transmission of data.

[0004] However, with the popularization of high-performance computing and big data applications, higher requirements are placed on the efficiency and response speed of read and write operations of virtual machines. The method of managing and scheduling read and write requests through the middle layer has multiple context switches and data copies, and its performance and flexibility can no longer meet current needs. Therefore, a storage access solution with better performance, higher reliability and more flexibility is urgently needed. Summary of the invention

[0005] In view of this, an embodiment of this specification provides a data reading and writing method. One or more embodiments of this specification also relate to a data reading and writing device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0006] An embodiment of the present specification provides a data reading and writing method, which is applied to kernel state, wherein the kernel state includes an asynchronous I / O layer and a driver layer, and the asynchronous I / O layer includes a request queue; the method includes:

[0007] Receive data read and write requests sent by the user state through the asynchronous I / O layer, and store the data read and write requests in the request queue;

[0008] When the asynchronous I / O layer determines that the data read / write request is a pass-through type, the data read / write request is sent to the driver layer;

[0009] The driver layer parses the data read and write request and sends the request information of the data read and write request to the target processing end device for read and write processing.

[0010] The asynchronous I / O layer receives data read and write requests sent from the user state and stores these requests in the request queue. After the asynchronous I / O layer determines that the request is a pass-through type, it directly sends the request to the driver layer, significantly reducing the number of context switches and data copy overhead from the user state to the kernel state, improving the efficiency and response speed of I / O operations, and also reducing the involvement of the middle layer, simplifying the system complexity, reducing the risk of system instability, and improving the reliability and stability of the system. The driver layer parses the request and sends the request information to the processing end device for read and write processing, optimizing the scheduling of I / O requests, reducing disk seek time and I / O latency, and improving the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a process flow chart of a data read and write request processing method in the prior art;

[0012] Figure 2 It is a block data read and write request encapsulation structure schematic diagram in the prior art;

[0013] Figure 3 is an execution flow chart of a data reading and writing system provided by an embodiment of this specification;

[0014] Figure 4 is a flow chart of a data reading and writing method provided by an embodiment of this specification;

[0015] Figure 5 This is a schematic diagram of a target data format requested by a virtio_blk driver layer in the prior art;

[0016] Figure 6 It is a structural schematic diagram of a user-mode specified read / write buffer in the prior art, in which the address information and length information are continuous and discrete;

[0017] Figure 7 It is a schematic diagram of a request information format for a user-mode specified read / write buffer of continuous type and discrete type provided by an embodiment of this specification;

[0018] Figure 8 It is a schematic diagram of the structural framework of a data reading and writing method provided by an embodiment of this specification;

[0019] Fig. 9 This is a physical address format conversion diagram provided by an embodiment of this specification;

[0020] Fig.10 is a processing flow chart of a data reading and writing method provided by an embodiment of this specification;

[0021] Fig.11It is a structural schematic diagram of a data reading and writing device provided by an embodiment of this specification;

[0022] Fig.12 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0023] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0024] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0025] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0026] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0027] First, the terms involved in one or more embodiments of this specification are explained.

[0028] User-space devices refer to devices or applications running in user space, which are executed in the user mode of the operating system. User-space devices usually include various applications, libraries, and services that directly interact with users or process user data, such as common browsers, text editors, media players, etc.

[0029] Kernel-Space Device refers to device drivers or system services that run in kernel space. They are executed in the kernel mode of the operating system. Kernel-Space Device directly manages hardware resources, handles system-level tasks, and provides core system functions, such as hard disk drivers, network drivers, file system drivers, etc.

[0030] virtio_blk: The virtio block device driver (virtio Block Device Driver, abbreviated as virtio-blk) is a block device driver based on the Virtio standard, which is used to efficiently process disk I / O requests in a virtualized environment.

[0031] io_uring: I / O ring queue (Input / Output User Ring, abbreviated as io_uring) is a new asynchronous I / O framework that optimizes I / O operation performance by using ring queues to efficiently transfer information between user mode and kernel mode.

[0032] Polling mechanism: The polling mechanism is a mechanism that actively queries the completion status of I / O requests. It allows user-mode applications to actively check whether the I / O operation is completed instead of waiting for interrupt notifications, which can improve the resource utilization and I / O throughput of the central processor.

[0033] See also Figure 1 to Figure 2 , Figure 1 A process flow chart of a method for processing a data read and write request is shown. Figure 2 A block data read and write request encapsulation structure diagram is shown in FIG. At present, the typical data read and write access process through a virtual block device is as follows: Figure 1 As shown in the figure: the user state program initiates a data read and write request through a system call, and passes the file information, the address and length of the buffer requested by the user state to the kernel state; the virtual file system layer in the kernel state encapsulates the data read and write request, wherein the virtual file system layer includes a file system (such as ext4 file system or XFS file system) for file management, and the virtual file system layer encapsulates the data read and write request structure as shown in the figure: Figure 2As shown: the file address information, starting sector information, read and write size and other information are encapsulated into a block device read and write request. Each block device read and write request points to the next block device read and write request through a block device read and write request pointer. The read and write buffer array is composed of the memory page where the buffer is located, the offset within the page, and the length; after obtaining the encapsulated block device read and write request, the block device read and write request is sent to the block layer for processing. The block layer extracts the data read and write request from the block I / O request list, and encapsulates the device read and write request into a request structure in a specific format, and forms a request structure queue, which is sent to the underlying device through the request structure queue. The block I / O request list is a data structure used to manage and temporarily store block I / O requests; the virtual block device driver can write to the virtual queue after receiving the request structure.

[0034] However, in the above scheme, the data read and write requests sent by the user-mode program need to be sent down through the virtual file system layer, the block layer, and the virtual block device driver layer, and at the same time, they need to be integrated in the block layer to form a request structure queue and other operations. Therefore, the format requirements for the data read and write requests sent by the user-mode program are high, and they must meet the requirements between the various layers in the kernel state before data read and write operations can be performed, that is, more custom requests cannot be supported, and the scalability is poor. At the same time, the processing process of multiple intermediate layers will perform multiple context switches and data copies, which will not only cause additional performance overhead, but also reduce the overall stability of the system.

[0035] In response to the above problems, in this specification, a data reading and writing method is provided. This specification also involves a data reading and writing device, a data reading and writing system, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0036] See also Figure 3 , Figure 3 FIG. 1 shows an execution flow chart of a data reading and writing system provided according to an embodiment of the present specification. Figure 3 As shown, the data reading and writing system 300 includes:

[0037] User state device 302, kernel state device and multiple processing end devices, the kernel state device includes an asynchronous I / O layer 304 and a driver layer 306, the asynchronous I / O layer includes a request ring queue;

[0038] User state device 302, used to send data read and write requests to kernel state devices;

[0039] The asynchronous I / O layer 304 is used to store the data read / write request in the request ring queue, and send the data read / write request to the driver layer when determining that the data read / write request is a pass-through type;

[0040] The driver layer 306 is used to parse the data read and write request and send the request information of the data read and write request to the target processing end device among the multiple processing end devices;

[0041] The target processing end device 308 is used to perform read and write processing based on the request information.

[0042] The data reading and writing system provided in the embodiments of this specification, through a layered architecture and asynchronous processing, stores the data reading and writing requests sent by the user-mode device into the request ring queue via the asynchronous I / O layer, and sends the pass-through type data reading and writing requests directly to the driver layer, thereby reducing the context switching overhead and the number of data copies; after the driver layer parses, the request information corresponding to the data reading and writing request is sent to the target processing end device for processing, thereby improving the throughput and response speed of the data reading and writing processing.

[0043] Corresponding to the above data reading and writing system, this specification provides a data reading and writing method in one embodiment, see Figure 4 , Figure 4 A flow chart of a data reading and writing method provided according to an embodiment of the present specification is shown. The method is applied to a kernel-mode device. The kernel-mode device includes an asynchronous I / O layer and a driver layer. The asynchronous I / O layer includes a request queue. The method includes:

[0044] Step 402: Receive a data read / write request sent by the user state through the asynchronous I / O layer, and store the data read / write request in a request queue.

[0045] User Space is the general term for the memory area where ordinary applications run. Each user space process has its own independent user space. These processes can only access the memory area allocated to them and cannot directly access the memory of other processes or key system data.

[0046] User space devices refer to devices or applications running in user space, which are executed in the user mode of the operating system. User space devices usually include various applications, libraries, and services that directly interact with users or process user data, such as common browsers, text editors, media players, etc.

[0047] Kernel Space is the general term for the operating system kernel and its drivers, etc., which includes all codes and data structures related to core functions such as hardware interaction, process scheduling, and memory management.

[0048] Kernel space devices refer to device drivers or system services that run in kernel space. They are executed in the kernel mode of the operating system. Kernel space devices directly manage hardware resources, handle system-level tasks, and provide core system functions, such as hard disk drivers, network drivers, file system drivers, etc.

[0049] The asynchronous I / O layer is a mechanism that allows applications in user mode to initiate I / O operations in a non-blocking manner. It implements efficient I / O processing by submitting I / O requests to the kernel and notifying user mode applications after the request is completed, such as the Portable Operating System Interface Asynchronous I / O layer (POSIX AIO) and the io_uring layer.

[0050] The driver layer is part of the operating system kernel and is responsible for managing and controlling hardware devices. For storage devices, the driver layer is responsible for processing I / O requests and sending them to the actual hardware devices, such as the virtio block device driver (virtio_blk), virtio graphics device driver (virtio_gpu), etc.

[0051] A request queue is a data structure used in an operating system or application to temporarily store and manage I / O requests. Its function is to collect I / O requests from user-mode applications and pass these requests to the corresponding driver for processing in sequence. Request queues can include linear queues, ring queues, and other types.

[0052] The Submission Queue (SQ) is a ring buffer, which is a fixed-size data structure with two ends connected to form a ring structure. The main function of the ring buffer is to efficiently manage data flow by reusing the used space, avoiding the frequent memory allocation and release operations in traditional linear buffers.

[0053] Applications in user mode can put I / O requests into the request ring queue. Kernel mode will take out the I / O requests from the request ring queue and process them.

[0054] Data read and write requests refer to I / O operation requests initiated by applications or services running in user space to the kernel. These requests usually involve read or write operations on resources such as files, networks, and storage devices. User-mode applications can submit these requests through system calls or asynchronous I / O mechanisms. Data read and write requests include file read and write requests, network read and write requests, and storage device read and write requests.

[0055] The kernel state includes an asynchronous I / O layer and a driver layer. The asynchronous I / O layer is located on the side of the kernel state close to the user state. The request ring queue is located in the asynchronous I / O layer. The asynchronous I / O layer receives data read and write requests sent by the user state through the request ring queue, and sends the data read and write requests to the driver layer through the request ring queue; the driver layer is between the asynchronous I / O layer and the processing end. The driver layer receives data read and write requests sent by the asynchronous I / O layer, and sends the data read and write requests to the processing end.

[0056] Specifically, the user state sends a data read and write request to the kernel state, and the data read and write request includes the data read and write operations required by the application running in the user state; after receiving the data read and write request, the kernel state stores the data read and write request in the request ring queue in the asynchronous I / O layer.

[0057] Optionally, the data read and write request can be stored in the request ring queue in the form of a submission queue entry (SQE). A submission queue entry is an entry in the request ring queue and can represent a specific data read and write request. Each submission queue entry contains all necessary information of the corresponding data read and write request, such as operation type, operation priority, read and write address starting position, buffer address and length, etc.

[0058] Exemplarily, the application of the data reading and writing method in the data storage scenario is used as an example to further illustrate. There is a log recording application A running in the user state, and it is necessary to write 512 bytes of log data into a file named logfile.txt in the user state file management system. Then, the kernel state starts writing from the 1048576th (i.e., 2^20) sector, and then generates a data write request X and sends it from the user state to the kernel state; the kernel state receives the data write request X, initializes the SQ queue in the io_uring layer, and stores the data write request X in the SQ in the form of SQE. The information in the SQE includes: operation type [W] (W stands for write operation WRITE), operation priority [1] (the priority can be set to 5 levels from high to low, corresponding to 1-5), read and write start position [1048576] (refers to writing from the 1048576th sector), buffer address [0x7f8a4c001000], buffer length [512 bytes], etc.

[0059] In the steps of the embodiments of this specification, non-blocking asynchronous I / O operations are implemented by passing data read and write requests generated in user mode to kernel mode and managing and processing them through request queues, thereby providing a data basis for subsequent processing of I / O requests.

[0060] Step 404: When the asynchronous I / O layer determines that the data read / write request is of a pass-through type, the data read / write request is sent to the driver layer.

[0061] A pass-through data read / write request refers to a data read / write request that can be directly passed to the driver layer. The data read / write request does not need to be processed by the kernel's intermediate layer (such as the Block layer, VFS layer, etc.), and is directly communicated between the user-state application and the kernel-state driver.

[0062] When the asynchronous I / O layer confirms that the data read / write request is of the pass-through type, the data read / write request can be sent directly to the driver layer. Specifically, the method for the asynchronous I / O layer to confirm that the data read / write request is of the pass-through type is as follows: judging by determining whether the operator (opcode) included in the data read / write request for specifying the operation type of the read / write request contains a specific operation handle. If the operator of the data read / write request includes a specific operation handle (e.g., uring_cmd), then the operation type of the data read / write request is judged to be of the pass-through type; if the operator of the data read / write request only includes general operation handles (e.g., write, read), then the operation type of the data read / write request is judged not to be of the pass-through type.

[0063] For direct data read and write requests, if the operator contains a specific operation handle (such as .uring_cmd), it means that the hardware device (such as some advanced storage devices or network interface cards) or driver device (such as the virtio_blk driver under the io_uring framework) of the data read and write system supports processing direct data read and write requests, so the data read and write request can be directly sent to the driver layer for processing. By bypassing the processing of the middle layer, the data read and write system reduces context switching, reduces latency, and improves concurrency.

[0064] For non-through data read and write requests, when the operators only include general operation handles (such as write and read), it means that the call of the data read and write system follows the standard path and does not support processing through data read and write requests. It can only support general processing operations (such as writing data write or reading data read) to ensure the compatibility and stability of the system.

[0065] Exemplarily, the application of the data reading and writing method in the data storage scenario is used as an example to further illustrate. Taking the above-mentioned log recording application A as an example: the asynchronous I / O layer io_uring layer parses the SQE corresponding to the data write request X in the SQ queue, and obtains the file operator [pass-through], then determines that the data write request X is a pass-through type, and directly sends the data write request X to the driver layer.

[0066] In the steps of the embodiments of this specification, by passing the direct type of data read and write operations directly to the driver layer through the asynchronous I / O layer, the processing overhead of the kernel middle layer is reduced, thereby reducing data copying and context switching, reducing the latency of I / O operations, and improving the overall I / O throughput, while providing a data basis for the subsequent driver layer to parse data read and write requests.

[0067] Step 406: The driver layer parses the data read / write request and sends the request information of the data read / write request to the target processing end device for read / write processing.

[0068] The processing end device refers to the physical or virtual storage device that actually performs data read and write operations. This device can be a variety of types of storage media, depending on the system configuration and application scenario, such as hard disk drives (HDDs), solid-state drives (SSDs), network storage devices, etc. The target processing end device refers to the processing end device that processes the data read and write request.

[0069] Data read and write requests may include information such as operation type, operation priority, read and write address starting position, buffer address and length. After the driver layer parses the data read and write request to obtain the above information, it can convert it based on this information to obtain request information, and send it to the target processing end device in the form of request information for read and write processing.

[0070] Exemplarily, the application of the data reading and writing method in the data storage scenario is used as an example to further illustrate. Taking the above-mentioned log recording application A as an example: the driver layer virtio_blk layer parses the SQE of the data write request X, obtains the operation type [W], operation priority [1], read and write start position [1048576], buffer address [0x7f8a4c001000], buffer length [512 bytes] and other information, converts the buffer address [0x7f8a4c001000] based on this information, converts the buffer address [0x7f8a4c001000] into the actual physical address [0x123456789000], obtains the request information X corresponding to the data write request X, and sends it to the target processing end device SSD.

[0071] In the embodiments of the present specification, data read and write requests sent from user mode are received and stored in a request queue. After the asynchronous I / O layer determines that the request is a pass-through type, the request is directly sent to the driver layer, which significantly reduces the number of context switches from user mode to kernel mode and the data copy overhead, improves the efficiency and response speed of I / O operations, and also reduces the involvement of the middle layer, simplifies the system complexity, reduces the risk of system instability, and improves the reliability and stability of the system. The request is parsed at the driver layer and the request information is sent to the processing end device for read and write processing. The multi-queue architecture is used to allow multiple I / O requests to be processed in parallel, further improving the processing efficiency of I / O operations. At the same time, the scheduling of I / O requests is optimized, the disk seek time and I / O delay are reduced, and the overall performance is improved.

[0072] In an optional embodiment of the present specification, receiving a data read / write request sent by a user mode and storing the data read / write request in a request queue includes:

[0073] Receive a data read / write request sent by a user state, wherein the data read / write request is encapsulated with request header information;

[0074] Store data read and write requests in the request queue.

[0075] The request header information is a part of each data read / write request, which contains metadata and control information about the data read / write request, reflects the specific attributes and control parameters of the data read / write request, and is used to guide the kernel-mode device on how to handle the data read / write request, ensuring correct parsing, execution, and feedback of the results. The request header information includes the read / write type type, read / write priority ioprio, and read / write starting position sector. The request header information can be configured by the user in user mode, and can also be automatically generated by an application running in user mode. The embodiments of this specification do not make specific limitations on this.

[0076] After receiving the data read / write request sent by the user mode, the data read / write request including the request header information may be written into the request queue.

[0077] Exemplarily, the application of the data reading and writing method in the data storage scenario is used as an example to further illustrate. Taking the above-mentioned log recording application A as an example, the log recording application runs in the user state and needs to write 512 bytes of log data to a file named logfile.txt in the user state file management system. Then, the corresponding kernel state starts writing from the 1048576th sector, and a data write request X is generated and sent from the user state to the kernel state device; the kernel state device receives the data write request X, and the data write request X includes the request header information: read-write type type[W], read-write priority ioprio[1] and starting position sector[1048576], then the data write request X can be stored in the SQ in the form of SQE.

[0078] In the embodiments of the present specification, data read and write requests are encapsulated into a structure including request header information and stored in a request queue. Since the request header information includes detailed metadata and control information (such as read and write type type, priority ioprio and starting position sector) for each data read and write request, the kernel-mode device can directly parse the request without additional queries or calculations, thereby reducing context switches and additional system calls caused by lack of necessary information, thereby achieving efficient, concurrent and flexible I / O operations, which not only reduces the overhead of system calls, but also improves I / O performance and maintainability of applications.

[0079] In an optional embodiment of the present specification, the driver layer parses the data read and write request, and sends the request information of the data read and write request to the target processing end device for read and write processing, including:

[0080] The driver layer parses the data read and write request to obtain the request header information;

[0081] Determine the address information and length information of the data buffer requested in user mode;

[0082] Based on the request header information, address information and length information, obtain request information of the data read and write request;

[0083] The request information is sent to the target processing end device for reading and writing processing.

[0084] Data buffer is a memory area in a computer system used to temporarily store data. The data buffer requested by the user state refers to a memory area allocated by the user state application for read or write operations. This memory area is located in the user space memory and is managed by the user state application. Data buffers can be divided into input buffers, output buffers, double buffers, etc. according to the read and write types.

[0085] The address information of the data buffer refers to the starting address of the data buffer in the memory. This address is usually a pointer pointing to the first byte of the buffer. The length information of the data buffer refers to the size of the data buffer, that is, the amount of data that the data buffer can accommodate. It is usually in bytes. The length information of the data buffer ensures that the read and write operations do not exceed the boundaries of the buffer, thereby avoiding memory access errors. For example, if logging application A needs to write 512 bytes of log data to the storage system, then logging application A will apply for a data buffer with a length of 512 bytes. The starting address of the buffer is [0x7f8a4c001000].

[0086] When the driver layer sends the read / write request to the target processing device for read / write processing, it needs to parse the read / write request and convert it into the request information in the target data format, and the target data format is usually determined by the driver on the driver layer of the device. For example, the virtio_blk block device driver needs to use the request information in the virtio command format because it needs to communicate with the block device in the virtualization environment compatible with the virtio standard.

[0087] See also Figure 5 , Figure 5 A schematic diagram of the target data format requested by the virtio_blk driver layer is shown, such as Figure 5 As shown: The target data format includes three parts: request header information, address-related information of the read / write buffer, and return value field information. In traditional file system and block layer access, the user-mode access request will be converted into information in this format.

[0088] By converting the request header information, address information and length information included in the data read and write request, the request information in the target format can be obtained, and the request information is sent to the target processing end device for read and write processing.

[0089] Exemplarily, the application of the data reading and writing method in the data storage scenario is used as an example to further illustrate. Taking the above-mentioned logging application A as an example, the data writing request X is converted into request information X through the virtio_blk driver layer, and the address information and length information of the data buffer are determined according to the data buffer applied for by the logging application A, wherein the address information is [0x7f8a4c001000], the length information is [512 bytes], and the request header information is parsed to obtain the read / write type type [W], read / write priority ioprio [1] and starting position sector [0]. Based on the address information [0x7f8a4c001000], the length information [512 bytes], the read / write type type [W], the read / write priority ioprio [1] and the starting position sector [0] included in the request header information, the data writing request X is obtained in accordance with the request information X of the vitrio command format; and the request information X is sent to the target processing end device SSD for read / write processing.

[0090] In the embodiments of the present specification, the data read and write requests sent by the user state are parsed and converted into the specific command format required by the driver program in the kernel state device through the driver layer. Combined with the address information and length information of the data buffer requested by the user state, the data read and write requests can be directly sent to the target processing end device for processing, thereby improving the performance and efficiency of I / O operations, reducing system overhead, and ensuring accurate data transmission.

[0091] In an optional embodiment of the present specification, based on the request header information, address information and length information, request information of the data read and write request is obtained, including: converting the address information of the data buffer into physical address information; encapsulating the request header information, physical address information and length information to obtain the request information of the data read and write request.

[0092] Physical address information refers to the address in the actual memory, which is the address directly accessed by the hardware device. The data buffer address requested in user mode is usually a virtual address, that is, the address seen by the user mode application. Each user mode application has its own virtual address space, which is allocated by the operating system.

[0093] In data read and write operations, since communication is required between the application in user mode and the hardware device in kernel mode, and the kernel mode device needs to read the actual physical address for data transmission, the address information of the data buffer needs to be converted into physical address information. The length information of the data buffer is determined according to the data length corresponding to the read and write data request, so the length information remains unchanged in the virtual address and physical address, so there is no need to convert the length information.

[0094] The virtual address can be converted into a physical address based on the mapping relationship between the virtual address and the physical address, for example, through a page table, a memory-mapped file (mmap for short), etc. The specific method used for conversion can be determined according to the actual application scenario or demand, and the embodiments of this specification do not make specific limitations on this.

[0095] After obtaining the converted physical address information, the request header information, physical address information and length information can be encapsulated to obtain the request information of the data read and write request. The request information meets the target format requirements of the driver layer and can be directly read by the target processing end device, so that it can be sent to the target processing end device for read and write processing.

[0096] Exemplarily, the application of the data reading and writing method in the data storage scenario is used as an example to further illustrate. Taking the above-mentioned log recording application A as an example, after obtaining the address information [0x7f8a4c001000], the length information [512 bytes], the request header information including the read-write type type [W], the read-write priority ioprio [1] and the starting position sector [0], the address information [0x7f8a4c001000] can be converted to obtain the physical address information [0x123456789000]; the read-write type type [W], the read-write priority ioprio [1] and the starting position sector [0], the physical address information [0x123456789000] and the length information [512 bytes] included in the request header information are encapsulated to obtain the request information X corresponding to the data write request X.

[0097] In the embodiments of the present specification, by converting the address information of the data buffer requested in the user state into the actual physical address information, and combining the request header information and the length information to encapsulate it into the request information that conforms to the target format of the driver layer, efficient data transmission is achieved, while reducing the communication overhead between the user state and the kernel state, ensuring that the hardware device can directly access the correct physical address for read and write operations, thereby improving the efficiency and performance of I / O operations, while ensuring the accuracy and integrity of the data.

[0098] In an optional embodiment of the present specification, the request information further includes a flag bit, and the flag bit is used to mark the type of the data read and write request; encapsulating the request header information, the physical address information, and the length information to obtain the request information of the data read and write request, including: when the data read and write request is a single request, encapsulating the request header information, the physical address information and the length information of the data buffer, and setting the flag bit to a non-vector read and write request type to obtain the request information of the data read and write request;

[0099] When the data read / write request is a request sequence, the request header information, the physical address information and the length information of each data buffer are encapsulated, and the flag is set to the vector class read / write request type to obtain the request information of the data read / write request.

[0100] Data read and write requests sent in user mode can be divided into non-vector read and write requests and vector read and write requests according to the number of requests included in the data read and write requests.

[0101] Specifically, the data read / write request may perform read / write processing on data of a single request, or may perform read / write processing on data of multiple requests arranged in sequence.

[0102] When reading and writing data of a single request, the data buffer address requested in user mode is continuous, that is, the read / write address of the data for the read / write operation is a continuous memory area. This type of request is suitable for situations where the data is already concentrated in a continuous memory.

[0103] When reading and writing multiple data arranged in sequence, the buffer address requested in user mode is discrete, that is, the multiple data to be read and written are distributed in multiple discontinuous memory areas, each of which is called a "vector". This type of request is usually used in situations where data needs to be read or written from multiple different locations.

[0104] See also Figure 6 , Figure 6 A schematic diagram of the structure of the address information and length information of the read and write buffer specified by the user state as continuous and discrete types is shown, such as Figure 6 As shown:

[0105] When the read-write buffer specified in the user state is continuous, the address-related information of the read-write buffer includes address information and length information, wherein the address information is the data storage address to be written to the file or the storage address of the data content read from the file, and the length information is the length of the data to be written / read; when the read-write buffer specified in the user state is discrete, the address-related information of the read-write buffer includes a starting address pointer and length information, wherein the starting address pointer points to the starting address and is represented by a read-write buffer array, and the read-write buffer array includes multiple discrete buffer addresses and length information; the length information is used to indicate the number of buffers.

[0106] When it is determined that the data read / write request is a non-vector type request, the continuous address information corresponding to the data buffer can be converted into physical address information, and the physical address information, length information and request header information can be encapsulated. At the same time, the flag bit is set to the non-vector type read / write request type to obtain the request information of the data read / write request.

[0107] When it is determined that the data read and write request is a vector type request, the discrete address information of multiple data buffers represented by vectors can be converted into multiple physical address information, and the physical address information and corresponding length information of each data buffer can be spliced ​​to obtain the physical address information and length information represented by a vector, and the physical address information, length information and request header information can be encapsulated in vector representation, and the flag bit can be set to the vector type of read and write request to obtain the request information of the data read and write request.

[0108] That is, see Figure 7 , Figure 7 FIG. 1 shows a schematic diagram of a request information format for a user-mode specified read / write buffer of continuous type and discrete type according to an embodiment of the present specification, such as Figure 7 As shown:

[0109] When the read / write buffer specified in the user state is continuous, the request information format includes operation request header information, address information, length information and flag bits, wherein the request header information includes operation type, operation priority, and starting address information, the address information is the data storage address to be written to the file or the storage address of the data content read from the file, the length information is the length of the data to be written / read, and the flag bit is 0: indicating non-vector reading and writing; when the read / write buffer specified in the user state is discrete, the request information format includes operation request header information, address information, length information and flag bits, wherein the request header information includes operation type, operation priority, and starting address information, the address information points to the starting address, and is represented by a read / write buffer array, the read / write buffer array includes multiple discrete buffer addresses and length information, the length information is used to indicate the number of buffers, and the flag bit is 1: indicating vector reading and writing.

[0110] For example, in the case where the read / write buffer specified in the user state is continuous, taking the above-mentioned logging application A as an example, it needs to write log data to a disk file, and these log data are stored in a continuous memory area in the data buffer: Area: 0x7f8a4c001000-0x7f8a4c001fff (4096 bytes)

[0111] In this case, logging application A can use non-vectored class requests to write the data in this contiguous memory region to a disk file.

[0112] For example, in the case where the read / write buffer specified in the user state is continuous, taking the above-mentioned logging application A as an example, it needs to write log data to a disk file, and the log data is distributed in three different memory areas:

[0113] Region 1: 0x7f8a4c001000-0x7f8a4c001200 (512 bytes)

[0114] Region 2: 0x7f8a4c002000-0x7f8a4c002200 (512 bytes)

[0115] Region 3: 0x7f8a4c003000-0x7f8a4c003200 (512 bytes)

[0116] In this case, logging application A can use vector class requests to write data of these three areas to the disk file in one data read and write request.

[0117] When it is determined that the data read and write request is a vector type request, the discrete address information of multiple data buffers represented by vectors can be converted into multiple physical address information, and the physical address information and corresponding length information of each data buffer can be spliced ​​to obtain the physical address information and length information represented by a vector, and the physical address information, length information and request header information can be encapsulated in vector representation, and the flag bit can be set to the vector type of read and write request to obtain the request information of the data read and write request.

[0118] In the embodiments of the present specification, efficient data transmission is achieved by distinguishing between non-vector and vector data read and write requests and setting corresponding flag bits in the request information; for a single-request data read and write request, non-vector request information is used to process the data in the continuous memory area; for multiple-request data read and write requests, vector request information is used to process the data in the discrete memory area, thereby reducing the number of system calls, improving the efficiency and performance of I / O operations, simplifying data processing logic, and ensuring the accuracy and integrity of data transmission.

[0119] In an optional embodiment of the present specification, after obtaining the request information of the data read / write request based on the request header information, the address information and the length information, the method further includes: storing the request information in a core request queue based on the correspondence between the data read / write request and the request information; allocating the request information from the core request queue to a hardware request queue corresponding to each processing end device based on the device information of each processing end device;

[0120] Sending the request information to the target processing end device for read and write processing includes: sending the request information to the target processing end device for read and write processing through a hardware request queue corresponding to the target processing end device.

[0121] The core request queue (Context, CTX for short) is a centralized request queue used to store and manage all submitted data read and write requests and their corresponding request information. All data read and write requests are first stored in CTX to ensure the orderliness and traceability of the requests.

[0122] Optionally, CTX can be bound to a central processing unit (CPU), ensuring that data read and write requests can be executed on the same CPU, reducing cross-core communication overhead and cache inconsistency issues.

[0123] The Hardware Context (HCTX) is a specific hardware request queue corresponding to each processing end device (such as SSD, network interface device, etc.), which is used to directly interact with the hardware and perform actual read and write operations. The request information in the HCTX can be directly sent to the target processing end device for read and write processing, reducing the overhead of the middle layer; at the same time, each HCTX can store multiple request information, and multiple HCTXs can work in parallel, further improving the parallel processing capability and throughput of the system.

[0124] The device information of each processing end device includes the device type, current load, performance parameters and other information of each processing end device. When CTX allocates request information, it can comprehensively consider the needs of the request information and the device information to make the allocation. For example, if a request information is a request information for data writing and has a higher priority, CTX will allocate the request information to the HCTX corresponding to the processing end device whose device type is a storage device, whose current load is low, and whose performance parameters are high, and the processing end device will process the request information.

[0125] After the driver layer obtains the request information corresponding to the data read and write request, the request information can be stored in the core request queue. The core request queue can manage multiple hardware request queues. According to the device information of each processing end device, it can be determined which processing end device is currently suitable for processing the request information, that is, the request information can be assigned to the hardware request queue corresponding to the processing end device, and through the hardware request queue, all request information included in the queue can be sent to the corresponding processing end device for read and write processing.

[0126] Exemplarily, the application of the data reading and writing method in the scenario of data storage is taken as an example to further illustrate. Take the above-mentioned log recording application A as an example. The data write request X sent by the log recording application A corresponds to the request information X, and the request information X is stored in the core request queue bound to the central processor. The request information X is a data write type request information with a high priority. The core request queue will confirm that the device type of the processing end device SSD-1 is a storage device, the current load of the device is low, and the device performance parameters are high according to the request information of each processing end device; then the request information X is assigned to the hardware request queue corresponding to the processing end device SSD-1, and the request information X is sent to the processing end device SSD-1 through the hardware request queue to complete the data write processing.

[0127] In the embodiments of this specification, by combining the centralized management of core request queues and specific hardware request queues, as well as an intelligent allocation strategy based on device information, cross-core communication overhead and cache inconsistency problems are reduced, efficient, flexible and optimized data read and write request processing is achieved, and the system's concurrent processing capabilities, throughput and overall performance are significantly improved.

[0128] In an optional embodiment of the present specification, the asynchronous I / O layer determines that a data read / write request is a pass-through type, including: reading an operator of the data read / write request; and determining that the data read / write request is a pass-through type when the operator of the data read / write request includes a specific handle.

[0129] The operator (Operation Code, abbreviated as opcode) of a data read or write request refers to an operation code or handle used to specify a specific file operation type. It can be a function name, macro definition, or other form of identifier used to tell the operating system or driver the specific operation to be performed. Depending on different usage scenarios or purposes, operators can have multiple types, such as: read operator (Read), write operator (Write), file operator, and special operator.

[0130] A handle is an identifier used to reference a resource or object in the system. Its function is to allow applications to indirectly access these resources without directly managing the underlying implementation details. Depending on different usage scenarios or purposes, handles can be of various types, such as device handles, event handles, special handles, etc.

[0131] The specific handle may be a special handle used to identify a resource with a specific attribute or function. In the embodiment of this specification, the specific handle is a ".uring_cmd" handle.

[0132] The .uring_cmd handle is a specific handle that is specifically used to identify passthrough type commands, which allows user mode applications to send commands directly to the device driver through the io_uring framework without going through the common block layer or other intermediate layers.

[0133] Optionally, the specific handle of the data read / write request may be provided by a character device interface that sends the data read / write request.

[0134] The character device interface refers to the interface used in the operating system to communicate with character devices. Character devices are usually read and written in the form of byte streams. The character device interface provides functions and methods so that user-mode applications can interact with these devices.

[0135] Specifically, the character device interface can be added when the driver layer is initialized. The user can provide a user-mode device (such as a character device) to call the character device interface and directly send data read and write requests to the driver layer through the asynchronous I / O layer. In this case, the data read and write requests sent through the character device interface will be added with a specific handle (such as) to identify the data read and write request as a pass-through type.

[0136] In the embodiments of the present specification, the asynchronous I / O layer determines whether the data read / write request is of a pass-through type by identifying whether the operator of the data read / write request contains a specific handle, thereby accurately and efficiently completing the determination of the data read / write request type, and enabling user-mode applications to communicate efficiently directly with device drivers, bypassing the common block layer and other intermediate layers, thereby greatly reducing the latency of I / O operations and system resource consumption, and improving the overall performance and response speed of the system.

[0137] In an optional embodiment of the present specification, the asynchronous I / O layer also includes a result queue; after the driver layer parses the data read and write request, and sends the request information of the data read and write request to the processing end device for read and write processing, it also includes: receiving the data read and write results fed back by the target processing end device; storing the data read and write results in the result queue of the asynchronous I / O layer through the driver layer, and the data read and write results in the result queue are used to feedback to the user state.

[0138] The Completion Queue is a data structure used to store the results of completed I / O requests. After the I / O request is processed, it stores the processing results (such as read data, write status, etc.) from the operating system or driver so that other applications can obtain these results.

[0139] The Completion Ring Queue (CQ) is used to store the result information of completed data read and write operations, which is ultimately used to provide feedback to user-mode applications.

[0140] Optionally, the data read and write results corresponding to the data read and write request can be stored in the CQ in the form of a completion event entry (CompletionQueue Entry, CQE for short). CQE is an entry in the result ring queue, indicating the result information after a specific I / O operation is completed. Each CQE contains the processing result data about the data read and write operation, helping the user-mode application to understand the execution status of the request.

[0141] Data read and write results refer to the information fed back to the driver layer by the processing end device (such as a hardware storage device or a virtualized device) after completing the data read and write request submitted by the user state. These results contain specific information about the operation status and returned data, and are used to inform the user state application of the execution status of the request. Data read and write results can include: Operation Status, Error Code, Returned Data and other information. Specifically:

[0142] Operation status: Success: indicates that the data read and write operation is completed successfully; Failure: indicates that the data read and write operation is not completed successfully, and may be accompanied by a specific error code or reason.

[0143] Error code: If the operation status is failed, an error code is provided to help diagnose the problem. Common error codes include: EIO: input / output error; ENOSPC: insufficient disk space; ETIMEDOUT: operation timed out; EACCES: insufficient permissions; EINVAL: invalid parameter.

[0144] Return data: For data read operations, the target data read from the target processing end device is returned; for data write operations, confirmation information that the target data has been successfully written to the target physical address is returned.

[0145] After receiving the data read and write results returned by the target processing end device, the driver layer will write the data read and write results into the result ring queue of the asynchronous I / O layer, and feedback to the user state through the result ring queue.

[0146] Optionally, for a data write operation, the feedback information includes: an operation status and confirmation information; for a data read operation, the feedback information includes: an operation status and target data.

[0147] Exemplarily, the application of the data reading and writing method in the scenario of data storage is taken as an example to further illustrate. Take the above-mentioned logging application A as an example. When the logging program A writes 512 bytes of log data to a file named logfile.txt in the user-mode file management system, the corresponding kernel mode starts writing from the 1048576th sector, and generates a data write request X and corresponding request information X. After the virtio_blk driver layer sends the request information X to the target processing end device SSD, the target processing end device obtains the request header information including the read / write type type[W], read / write priority ioprio[1] and starting position sector[1048576] by reading the request information X. Physically The virtio_blk driver layer receives the data read / write result X and writes it to the CQ of the io_uring layer in the form of CQE. Then, the logging application A can receive the data read / write result X of the data write request X through CQ.

[0148] Exemplarily, the application of the data reading and writing method in the scenario of data reading is taken as an example to further illustrate. Take the above-mentioned log recording application A as an example. The user reads the historical log data Y of size 4096 bytes in the file named history_logfile.txt in the user-state file management system through the log recording application A running in the user state. The corresponding kernel state reads from the 2097152nd (i.e. 2^21) sector, then the log recording application A sends a data read request Y and applies for a data buffer with a length of [4096 bytes]; after the vitrio_blk driver layer converts the data read request Y into the corresponding request information Y, it sends the request information Y to the target processing end device SSD; The target processing device SSD reads the historical data Y in the corresponding sector 2097152 according to the physical address information included in the request information Y, and returns the data read and write result Y: operation status [Success], return data [historical log data Y]; after receiving the data read and write result Y, the virtio_blk driver layer writes the data read and write result Y into the CQ of the io_uring layer in the form of CQE, and writes the historical data Y into the requested data buffer, then the logging application A can receive the data read and write result Y about the data read request Y through CQ.

[0149] In the embodiments of this specification, by introducing the result ring queue (CQ) and completion event entry (CQE) in the asynchronous I / O layer, an efficient and reliable data read and write operation feedback mechanism is implemented, ensuring that each data read and write request can obtain accurate result feedback, simplifying the management and processing logic of the operation results by the user-mode application, thereby significantly improving the system performance, reliability and user experience.

[0150] In an optional embodiment of the present specification, before the data read and write result is stored in the result queue of the asynchronous I / O layer via the driver layer, the method further includes: receiving a result query request sent by the user state; determining completed target request information based on the result query request;

[0151] Storing the data read and write results into the result queue of the asynchronous I / O layer via the driver layer includes: storing the data read and write results corresponding to the target request information into the result queue of the asynchronous I / O layer via the driver layer.

[0152] After sending a data read / write request, the user state can actively send a result query request to check whether the data read / write request has been completed, instead of waiting for an interrupt to arrive. That is, the user state actively sending a result query request can be regarded as an active polling mechanism, through which the user state application can choose when to query the status of which requests according to actual needs, instead of relying on fixed interrupts or timers, so that it can know whether certain key operations are completed as soon as possible, avoiding the extra waiting time consumed by waiting for interrupts, and improving the processing efficiency of data read / write tasks.

[0153] Optionally, active polling is performed through the poll mechanism of the io_uring layer. The poll mechanism allows user-mode applications to actively query whether there are completed data read and write operations from the result queue by calling the polling interface of io_uring.

[0154] Optionally, you can poll the request information included in virtio_blk to query whether the data read and write operation is completed. The request information will be queued in the virtualization queue description table (vring descriptiontable) in the virtualization queue (virtqueue) for processing. Each virtualization queue contains multiple descriptor information (descriptor), and each descriptor points to a memory area for storing request or response data. Descriptors are connected in a linked list to form a descriptor chain. By querying the descriptor chain in the virtqueue to find the status field corresponding to the request information, it is determined whether the request information is completed, that is, the operation status of the request information is determined according to the status field. [Success] or [Failure]. The status field is used to indicate the completion status or result of the I / O request, which is updated by the host (Host) and notified to the virtual machine (Guest).

[0155] Exemplarily, the application of the data reading and writing method in the scenario of data writing and reading is taken as an example to further illustrate. Take the above-mentioned logging application A as an example. Logging application A needs to: write log data X of size 512 bytes to a file named logfile.txt in the storage system, read historical log data Y of size 4096 bytes from a file named history_logfile.txt in the storage system, and write configuration update data Z of size 2048 bytes to a file named config_update.txt in the storage system; for the above three data reading and writing tasks, logging application A sends three data reading and writing requests through the io_uring layer: data reading and writing request X, data reading and writing request Y, data reading and writing request Z to the virtio_blk driver layer; the virtio_blk driver layer converts data reading and writing request X, data reading and writing request Y, data reading and writing request Z to obtain corresponding request information X, request information Y, and request information Z. io_uring layer.

[0156] In the embodiments of this specification, the active polling mechanism is implemented by sending a result query request from the user state to query whether the data reading and writing results are completed, so that the user state application can flexibly and timely query the completion status of the data reading and writing requests, avoiding the delay caused by relying on interrupts, thereby significantly improving the processing efficiency of data reading and writing tasks and the overall performance of the system.

[0157] The following combination Figures 8 to 10 , taking the application of the data reading and writing method provided in this specification in the vitrio_blk driver under the io_uring framework as an example, the data reading and writing method is further explained. Among them, Figure 8 A schematic diagram showing a structural framework of a data reading and writing method provided by an embodiment of this specification is shown. Fig. 9 A schematic diagram of physical address format conversion provided by an embodiment of this specification is shown. Fig.10 A processing flow chart of a data reading and writing method provided by an embodiment of this specification is shown.

[0158] like Figure 8 As shown in the figure: the application running on the user-mode device sends a data read and write request through the character device, the kernel-mode device receives the data read and write request sent by the user-mode device, and stores the data read and write request in the request ring queue; the io_uring layer detects the data read and write request in the request ring queue, and when it detects that it includes the .uring_cmd handle, it determines that the type of the data read and write request is a pass-through type, and sends the data read and write request to the virtio_blk driver layer in the form of a request queue entry (SQE); the virtio_blk driver layer constructs request information that conforms to the target format for the data read and write request, and converts the address information included in the data read and write request to obtain the physical address information, and converts the physical address information and the data read and write request included in the data read and write request. The length information and the request header information are encapsulated together into the request information; the request information is stored in the core request queue, and based on the device information of each processing end device, the request information is allocated to the hardware request queue corresponding to the target processing end device, and the request information encapsulated as the request information structure is sent down through the hardware request queue to the target processing end device, and the target processing end device manages the request information through the virtualization queue description table in the virtual queue; the target processing end device completes the read and write operations after receiving the request information, and returns the data read and write results; the result query request sent by the user state device is received, the completed target request information is confirmed, the data read and write results corresponding to the completed target request information are stored in the result ring queue, and the data read and write results are fed back to the user state device through the result ring queue.

[0159] like Fig. 9As shown: Since the virtio-blk driver complies with the virtio framework. The virtio framework has a general descriptor convention, namely the virtual queue descriptor structure. Each virtio device has one or more virtual queue description tables for storing descriptor information, which is used to implement information exchange between the driver layer (such as the virtio-blk driver) and each processing end device. Among them, the virtual queue descriptor structure includes the address, length, operation information, index and other information corresponding to the read and write operations in the request information, among which the address and length information have been described in detail in the previous text and will not be repeated here; the operation information is the operation mode of the data read and write operation included in the request information, for example, R represents reading data (Read), W represents writing data (Write), and NXT represents executing the next request information (Next); the index information is the pointing information after the current request information is completed, pointing to the request information identified by the next sequence number. Specifically, the first line of the virtual queue descriptor structure includes the virtual block device request header information of the request information, the last line includes the status field of the request information, and the middle line includes the address information corresponding to each data read and write operation. By converting the address information corresponding to each data read and write operation in the virtual queue description table, address-related information in a data format that meets the requirements of the virtio framework is obtained.

[0160] correspond Figure 8 The structural framework diagram of the data reading and writing method is as follows: Fig.10 As shown, the data reading and writing method includes the following steps:

[0161] Step 1002: receiving a data read / write request sent by the user state through the asynchronous I / O layer, and storing the data read / write request in a request ring queue;

[0162] Step 1004: the io_uring layer detects the data read and write request in the request ring queue, and when a .uring_cmd handle is detected, determines that the type of the data read and write request is a pass-through type, and sends the data read and write request to the virtio_blk driver layer;

[0163] Step 1006: the virtio_blk driver layer constructs a request message that conforms to the target format for the data read / write request, converts the address information included in the data read / write request to obtain the physical address information, encapsulates the physical address information together with the length information and the request header information included in the data read / write request into the request message, and specifies the type of the flag bit;

[0164] Step 1008: storing the request information in the core request queue, and based on the device information of each processing end device, allocating the request information to the hardware request queue corresponding to the target processing end device, and sending the request information to the target processing end device through the hardware request queue;

[0165] Step 1010: After receiving the request information, the target processing end device completes the read and write operation and returns the data read and write result;

[0166] Step 1012: Receive the result query request sent by the user state, confirm the completed target request information, store the data read and write results corresponding to the completed target request information in the result ring queue, and feed back the data read and write results to the user state through the result ring queue.

[0167] In the embodiments of this specification, through efficient direct-type request processing, intelligent request information construction and conversion, optimized request allocation and execution, and active polling result feedback mechanism, the overall performance, concurrent processing capability and response speed of data read and write operations are significantly improved, while the overhead of middle-layer processing and the delay of cross-core communication are reduced, ensuring the efficiency and stability of the system when processing a large number of data read and write requests, and improving the user experience.

[0168] Corresponding to the above method embodiment, this specification also provides a data reading and writing device embodiment, Fig.11 FIG. 1 shows a schematic diagram of the structure of a data reading and writing device provided by an embodiment of the present specification. Fig.11 As shown, the device comprises:

[0169] The receiving module 1102 is configured to receive a data read / write request sent by the user state through the asynchronous I / O layer, and store the data read / write request in a request queue;

[0170] The sending module 1104 is configured to send the data read / write request to the driver layer when the asynchronous I / O layer determines that the data read / write request is a pass-through type;

[0171] The processing module 1106 is configured to parse the data read / write request through the driver layer, and send the request information of the data read / write request to the target processing end device for read / write processing.

[0172] Optionally, receiving a data read / write request sent by a user state and storing the data read / write request in a request queue, the receiving module 1102 is further configured to: receive a data read / write request sent by a user state, wherein the data read / write request is encapsulated with request header information;

[0173] Store data read and write requests in the request queue.

[0174] Optionally, the processing module 1106 is further configured to: parse the data read and write request through the driver layer to obtain the request header information; determine the address information and length information of the data buffer requested in the user state; based on the request header information, address information and length information, obtain the request information of the data read and write request; and send the request information to the target processing end device for read and write processing.

[0175] Optionally, the processing module 1106 is further configured to: convert the address information of the data buffer into physical address information; encapsulate the request header information, the physical address information and the length information to obtain the request information of the data read and write request.

[0176] Optionally, the request information also includes a flag bit, which is used to mark the type of data read and write request; the processing module 1106 is further configured to: when the data read and write request is a single request, encapsulate the request header information, physical address information and length information of the data buffer, and set the flag bit to a non-vector read and write request type to obtain the request information of the data read and write request; when the data read and write request is a request sequence, encapsulate the request header information, physical address information and length information of each data buffer, and set the flag bit to a vector read and write request type to obtain the request information of the data read and write request.

[0177] Optionally, the processing module 1106 also includes: an allocation unit, configured to store the request information into a core request queue based on the correspondence between the data read and write requests and the request information; allocate the request information from the core request queue to a hardware request queue corresponding to each processing end device based on the device information of each processing end device; and send the request information to the target processing end device for read and write processing through the hardware request queue corresponding to the target processing end device.

[0178] Optionally, the sending module 1104 is further configured to: read an operator of the data read / write request; and determine that the data read / write request is a pass-through type when the operator of the data read / write request includes a specific handle.

[0179] Optionally, the asynchronous I / O layer also includes a result queue; the data reading and writing device also includes: a feedback unit, configured to receive data reading and writing results fed back by the target processing end device; the data reading and writing results are stored in the result queue of the asynchronous I / O layer through the driver layer, and the data reading and writing results in the result queue are used to feedback to the user state.

[0180] Optionally, the feedback unit is further configured to: receive a result query request sent by the user state; determine the completed target request information based on the result query request; and store the data reading and writing results corresponding to the target request information into the result queue of the asynchronous I / O layer through the driver layer.

[0181] The above is a schematic scheme of a data reading and writing device of this embodiment. It should be noted that the technical scheme of the data reading and writing device and the technical scheme of the above data reading and writing method belong to the same concept, and the details of the technical scheme of the data reading and writing device that are not described in detail can all be referred to the description of the technical scheme of the above data reading and writing method.

[0182] See also Fig.12 , Fig.12 The block diagram of a computing device 1200 according to one embodiment of the present specification is shown. The components of the computing device 1200 include but are not limited to a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and the database 1250 is used to store data.

[0183] The computing device 1200 also includes an access device 1240 that enables the computing device 1200 to communicate via one or more networks 1260. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1240 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0184] In one embodiment of the present specification, the above components of the computing device 1200 and Fig.12 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.12 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0185] The computing device 1200 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1200 may also be a mobile or stationary server.

[0186] The processor 1220 is used to execute the following computer program / instruction, which implements the steps of the above-mentioned data reading and writing method when executed by the processor.

[0187] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data reading and writing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data reading and writing method.

[0188] An embodiment of the present specification also provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above-mentioned data reading and writing method when executed by a processor.

[0189] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned data reading and writing method belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical scheme of the above-mentioned data reading and writing method.

[0190] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned data reading and writing method when executed by a processor.

[0191] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above data reading and writing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above data reading and writing method.

[0192] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0193] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0194] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0195] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0196] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data reading and writing method, applied to kernel state, wherein the kernel state includes an asynchronous I / O layer and a driver layer, and the asynchronous I / O layer includes a request queue; the method includes: Receive a data read / write request sent by a user state through the asynchronous I / O layer, and store the data read / write request in the request queue; When the asynchronous I / O layer determines that the data read / write request is a pass-through type, sending the data read / write request to the driver layer; The data read and write request is parsed by the driver layer, and the request information of the data read and write request is sent to the target processing end device for read and write processing, wherein the data read and write request is parsed by the driver layer, and the request information of the data read and write request is sent to the target processing end device for read and write processing, including: parsing the data read and write request by the driver layer to obtain request header information; determining the address information and length information of the data buffer requested in the user state; obtaining the request information of the data read and write request based on the request header information, the address information and the length information; and sending the request information to the target processing end device for read and write processing.

2. According to the method of claim 1, the step of receiving a data read / write request sent by a user mode and storing the data read / write request in the request queue comprises: Receiving a data read / write request sent by a user state, wherein the data read / write request is encapsulated with request header information; The data read and write request is stored in the request queue.

3. The method according to claim 1, wherein obtaining the request information of the data read / write request based on the request header information, the address information and the length information comprises: Converting the address information of the data buffer into physical address information; The request header information, the physical address information and the length information are encapsulated to obtain request information of the data read and write request.

4. The method according to claim 3, wherein the request information further includes a flag bit, and the flag bit is used to mark the type of the data read / write request; and the step of encapsulating the request header information, the physical address information, and the length information to obtain the request information of the data read / write request comprises: In the case where the data read / write request is a single request, encapsulating the request header information, the physical address information and the length information of the data buffer, and setting the flag bit to a non-vector read / write request type to obtain request information of the data read / write request; When the data read and write request is a request sequence, the request header information, the physical address information and the length information of each data buffer are encapsulated, and the flag is set to the vector class read and write request type to obtain the request information of the data read and write request.

5. The method according to claim 1, after obtaining the request information of the data read / write request based on the request header information, the address information and the length information, further comprises: Based on the correspondence between the data read and write request and the request information, storing the request information in a core request queue; Based on the device information of each processing end device, the request information is distributed from the core request queue to the hardware request queue corresponding to each processing end device; The sending of the request information to a target processing end device for reading and writing processing includes: The request information is sent to the target processing end device for read and write processing through the hardware request queue corresponding to the target processing end device.

6. The method according to claim 1, wherein when the asynchronous I / O layer determines that the data read / write request is a pass-through type, before sending the data read / write request to the driver layer, the method further comprises: Read the operator of the data read / write request; In a case where the operator of the data read / write request includes a specific handle, it is determined that the data read / write request is a pass-through type.

7. The method according to any one of claims 1 to 6, wherein the asynchronous I / O layer further comprises a result queue; after the driver layer parses the data read / write request and sends the request information of the data read / write request to the target processing end device for read / write processing, the method further comprises: Receiving data reading and writing results fed back by the target processing end device; The data reading and writing results are stored in the result queue of the asynchronous I / O layer via the driver layer, and the data reading and writing results in the result queue are used to feed back to the user state.

8. The method according to claim 7, before storing the data reading and writing result in the result queue of the asynchronous I / O layer via the driver layer, further comprising: Receiving a result query request sent by the user state; Based on the result query request, determine completed target request information; The step of storing the data reading and writing result into the result queue of the asynchronous I / O layer via the driver layer includes: The data reading and writing results corresponding to the target request information are stored in the result queue of the asynchronous I / O layer through the driver layer.

9. A data reading and writing device, applied to kernel state, the kernel state includes an asynchronous I / O layer and a driver layer, the asynchronous I / O layer includes a request queue; the device includes: A receiving module, configured to receive a data read / write request sent by a user state through the asynchronous I / O layer, and store the data read / write request in the request queue; A sending module, configured to send the data read / write request to the driver layer when the asynchronous I / O layer determines that the data read / write request is a pass-through type; The processing module is configured to parse the data read / write request through the driver layer, and send the request information of the data read / write request to the target processing end device for read / write processing, wherein the processing module is further configured to: parse the data read / write request through the driver layer to obtain request header information; Determine the address information and length information of the data buffer requested in the user mode; obtain the request information of the data read and write request based on the request header information, the address information and the length information; send the request information to the target processing end device for read and write processing.

10. A data reading and writing system, comprising a user state, a kernel state and a plurality of processing end devices, wherein the kernel state comprises an asynchronous I / O layer and a driver layer, and the asynchronous I / O layer comprises a request queue; The user state is used to send a data read and write request to the kernel state; The asynchronous I / O layer is used to store the data read / write request into the request queue, and when determining that the data read / write request is a pass-through type, send the data read / write request to the driver layer; The driver layer is used to parse the data read and write request and send the request information of the data read and write request to the target processing end device among the multiple processing end devices, wherein the driver layer is further configured to: parse the data read and write request to obtain the request header information; determine the address information and length information of the data buffer requested by the user state; based on the request header information, the address information and the length information, obtain the request information of the data read and write request; and send the request information to the target processing end device; The target processing end device is used to perform read and write processing based on the request information.

11. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the data reading and writing method according to any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the steps of the data reading and writing method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the data reading and writing method according to any one of claims 1 to 8.

Images