A fault-tolerant method and system for DMA efficient data transmission

Abstract

The present invention relates to the field of DMA data transmission, specifically a fault-tolerant method and system for efficient DMA data transmission, which addresses the prior art issue of the probability of reading erroneous data from a DMA buffer. The present invention connects a host device driver interface and a DMA; the DMA is provided in a peripheral driver and includes an IO pool and a hardware queue for caching IO data; the DMA obtains IO data and verifies the data; the present invention improves DMA transmission efficiency, availability, reliability, and data consistency through methods such as DMA ring buffering and CPU balancing; transforms data sent to peripherals to strengthen the IO data ID; and performs correctness verification on data returned by the peripherals via DMA, thereby reducing the probability of reading erroneous data from the DMA buffer.

Description

Technical Field

[0001] The present invention relates to the field of DMA data transmission, and in particular to a fault-tolerant method and system for DMA efficient data transmission. Background Art

[0002] DMA (Direct Memory Access) is a feature in computer systems. DMA is commonly used for communication between hardware devices such as network cards, RAID controller cards, and hard drives, and software operating systems. DMA data transfers occur directly between the peripheral device and host memory, bypassing the CPU. When processing DMA buffered data, the software operating system typically achieves high-speed transmission and accurate data read and write. Data read and write operations between the peripheral device and the host are performed directly through the DMA buffer. During reads, the host reads data from the DMA-mapped main memory. The main memory data must be cached by the CPU before it can be processed.

[0003] CPU load can affect cache data updates, which in turn affects the accuracy of DMA data reads, especially under heavy CPU load. DMA updates data in main memory, and under heavy CPU load, old copies of data may remain in the CPU cache. This can result in stale data being read from the cache and incorrect data being read from the DMA cache.

[0004] With the exception of X86, most current CPUs use a weak consistency model, meaning the hardware doesn't automatically maintain cache coherency. DMA writes can result in memory refresh errors and CPU cache failures. For peripherals like hard drives, network devices, and RAID cards, this DMA behavior can cause data read and write timeouts, reduced I / O bandwidth, and data inconsistencies.

[0005] Data read from the DMA cache has a probability of reading erroneous data. Many peripheral drivers enable data verification. When data errors are detected, they often directly shut down or restart the peripheral. While these methods can reduce the possibility of data errors to a certain extent, due to imperfect verification methods, some data errors may not be detected, causing serious errors and impacts. Furthermore, the direct shutdown and restart of peripherals significantly reduces product experience, stability, and efficiency.

[0006] There is an urgent need for a new type of DMA high-efficiency data transmission fault-tolerant method and system thereof that can solve the above problems. Summary of the Invention

[0007] The present invention provides a fault-tolerant method and system for DMA efficient data transmission, which solves the problem of the probability of reading erroneous data from a DMA buffer in the prior art.

[0008] The technical solution of the present invention is implemented as follows: a fault-tolerant method for DMA efficient data transmission, comprising the following:

[0009] Step 1: Initialize the system: Initialize the IO pool and hardware queues. Based on the number of CPUs and the maximum number of IOs accepted by the peripherals (M), the length of the IO pool is less than M. Set the number of hardware queues (N), where N is less than the number of CPUs. Initialize the input cache queue (IQ) and output cache queue (OQ) of each hardware queue. The queue lengths of the input cache queue (IQ) and output cache queue (OQ) are X, where X = MIN (M / N, 32). Initialize the PI and CI of the input cache queue (IQ) and output cache queue (OQ). PI refers to the producer index, representing the position of the circular queue where data should currently be written. CI refers to the consumer index, representing the position of the circular queue where the consumer should currently read data.

[0010] Step 2: The IO pool receives IO data from the host device driver interface. If there is free space in the IO pool, the IO data is saved. If there is no free space, a queue busy message is returned to the device driver interface.

[0011] Step 3: The hardware queue obtains the IO data from the IO pool as IO metadata, and saves the data index, ID, and IO metadata of the IO pool to the input buffer queue IQ, and modifies the PI; wherein, the IO metadata represents the attributes of the IO data, including the read and write operation type, the DMA address corresponding to the IO data, and the IO data size; the data index of the Io pool refers to the storage location index of the IO data in the Io pool, through which the IO data can be quickly found in the Io pool; the ID represents the unique identifier of the IO data, which is used to uniquely identify specific IO data, and is the data ID contained in the IO data;

[0012] Step 4: The peripheral processes IO metadata: The peripheral receives IO metadata based on the change of PI, obtains IO metadata and processes it. After processing, the CI of the input buffer queue IQ is modified, the processing results are written to the IO pool queue and the output buffer queue OQ, and the PI of the output buffer queue OQ is modified.

[0013] Step 5: Output Buffer Queue (OQ): After receiving the PI change, the OQ processes the IO response message sent by the peripheral. It accesses the cache pointed to by the CI in the OQ and the IO pool index stored in the cache. Based on the IO pool index, it finds the corresponding IO data. It compares the processed IO data with the IO metadata and verifies the data, including data queue verification, IO data reference verification, and data ID verification. If the verified data is correct, it proceeds to step 6.

[0014] Step 6: Modify the CI of the output buffer queue OQ to complete the IO data processing, release the space of the IO data in the IO pool and hardware queue, and return the processed IO data to the device driver interface.

[0015] Specifically, step 3 includes: selecting a hardware queue based on the idleness of each hardware queue to obtain IO metadata; determining whether the ID in the original IO metadata can uniquely identify the IO metadata in the hardware queue, that is, whether the ID can uniquely identify the IO metadata in the input buffer queue IQ and the output buffer queue OQ; and modifying the IO metadata ID if it cannot be uniquely identified.

[0016] Method 1: Generate an auto-increment ID based on the hardware queue length X. Mapping is performed internally within the hardware queue, recording the mapping relationship between the request_ID and the data VID + request_ID. This modified ID makes the ID of the IO metadata unique within a separate hardware queue. The auto-increment ID is a unique identifier that is automatically generated and incremented. This ID is combined with the original ID of the IO data to generate a new ID, enhancing its uniqueness.

[0017] The second method is to analyze the IO metadata data structure sent by the device driver interface and use the unused space in the IO metadata data structure to regenerate the data VID. The length of the input buffer queue IQ and the output buffer queue OQ of the hardware queue are both less than 32. Only 5 bytes of space are needed to generate an automatically growing VID. Using 5 bytes of space can ensure the uniqueness of the ID. The 5-byte space can come from the reserved data area of ​​the IO metadata, or the original ID can be modified to use the original ID + 5 bytes to generate a new IO data ID. The data VID refers to the temporary data unique identifier generated by the reserved space in the original IO metadata structure during the data processing process. The reserved space must not affect data processing. After the peripheral device completes data processing, the storage content of the reserved space can be returned as is.

[0018] If the data verification fails in step 5, proceed to step 7 to set a memory barrier: if the CI in the OP queue is verified to have a data verification error, set a memory barrier to maintain data consistency between the DMA cache data and the CPU cache data, re-read the DMA cache data, and perform data verification; if the verified data is correct, proceed to step 6; if the verification fails, proceed to step 8;

[0019] Step 8: Clear the CPU cache: Reread the DMA cache data and perform data verification; if the verified data is correct, proceed to step 6; if the correct data is still not read, then record the CI value of the output cache queue OQ unreachable data in the hardware queue, classify the IO metadata as unreachable data, and proceed to step 9;

[0020] Step 9: Modify the CI of the output buffer queue OQ: notify the peripheral device to complete this processing, but do not notify the device driver interface of the failure of this processing; set the timeout processing function for the unreachable data. The timeout processing function is asynchronous processing, that is, after asynchronously waiting for the set time, notify the device driver interface that the unreachable data has timed out; wait for data timeout;

[0021] Step 10: The output buffer queue OQ continues to wait for new data: within the timeout period, the output buffer queue OQ is waited for new data to arrive; if there is no new data, the device driver interface is notified that the IO data processing has timed out; if there is new data, the process proceeds to step 11;

[0022] Step 11: The output buffer queue OQ receives new data from the peripheral: that is, the PI of the output buffer queue OQ changes, then the hardware queue is checked to see if there is any record of unreached data. If there is no unreached data, then go to step 5; if there is unreached data, then go to step 12;

[0023] Step 12: Determine whether the CI of the unreachable data points to the output buffer queue OQ position: whether the data at this position has been updated. If not, or the data check is incorrect, then notify the device driver interface that the IO data processing has timed out, delete the record of the unreachable data in the hardware queue, and complete the processing of the unreachable data; if the data has been updated and the data check is correct, then proceed to step 13;

[0024] Step 13: Access the data at the CI location of the unreached data, complete the IO data processing, and return the IO data processing result to the device driver interface; regardless of whether the data is correct, the hardware queue deletes the record of the unreached data; resume the new data processing in the output buffer queue OQ, that is, enter step 5.

[0025] A fault-tolerant system for DMA efficient data transmission, connecting a host device driver interface and a peripheral device for transmitting IO data; including DMAs connected to the device driver interface and the peripheral device respectively;

[0026] The DMA is set in the peripheral driver, including an IO pool and a hardware queue for caching IO data. It receives IO data from the host device driver interface, caches the IO data through DMA, and transmits the IO data to the peripheral through DMA. After the peripheral completes processing the IO data, it returns the processing result through DMA; the hardware queue mainly includes an output cache queue OQ and an input cache queue IQ; the IO pool is connected to the host through the device driver interface and transmits the buffered IO data to the input cache queue IQ of the hardware queue; the peripheral receives IO data from the input cache queue IQ and writes the processed IO data into the output cache queue OQ; the DMA obtains the IO data and verifies the data.

[0027] The IO pool caches IO data from the host device driver interface in a circular queue manner; the unit of the IO pool circular queue is a request for data structure memory space; the request includes data ID, IO metadata, IO metadata including IO data type, DMA physical address information corresponding to the IO data buffer, and IO data time information; when the input cache queue IQ has idle resources, the hardware queue sends the information in the IO pool to the input cache queue IQ for caching the data index, original data and DMA data type information of the IO pool; the original data includes data VID and request_ID; the input cache queue IQ includes two pointer indexes pointing to the circular queue PI and CI respectively.

[0028] The peripheral obtains the original data from the input buffer queue IQ, obtains the complete information of the IO data through the DMA physical address information and processes it; the peripheral determines whether new data has arrived based on the change of PI, processes the new IO data, and modifies CI after processing is completed.

[0029] The output buffer queue OQ caches the IO pool index information, IO metadata and IO data processing results returned after the peripheral device completes IO processing; the output buffer queue OQ adopts a circular queue method, and the output buffer queue OQ includes two pointer indexes PI and CI pointing to the output buffer queue OQ respectively.

[0030] The device driver interface may be a host storage layer protocol or a host network layer protocol.

[0031] Improve the availability, reliability, and data consistency of DMA transmission, especially for PCIE-based peripheral cards, to avoid data inconsistency issues when the latest data in the DMA buffer cannot be accessed in a timely manner under conditions such as sustained high IO data pressure, high CPU load, and IO data bursts. The present invention improves DMA by providing a DMA ring buffer with hardware queues as units, with each CPU corresponding to a hardware queue. By uniquely identifying each data IO sent, and implementing memory barrier rereading, CPU cache clearing rereading, and readback techniques for DMA data, the problem of CPU cache not being able to be updated in a timely manner is reduced. This method improves data fault tolerance without affecting the original data processing efficiency of the original RAID card driver, and improves the reliability, availability, and data consistency of peripheral drivers such as RAID cards, network cards, and hard drives.

[0032] The present invention has the following beneficial effects:

[0033] The data sent to the peripherals is modified without destroying the original data structure, and the IO data ID is strengthened. The data returned by the peripherals through DMA is checked for correctness to reduce the probability of reading incorrect data from the DMA cache.

[0034] For incorrect data, DMA data is reread and re-verified by setting memory barriers and CPU cache cleanup, providing strong fault tolerance and ensuring data correctness.

[0035] Added DMA buffer readback method to avoid data loss;

[0036] Improve DMA transfer efficiency, availability, reliability, and data consistency through DMA ring buffering and CPU balancing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 : DMA high-speed transmission framework diagram;

[0039] Figure 2 : DMA data high-speed transmission and fault tolerance flow chart;

[0040] Figure 3 : Data processing diagram. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0042] Combine Figure 1 DMA high-speed transmission framework diagram, Figure 2 DMA data high-speed transmission and fault tolerance flow chart and Figure 3 As shown in the data processing diagram, the present invention discloses a fault-tolerant system for DMA efficient data transmission, which connects a host device driver interface and a peripheral device for transmitting IO data; and includes DMAs connected to the device driver interface and the peripheral device respectively;

[0043] The DMA is set in the peripheral driver, including an IO pool and a hardware queue for caching IO data. It receives IO data from the host device driver interface, caches the IO data through DMA, and transmits the IO data to the peripheral through DMA. After the peripheral completes processing the IO data, it returns the processing result through DMA; the hardware queue mainly includes an output buffer queue OQ and an input buffer queue IQ; the IO pool is connected to the host through the device driver interface and transmits the buffered IO data to the input buffer queue IQ of the hardware queue; the peripheral receives IO data from the input buffer queue IQ and writes the processed IO data into the output buffer queue OQ; the output buffer queue OQ returns the IO data to the host; the DMA obtains the IO data and verifies the data. In addition to data queue verification and IO data reference verification, it also verifies whether the data ID is correct.

[0044] The IO pool caches IO data from the host device driver interface in a circular queue manner; the unit of the IO pool circular queue is a request for data structure memory space; the request includes data ID, IO metadata, and IO metadata includes IO data type, DMA physical address information corresponding to the IO data buffer, and IO data time information; when the input cache queue IQ has idle resources, the hardware queue sends the information in the IO pool to the input cache queue IQ for caching the data index, original data and DMA data type information of the IO pool; the original data includes data VID and request_ID; request_ID is the device driver interface used to identify the uniqueness of IO data, that is, the original ID of IO data, and data VID is the data ID added in the present invention to ensure data uniqueness. Data VID and request_ID together constitute the data ID; the input cache queue IQ includes two pointer indexes PI and CI pointing to the circular queue respectively.

[0045] The peripheral obtains the original data from the input cache queue IQ, obtains the complete information of the IO data through the DMA physical address and processes it; the peripheral determines whether new data has arrived based on the change of PI, processes the newly arrived IO data, and modifies CI after processing is completed; based on the maximum number of caches that the peripheral can handle and the maximum number of caches that the host driver device can handle, it is decided whether PI and CI are stored in the peripheral register or DMA cache.

[0046] The output buffer queue OQ caches the IO pool index information, IO metadata, and IO data processing results returned after the peripheral device completes IO processing; the output buffer queue OQ adopts a circular queue method, and the output buffer queue OQ includes two pointer indexes pointing to the PI and CI of the output buffer queue OQ respectively. According to the physical characteristics of the peripheral, it is determined whether the PI and CI are stored in the peripheral register or DMA cache.

[0047] The device driver interface may be a host storage layer protocol or a host network layer protocol.

[0048] A fault-tolerant method for DMA efficient data transmission using the above system includes the following:

[0049] Step 1: Initialize the system: Initialize the IO pool and hardware queues. Based on the number of CPUs and the maximum number of IOs accepted by the peripherals (M), the length of the IO pool is less than M. Set the number of hardware queues (N), where N is less than the number of CPUs. Initialize the input buffer queue (IQ) and output buffer queue (OQ) of each hardware queue. The queue lengths of the input buffer queue (IQ) and output buffer queue (OQ) are X, where X = MIN (M / N, 32). Initialize the PI and CI of the input buffer queue (IQ) and output buffer queue (OQ).

[0050] CI, the consumer index, and PI, the producer index, are two key pointer variables used to manage and track the state of data in the circular queue buffer, thereby efficiently utilizing the array space.

[0051] The PI pointer indicates the location in the circular queue where the producer should currently write data. It always points to the next available write location in the circular queue. The producer writes data to the location pointed to by the PI pointer in the circular queue buffer. After writing the data, the producer updates the PI pointer to point to the next available write location. Normally, the PI pointer moves in a circular manner based on the size of the circular queue. That is, when the PI reaches the end of the queue, it "wraps around" to the beginning.

[0052] The CI pointer indicates the current location in the ring queue from which the consumer should read data. It always points to the next available read location in the ring queue. The consumer reads data from the location pointed to by the CI pointer in the ring queue buffer. After reading the data, the consumer updates the CI pointer to point to the next available read location. Similarly, the CI pointer moves cyclically based on the size of the ring queue. When a peripheral uses a ring queue to send data to the host, the peripheral is the producer and the host is the consumer. When a host uses a ring queue to send data to a peripheral, the host is the producer and the peripheral is the consumer.

[0053] Step 2: The IO pool receives IO data from the host device driver interface: if there is free space in the IO pool, the IO data is saved; if there is no free space, a queue busy message is returned to the device driver interface;

[0054] Step 3: The hardware queue obtains IO data from the IO pool as IO metadata, saves the data index, ID of the IO pool, and the IO metadata to the input buffer queue IQ, and modifies the PI;

[0055] Step 4: The peripheral processes IO metadata: The peripheral receives IO metadata based on the change of PI, obtains IO metadata based on the IO metadata, and processes it. After the processing is completed, the CI of the input buffer queue IQ is modified, the processing results are written to the IO pool queue and the output buffer queue OQ, and the PI of the output buffer queue OQ is modified.

[0056] Step 5: Output Buffer Queue (OQ): After receiving the PI change, the OQ processes the IO response message sent by the peripheral. It accesses the cache pointed to by the CI in the OQ and the IO pool index stored in the cache. Based on the IO pool index, it finds the corresponding IO data and compares the processed IO data with the IO metadata. It then verifies the data, including data queue verification, IO data reference verification, and data ID verification.

[0057] Step 6: Modify the CI of the output buffer queue OQ to complete the IO data processing, release the space of the IO data in the IO pool and hardware queue, and return the processed IO data to the device driver interface.

[0058] Specifically, step 3 includes: selecting a hardware queue based on the idleness of each hardware queue to obtain IO metadata; determining whether the ID in the original IO metadata can uniquely identify the IO metadata in the hardware queue, that is, whether the ID can uniquely identify the IO metadata in the input buffer queue IQ and the output buffer queue OQ; and modifying the IO metadata ID if it cannot be uniquely identified.

[0059] The first method: Generate a self-increasing ID based on the hardware queue length X, perform mapping within the hardware queue, record the mapping relationship between the request_ID and the data VID + request_ID, and modify the ID to make the IO metadata ID unique in a separate hardware queue;

[0060] The second method is to analyze the IO metadata data structure sent by the device driver interface and use the unused space in the IO metadata data structure to regenerate the data VID. The length of the input cache queue IQ and the output cache queue OQ of the hardware queue are both less than 32. Only 5 bytes of space are needed to generate an automatically growing VID. Using 5 bytes of space can ensure the uniqueness of the ID. The 5-byte space can come from the reserved data area of ​​the IO metadata, or the original ID can be modified to use the original ID + 5 bytes of space to generate a new IO data ID.

[0061] If the data verification fails in step 5, proceed to step 7 to set a memory barrier: if the CI in the OP queue is verified to have a data verification error, set a memory barrier to maintain data consistency between the DMA cache data and the CPU cache data, re-read the DMA cache data, and perform data verification; if the verified data is correct, proceed to step 6; if the verification fails, proceed to step 8;

[0062] Step 8: Clear the CPU cache: Reread the DMA cache data and perform data verification; if the verified data is correct, proceed to step 6; if the correct data is still not read, then record the CI value of the output cache queue OQ unreachable data in the hardware queue, classify the IO metadata as unreachable data, and proceed to step 9;

[0063] Step 9: Modify the CI of the output buffer queue OQ: notify the peripheral device to complete this processing, but do not notify the device driver interface of the failure of this processing; set the timeout processing function for the unreachable data. The timeout processing function is asynchronous processing, that is, after asynchronously waiting for the set time, notify the device driver interface that the unreachable data has timed out; wait for data timeout;

[0064] Step 10: The output buffer queue OQ continues to wait for new data: within the timeout period, the output buffer queue OQ is waited for new data to arrive; if there is no new data, the device driver interface is notified that the IO data processing has timed out; if there is new data, the process proceeds to step 11;

[0065] Step 11: The output buffer queue OQ receives new data from the peripheral device: that is, the PI of the output buffer queue OQ changes, then the hardware queue is checked to see if there is any record of unreached data. If there is no unreached data, then go to step 5; if there is unreached data, then go to step 12;

[0066] Step 12: Determine the CI of the unreachable data and point it to the output buffer queue OQ position: whether the data at this position has been updated. If not, or the data check is incorrect, then notify the device driver interface that the IO data processing has timed out, delete the record of the unreachable data in the hardware queue, and complete the processing of the unreachable data; if the data has been updated and the data check is correct, then proceed to step 13;

[0067] Step 13: Access the data at the CI location of the unreached data, complete the IO data processing, and return the IO data processing result to the device driver interface; regardless of whether the data is correct, the hardware queue deletes the record of the unreached data to avoid falling into a loop of processing the unreached data; regardless of whether the data is correct, the unreached data is only processed once, and the record of the unreached data will be deleted from the hardware queue; resume the new data processing in the output buffer queue OQ, that is, enter step 5.

[0068] The present invention improves the availability, reliability, and data consistency of DMA transmission, particularly for PCIE-based peripheral cards. This prevents data inconsistency issues when the latest data in the DMA buffer cannot be accessed in a timely manner under conditions such as sustained high I / O data pressure, high CPU load, or I / O data bursts. The present invention improves DMA by providing a DMA ring buffer with hardware queues as units, with each CPU corresponding to a hardware queue. The present invention also reduces issues caused by the CPU's cache not being able to be updated in a timely manner by uniquely identifying each transmitted I / O data and implementing memory barrier rereading, CPU cache clearing rereading, and readback techniques for DMA data. This improves data fault tolerance without affecting the original data processing efficiency of the original RAID card driver, and enhances the reliability, availability, and data consistency of peripheral drivers such as RAID cards, network cards, and hard drives.

[0069] The present invention has the following beneficial effects:

[0070] The data sent to the peripherals is modified without destroying the original data structure, and the IO data ID is strengthened. The data returned by the peripherals through DMA is checked for correctness to reduce the probability of reading incorrect data from the DMA cache.

[0071] For incorrect data, DMA data is reread and re-verified by setting memory barriers and CPU cache cleanup, providing strong fault tolerance and ensuring data correctness.

[0072] Added DMA buffer readback method to avoid data loss;

[0073] Improve DMA transfer efficiency, availability, reliability, and data consistency through DMA ring buffering and CPU balancing.

[0074] Of course, without departing from the spirit and essence of the present invention, technicians familiar with the field should be able to make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A fault-tolerant method for DMA efficient data transmission, characterized by: These include: Step 1: Initialize the system: Initialize the IO pool and hardware queues. Set the number of hardware queues N based on the number of CPUs and the maximum number of IOs accepted by the peripherals (M). The length of the IO pool is less than M, and N is less than the number of CPUs. Initialize the input cache queue IQ and output cache queue OQ of each hardware queue. The queue lengths of the input cache queue IQ and output cache queue OQ are X, where X = MIN (M / N, 32). Initialize the PI and CI of the input cache queue IQ and output cache queue OQ. PI refers to the producer index, which represents the position of the circular queue where data should be written. CI refers to the consumer index, which represents the position of the circular queue where the consumer should read data. Step 2: The IO pool receives IO data: The IO data comes from the host driver device driver interface. If the IO pool has free space, the IO data is saved; if the IO pool has no free space, the queue is busy. Step 3: The hardware queue obtains IO data from the IO pool as IO metadata, and saves the data index, ID, and IO metadata of the IO pool to the input buffer queue IQ, and modifies the PI; wherein, the IO metadata represents the attributes of the IO data, including the read / write operation type, the DMA address corresponding to the IO data, and the IO data size; the data index of the IO pool refers to the storage location index of the IO data in the IO pool, through which the IO data can be quickly found in the IO pool; the ID represents the unique identifier of the IO data, which is used to uniquely identify specific IO data, and is the data ID contained in the IO data; Step 4: The peripheral processes IO metadata: The peripheral receives IO metadata based on the change of PI, obtains IO metadata based on the IO metadata, and processes it. After the processing is completed, the CI of the input buffer queue IQ is modified, the processing results are written to the IO pool queue and the output buffer queue OQ, and the PI of the output buffer queue OQ is modified. Step 5: Output Buffer Queue (OQ): After receiving the PI change, the OQ processes the IO response message sent by the peripheral and accesses the cache pointed to by the CI in the OQ. The cache stores the IO pool index, finds the corresponding IO data based on the IO pool index, compares the processed IO data with the IO metadata, and verifies the data, namely, data queue verification, IO data reference verification, and data ID verification. If the verified data is correct, proceed to step 6. If the data verification fails, proceed to step 7. Step 6: Modify the CI of the output buffer queue OQ to complete the IO data processing, release the space of the IO data in the IO pool and hardware queue, and return the processed IO data to the device driver interface; Step 7: Set a memory barrier: If the CI in the OP queue is verified to have a data check error in the IO metadata response function, set a memory barrier to maintain data consistency between the DMA cache data and the CPU cache data, re-read the DMA cache data, and perform data verification; if the verified data is correct, proceed to step 6; if the verification fails, proceed to step 8; Step 8: Clear the CPU cache: Reread the DMA cache data and perform data verification; if the verified data is correct, proceed to step 6; if the correct data is still not read, then record the CI value of the output cache queue OQ unreachable data in the hardware queue, classify the IO metadata as unreachable data, and proceed to step 9; Step 9: Modify the CI of the output buffer queue OQ: notify the peripheral device to complete this processing, but do not notify the device driver interface of the failure of this processing; set the timeout processing function for the unreachable data. The timeout processing function is asynchronous processing, that is, after asynchronously waiting for the set time, notify the device driver interface that the unreachable data has timed out; wait for data timeout; Step 10: The output buffer queue OQ continues to wait for new data: within the timeout period, the output buffer queue OQ is waited for new data to arrive; if there is no new data, the device driver interface is notified that the IO data processing has timed out; if there is new data, the process proceeds to step 11; Step 11: The output buffer queue OQ receives new data from the peripheral device: that is, the PI of the output buffer queue OQ changes, then the hardware queue is checked to see if there is any record of unreached data. If there is no unreached data, then go to step 5; if there is unreached data, then go to step 12; Step 12: Determine the CI of the unreachable data and point it to the output buffer queue OQ position: whether the data at this position has been updated. If not, or the data check is incorrect, then notify the device driver interface that the IO data processing has timed out, delete the record of the unreachable data in the hardware queue, and complete the processing of the unreachable data; if the data has been updated and the data check is correct, then proceed to step 13; Step 13: Access the data at the CI location of the unreached data, complete the IO data processing, and return the IO data processing result to the device driver interface; regardless of whether the data is correct, the hardware queue deletes the record of the unreached data; resume the new data processing in the output buffer queue OQ, that is, enter step 5.

2. The fault-tolerant method for DMA efficient data transmission according to claim 1, wherein: Specifically, step 3 includes: selecting a hardware queue based on the idleness of each hardware queue to obtain IO metadata; determining whether the ID in the original IO metadata can uniquely identify the IO metadata in the hardware queue, that is, whether the ID can uniquely identify the IO metadata in the input buffer queue IQ and the output buffer queue OQ; and modifying the IO metadata ID if it cannot be uniquely identified. Method 1: Generate an auto-increment ID based on the hardware queue length X. Mapping is performed internally within the hardware queue, recording the mapping relationship between the request_ID and the data VID + request_ID. This modified ID makes the ID of the IO metadata unique within a separate hardware queue. The auto-increment ID is a unique identifier that is automatically generated and incremented. This ID is combined with the original ID of the IO data to generate a new ID, enhancing its uniqueness. The second method is to analyze the IO metadata data structure sent by the device driver interface and use the unused space in the IO metadata data structure to regenerate the data VID. The length of the input buffer queue (IQ) and output buffer queue (OQ) of the hardware queue is less than 32. Only 5 bytes of space are needed to generate an automatically growing VID. Using 5 bytes of space can ensure the uniqueness of the ID. The 5 bytes of space can be from the reserved data area of ​​the IO metadata, or the original ID can be modified to use the original ID + 5 bytes to generate a new IO data ID. The data VID is a temporary data unique identifier generated during the data processing process using the reserved space in the original IO metadata structure. The reserved space needs to not affect data processing, and after the peripheral device completes data processing, the storage content of the reserved space can be returned as is.

3. A fault-tolerant system for efficient data transmission using a DMA method according to claim 2, connecting a host device driver interface and a peripheral device for transmitting IO data; characterized in that: Including DMA connected to device driver interface and peripherals respectively; The DMA is set in the peripheral driver, including an IO pool and a hardware queue for caching IO data, receiving IO data from the host device driver interface, caching IO data, and transmitting IO data to the peripheral. After the peripheral processes the IO data, it returns the processing result; The hardware queue mainly includes an output buffer queue OQ and an input buffer queue IQ; The IO pool is connected to the host through the device driver interface and transmits the buffered IO data to the input buffer queue IQ of the hardware queue; the peripheral receives the IO data from the input buffer queue IQ and writes the processed IO data into the output buffer queue OQ; The DMA acquires IO data and verifies the data.

4. The fault-tolerant system for DMA efficient data transmission according to claim 3, characterized in that: The IO pool caches IO data from the host device driver interface in a circular queue manner; The unit of the IO pool circular queue is a request for data structure memory space; the request includes data ID and IO metadata; the IO metadata includes IO data type, IO data buffer and IO data time information; the IO data buffer corresponds to DMA physical address information; When the input buffer queue IQ has free resources, the hardware queue sends the information in the IO pool to the input buffer queue IQ: data index, original data and DMA data type information for caching the IO pool; the original data includes data VID and request_ID; The input buffer queue IQ includes two pointer indexes pointing to the circular queue PI and CI respectively.

5. The fault-tolerant system for DMA efficient data transmission according to claim 4, characterized in that: The peripheral obtains the original data from the input buffer queue IQ, obtains the complete information of the IO data through the DMA physical address information and processes it; The peripheral device determines whether new data has arrived based on the change of PI, processes the newly arrived IO data, and modifies CI after the processing is completed.

6. The fault-tolerant system for DMA efficient data transmission according to claim 5, characterized in that: The output buffer queue OQ caches the IO pool index information, IO metadata and IO data processing results returned after the peripheral device completes IO processing; The output buffer queue OQ adopts a circular queue mode. The output buffer queue OQ includes two pointer indexes, PI and CI, pointing to the output buffer queue OQ respectively.

7. A fault-tolerant system for DMA efficient data transmission according to any one of claims 4 to 6, characterized in that: The device driver interface is a host storage layer protocol or a host network layer protocol.

Images