PCIe-based data transmission method and device, equipment and medium

Abstract

The invention relates to the technical field of data transmission, in particular to a PCIe-based data transmission method and device, equipment and a medium. In the method, source equipment decomposes source data into at least one piece of sub-source data, a task queue is formed according to the at least one piece of sub-source data, the task queue comprises at least one task, the tasks are in one-to-one correspondence with the sub-source data, and each task comprises each piece of sub-source data and configuration information of each piece of sub-source data. And transmitting the task in the task queue to the target device based on the configuration information of the at least one sub-source data. Source data is decomposed to obtain sub-source data, a task queue is formed through configuration information, and corresponding operations are called based on the configuration information to copy the sub-source data to target equipment, so that it is guaranteed that the transmission process is executed independently and does not affect each other, and parallel transmission can be achieved to improve the transmission efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a PCIe-based data transmission method, device, equipment and medium. Background Art

[0002] With the popularity of various artificial intelligence (AI) chips, AI co-processing chip solutions are gradually applied to many AI solution scenarios. Since there will be a large amount of data interaction between the main processor and the co-processor, such as video stream data, model data, etc., if multi-channel high-speed data transmission becomes a bottleneck, the performance of the AI ​​co-processor chip cannot be fully utilized. At present, most AI data cards use the Peripheral Component Interconnect Express (PCIe) interface to transmit data. The PCIe direct memory access (DMA) channel can complete high-speed data transmission without the help of the central processing unit (CPU). Due to the limitations of the PCIe DMA channel, such as the need for continuous physical memory, the data transmission efficiency is relatively low for large data transmission.

[0003] At present, when transferring source data from the source device to the target device in PCIe-based data transmission, only one DMA channel can be used at a time, and other channels are idle. In addition, the tasks are executed in a serial loop, and each task needs to rely on the previous task to be completed. It is impossible to give full play to the characteristics of DMA that does not rely on the CPU. Especially when the amount of data is large, the efficiency of data transmission is not high. Therefore, how to reasonably configure the use of the buffer area and DMA channels in the device to improve the efficiency of data transmission has become an urgent problem to be solved. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a PCIe-based data transmission method, apparatus, device and medium to solve the problem of reasonably configuring the use of the cache area and DMA channel in the device to improve data transmission efficiency.

[0005] In a first aspect, an embodiment of the present application provides a data transmission method based on PCIe, wherein the data transmission method is run on a source device, the source device is connected to a target device via a DMA channel constructed by PCIe, and the source device stores source data to be transmitted, and the data transmission method includes:

[0006] Decomposing the source data into at least one sub-source data according to the capacity of the first buffer area in the source device;

[0007] According to the at least one sub-source data, a task queue is formed, wherein the task queue includes at least one task, the at least one task corresponds to the at least one sub-source data one by one, and each of the tasks includes each of the sub-source data and configuration information of each of the sub-source data;

[0008] Based on the configuration information of the at least one sub-source data, the tasks in the task queue are transmitted to the target device.

[0009] In a second aspect, an embodiment of the present application provides a data transmission device based on PCIe, wherein the data transmission device runs on a source device, the source device is connected to a target device through a DMA channel constructed by PCIe, and the source device stores source data to be transmitted, and the data transmission device includes:

[0010] A task decomposition module, configured to decompose the source data into at least one sub-source data according to the capacity of the first buffer area in the source device;

[0011] A task queue forming module, used for forming a task queue according to the at least one sub-source data, wherein the task queue includes at least one task, the at least one task corresponds to the at least one sub-source data one by one, and each of the tasks includes each of the sub-source data and configuration information of each of the sub-source data;

[0012] A data transmission module is used to transmit the tasks in the task queue to the target device based on the configuration information of the at least one sub-source data.

[0013] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data transmission method as described in the first aspect when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data transmission method as described in the first aspect is implemented.

[0015] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0016] In the process of connecting the source device to the target device through the DMA channel constructed by PCIe and transmitting the source data stored in the source device to the target device, the source device decomposes the source data into at least one sub-source data according to the capacity of the first buffer area in the source device, and forms a task queue according to the at least one sub-source data. The task queue includes at least one task, and the task corresponds to the sub-source data one by one. Each task includes each sub-source data and the configuration information of each sub-source data. Based on the configuration information of at least one sub-source data, the task in the task queue is transmitted to the target device, and the target device receives and processes the data in the task. Among them, the source data is decomposed to obtain the sub-source data and the task queue is formed through the configuration information. Based on the configuration information, the corresponding operations are called respectively to copy the tasks to the target device, so as to ensure that the transmission process is executed independently and does not affect each other, and parallelization can be achieved to improve the transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 This is a schematic diagram of an application environment of a PCIe-based data transmission method provided in Embodiment 1 of the present application;

[0019] Figure 2 It is a flowchart of a data transmission method based on PCIe provided in Example 1 of the present application;

[0020] Figure 3 It is a flowchart of a data transmission method based on PCIe provided in the second embodiment of the present application;

[0021] Figure 4 This is a schematic diagram of data interaction between a source device and a target device provided in Embodiment 2 of the present application;

[0022] Figure 5 This is a schematic diagram showing the relationship between time and thread during task execution provided in Example 2 of the present application;

[0023] Figure 6 It is a structural diagram of a PCIe-based data transmission device provided in Embodiment 3 of the present application;

[0024] Figure 7 It is a structural diagram of a computer device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0032] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0033] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0034] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0035] A PCIe-based data transmission method provided in the first embodiment of the present application can be applied in the following aspects: Figure 1 In an application environment, the target device communicates with the source device. Among them, the target device is provided with a computer device such as a PCIe interface card, an AI acceleration card, a PCIe network card, etc., and the source device can be a computer device such as a server, an embedded board, an x86 host, etc. with a PCIe interface. This communication can be used in application scenarios requiring high-speed data transmission and high computing power, and the two are connected through a PCIe bus communication. The target device and the source device can be set according to demand, and the target device can include but is not limited to a handheld computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a cloud terminal device, a personal digital assistant (personal digital assistant, PDA) and other computer devices. The source device can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0036] See also Figure 2 , is a flow chart of a PCIe-based data transmission method provided in the first embodiment of the present application. The above data transmission method can be applied to Figure 1The source device in the computer device corresponding to the source device is connected to the computer device corresponding to the target device to perform data exchange transmission. The source device is connected to the target device through the DMA channel constructed by PCIe. The number of DMA channels is related to the underlying driver of PCIe and the system architecture of the source device and the target device, and can be specifically configured according to needs. For example, the source device is a server, and the target device is a client. The data transmission method of the first embodiment of the present application is applied to the server. The client is operated to download the corresponding data from the server. When the server responds to the operation of the client, the server sends the data to the client through data processing and the assistance of the PCIe communication protocol, and then stores it in the target address of the client's memory. The specific content will be described in detail below.

[0037] Since the storage address on the source data side and the copy address on the target device side are physically discontinuous, it is necessary to apply for an intermediate physically continuous data buffer. During the communication between the source device and the target device through PCIe, the protocol provides corresponding buffers (i.e., Buffers) on both the source device and the target device, and the two buffers are connected through a DMA channel to realize the process of copying data from one buffer to another. In order to distinguish the names, the buffer of the source device is named the first buffer, and the buffer of the target device is named the second buffer. Therefore, the copying of data is mainly divided into three steps: copying from the source data storage to the first buffer, copying from the first buffer to the second buffer, and copying from the second buffer to the target address of the target device.

[0038] The data transmission method depends on the interaction between the source device and the target device. If the data transmission method is run in the source device, Figure 2 As shown, the data transmission method may include the following steps:

[0039] Step S201, decomposing source data into at least one sub-source data according to the capacity of the first buffer area in the source device.

[0040] In this application, source data refers to data stored in the memory of the source device. For the copy of the source data, if the source data has a large amount of data, one copy may be affected by bandwidth, memory, etc., resulting in copy interruption or high loss rate, or even failure to copy. Therefore, for source data with a large amount of data, the total copy task of one copy can be divided into multiple tasks to complete, and accordingly, the source data is decomposed to obtain multiple sub-source data.

[0041] When the amount of source data is larger than the capacity of the first buffer area, it is necessary to divide the source data into a size smaller than the capacity of the first buffer area, and then loop through the above three steps to transfer all the sub-source data to the target device.

[0042] Optionally, according to the capacity of the first buffer area in the source device, the source data is decomposed into at least one sub-source data, including:

[0043] Determine the decomposition number according to the data volume of the source data and the capacity of the first buffer area;

[0044] Get the storage address of the source data;

[0045] Using the decomposition number to evenly decompose the storage address, at least one source address and a target address corresponding to each source address are obtained;

[0046] The data between each source address and the corresponding target address is determined as corresponding sub-source data to obtain at least one sub-source data.

[0047] Among them, the Size of the source data is compared with the Buffer_Size of the cache area. If the Size of the source data is larger than the Buffer_Size of the cache area, the source data can be decomposed into n sub-source data. Correspondingly, the total copy task is decomposed into tasks, and the decomposition of the source data is performed at its corresponding storage address. After the decomposition, the source data is still stored in the memory of the source device, but is divided into sub-source data (that is, the data between the source address and the target address is a sub-source data).

[0048] Step S202: forming a task queue according to at least one sub-source data.

[0049] The task queue includes at least one task, the at least one task corresponds to at least one sub-source data one by one, and each task includes each sub-source data and configuration information of each sub-source data.

[0050] The configuration information is used to guide the corresponding copy operation to copy the corresponding task. For example, the configuration information includes the task status, which is used to represent the execution status of the task.

[0051] The order of the task queue can be sorted according to the order of the source address of the sub-source data. When executing tasks in the task queue, the corresponding operation threads are used to operate the corresponding tasks in sequence, including operations on the sub-source data, and updating and releasing the configuration information in the task.

[0052] Optionally, the configuration information includes at least one of task status, data size, first buffer area application information and second buffer area application information, the task status is used to record the first state, second state and third state of the sub-source data in the task, the data size is used to guide the application of the DMA channel, the first buffer area application information is used to record the information of the first buffer area of ​​the storage task, and the second buffer area application information is used to record the information of the second buffer area of ​​the storage task.

[0053] Optionally, the configuration information further includes a source address, a destination address and a direction of copying of each sub-source data, where the direction includes copying from a source device to a target device and / or copying from a target device to a source device.

[0054] In the subsequent operation process of step S203, the information included in the configuration information can be used to guide the thread to perform corresponding copy operations.

[0055] The task status is used to filter tasks that meet a status during the operation process. Taking the task in the first status as an example, the first status represents that the sub-source data is in the memory of the source data. The task including the sub-source data is copied from the memory of the source device to the first cache area to realize the copying of the sub-source data to the first cache area. If there is no task in the first status in the task queue, it does not need to be executed, which can represent that the sub-source data has been copied from the memory of the source device to the first cache area.

[0056] The data size is used to guide the application of a DMA channel that can transfer or better transfer the data size, thereby improving the efficiency of data transfer.

[0057] The first cache area application information is used to record the information of the first cache area for storing tasks including sub-source data. After recording the information of a first cache area, for the corresponding operation, the first cache area has been occupied and cannot be applied for and used at this time. Subsequently, if the first cache area is used up, the information of the first cache area can be removed from the first cache area application information to realize the release of the first cache area, that is, the first cache area can be applied for and used at this time.

[0058] The second cache area application information is used to record the information of the second cache area for storing tasks including sub-source data. After recording the information of a second cache area, for the corresponding operation, the second cache area has been occupied and cannot be applied for and used at this time. Subsequently, if the second cache area is used up, the information of the second cache area can be removed from the second cache area application information to realize the release of the second cache area, that is, the second cache area can be applied for and used at this time.

[0059] For example, the configuration information of the task corresponding to the sub-source data includes the task state, the target address, the source address, the data size, the data copy direction, the buffer area information of the source device (i.e., the first buffer area application information), and the buffer area information of the target device (i.e., the second buffer area application information), wherein the task state is used to characterize the state of the task during the execution process, and may include a waiting state, an initialization state, a source device copying, a source device copying completed, a DMA channel copying, a DMA channel copying completed, a target device copying, a target device copying completed, etc., wherein the first state may include a waiting state and an initialization state, and the second state may include a source device copying completed state, etc. The third state may include states such as DMA channel copy completion, the fourth state may include states such as target device copy completion, the fifth state is included before the second state, the fifth state is the source device copying, the sixth state is included before the third state, the sixth state is the DMA channel copying, the seventh state is included before the fourth state, the seventh state is the target device copying; the data copy direction is from the source device to the target device or from the target device to the source device, the target address, source address, data size and data copy direction are configurable when the above decomposition creates the task; the source device's cache information and the target device's cache information are for subsequent execution of tasks. Refer to the following table:

[0060]

[0061] Step S203: based on the configuration information of at least one sub-source data, the tasks in the task queue are transmitted to the target device.

[0062] Among them, for any sub-source data, according to the configuration information of the sub-source data, the task including the sub-source data can be transmitted to the target device. Specifically, the source address and target address of the task can be obtained according to the configuration information, so as to copy the task to the first buffer area on the source device side, apply for the corresponding DMA channel, and copy the task stored in the first buffer area to the second buffer area on the target device side. Subsequently, the target device copies the task stored in the second buffer area to realize the transmission from the sub-source device to the target device.

[0063] The configuration information of the task can ensure that each task is independent. All tasks will be placed in the task queue. The operations on the source device and the target device only need to query the configuration information of the task to guide the corresponding operations to realize data transmission.

[0064] In the embodiment of the present application, when the source device is connected to the target device through the DMA channel constructed by PCIe and the source data stored in the source device is transmitted to the target device, the source device decomposes the source data into at least one sub-source data according to the capacity of the first buffer area in the source device, forms a task queue according to the at least one sub-source data, and the task queue includes at least one task, and the task corresponds to the sub-source data one by one. Each task includes each sub-source data and the configuration information of each sub-source data. Based on the configuration information of at least one sub-source data, the sub-source data in the task queue is transmitted to the target device, and the target device receives and processes the sub-source data in the task queue. Among them, the source data is decomposed to obtain the sub-source data and the task queue is formed through the configuration information. Based on the configuration information, the sub-source data is copied to the target device by calling the corresponding operations respectively, so as to ensure that the transmission process is executed independently and does not affect each other, and parallelization can be implemented to improve the transmission efficiency.

[0065] See also Figure 3 , is a flow chart of a PCIe-based data transmission method provided in the second embodiment of the present application. The second embodiment is an improvement made on the basis of step S203 in the first embodiment above, such as Figure 3 , the data transmission method further includes:

[0066] Step S301, using each idle first thread, selects a first task in a first state from the task queue, and uses the first thread to copy the first task to a target first cache area, and after the copying is completed, changes the task state of the first task to a second state, releases the first thread, until there is no task in the first state in the task queue.

[0067] In the present application, the first thread may refer to a task copy thread set on a source device, which first applies for a first cache area and writes the information of the target first cache area into the configuration information in the task, and then subsequently copies the task to the applied first cache area, and after the task area in the first cache area is copied, the thread using the copy task releases the first cache area so that it can be applied for and used by other idle first threads when executing other copy tasks. When the first thread runs, the first cache area applied for from all cache areas of the source device is written with the information of the first cache area in the configuration information of the task corresponding to the sub-source data, that is, the cache area corresponding to the first cache area information in the configuration information is identified as the target first cache area, which can indicate that other first threads know that the first cache area is occupied, that is, in use, and thus cannot apply for the first cache area, and the release of the first cache area can be achieved by deleting the tasks stored in the first cache area.

[0068] The first task in the first state may refer to the task corresponding to the sub-source data that has not been copied to the first buffer area. The first state may be characterized as the state in which the task has not been copied to the first buffer area (such as the waiting state, initialization state, etc. in the above embodiment 1). Subsequently, if the task has been copied to the first buffer area, the task state of the sub-source data is changed to the second state. The second state may represent the state in which the sub-source data is copied to the first buffer area (such as the source device copy completion state in the above embodiment 1). Thus, data of different task states may be distinguished in the task queue to facilitate thread calling. In addition, during the task copying process, the first state is changed to the fifth state, and the fifth state is the source device copying the task to the first buffer area.

[0069] The first thread can be one or more. When a thread is not in use, it is idle. The idle thread can find a task that meets the conditions (i.e., the first task in the first state) from the task queue, and then enter the execution process of the task. After the execution is completed, the first thread can be released to process the next task in the task queue.

[0070] Since there may be one or more first cache areas in the source device, when a task is executed, a first cache area will be applied for on the source device. The applied first cache area is not currently occupied, so the corresponding data can be copied to the applied first cache area.

[0071] For example, there are two first threads, and the two first threads can copy the sub-source data in two initial states in the task queue at the same time, including applying for a first buffer area respectively and copying the corresponding sub-source data to the corresponding first buffer area. If there is only one first buffer area available at this time, only one of the two threads can apply for the available first buffer area and perform the copying.

[0072] Step S302, using each idle second thread, selects a second task in the second state from the target first cache area, and copies the second task to a target second cache area on the target device side through an applied DMA channel. After the copying is completed, the task state of the second task is changed to the third state, and the target first cache area and the second thread where the second task is located are released until there is no task in the second state in the target first cache area.

[0073] In this application, when executing a task, after copying the task to the target first buffer, it is necessary to apply for an idle second thread, and select a task in the second state from all the target first buffers through the second thread, and then apply for a DMA channel to copy the task of the target first buffer to the target device through the DMA channel, and apply for an unoccupied target second buffer area on the target device side through the second thread after the DMA channel is applied, and write the information of the target second buffer area into the configuration information of the selected task, so as to realize the data transmission such as the copy of the DMA channel. Specifically, after applying for the DMA channel, the second buffer area on the target device side can be applied, and the information of the second buffer area can be written into the configuration information of the corresponding task, which can indicate that other second threads know that the DMA channel and the second buffer area are occupied, that is, in use, and thus cannot apply for the DMA channel and the second buffer area; releasing the DMA channel and the target second buffer area can be achieved by deleting the task corresponding to the sub-source data in the target second buffer area. At this time, the triggering conditions for releasing the DMA channel and the second buffer area are the same, and both of them are the sub-source data of the second buffer area are copied out.

[0074] The second thread may refer to a DMA thread that executes a task, which is responsible for applying for a DMA channel and a second buffer area, and then copying the data in the first buffer area to the second buffer area, and releasing the first buffer area after the copy is completed. After the DMA thread is finished, it will obtain the task that meets the conditions from the target first buffer area again, and then enter the execution process of the task. The task in the second state may refer to the task copied to the first buffer area, and the second state may be characterized as the completion of the source device copy. Subsequently, if the task is copied to the second buffer area, the task state of the task is changed to the third state, and the third state may represent the state in which the sub-source data is copied to the second buffer area (such as the target device copy completion state in the above-mentioned embodiment 1). In addition, during the task copying process, the second state is changed to the sixth state, and the fifth state is the source device buffer copying task to the target device buffer.

[0075] The second thread can be one or more. When a thread is not in use, it is idle. The idle thread can find a task that meets the conditions (i.e., a task in the second state) from the task queue, and then enter the execution process of the task. After the execution is completed, the second thread can process the next task in the task queue. Among them, the DMA thread sends a corresponding application, i.e., a cache application, to the target device. After the cache application is received by the target device, it is used to apply for a second cache at the target device. The capacity of the second cache is not less than the capacity of the first cache. There may be caches of different types and sizes in the target device. In order to copy the data of the first cache to the second cache, the capacity of the second cache is required to be equal to or larger than the capacity of the first cache. In addition, due to the nature of the DMA channel, the first cache and the second cache are also required to be continuous in physical space.

[0076] Among them, each time the sub-source data in the first cache area is copied to the second cache area, the data in the first intermediate cache area can be discarded and deleted, thereby releasing the first cache area, and the released first cache area can be called by the first thread corresponding to the next task, that is, using the source device copy thread to copy the corresponding task to the released first cache area.

[0077] As for DMA channels, since their number is limited, after the sub-source data in the first buffer area is copied to the second buffer area, the transmission process of the DMA channel corresponding to the task ends, and the execution status of the update task is DMA channel copy completed.

[0078] Of course, in this process, the source device needs to exchange information with the target device to confirm the usage status of the DMA channel. After the second cache area of ​​the target device confirms that the copying is completed, the usage completion status information of the DMA channel can be generated, proving that the usage status of the DMA channel is completed and can be released. Specifically, the updated status is the task status of each sub-source data in the task queue.

[0079] Step 303, using each idle third thread to select a third task in the third state from the target second cache area, and using the third thread to copy the third task to the target area of ​​the target device, and after the copying is completed, using the third thread to update the task state of the third task to the fourth state until there are no tasks in the third state in the target second cache area, and release the target second cache area corresponding to the third task and the DMA channel corresponding to the third task.

[0080] In the present application, there may be multiple idle threads for performing task transmission each time.

[0081] In the present application, the third thread is used to copy the data stored in the target second cache area to the target address. By selecting the task in the third state in the task queue, the data stored in the second cache area can be determined.

[0082] After each task in the second buffer area is copied to the target area, the task in the second buffer area can be discarded and deleted, thereby releasing the second buffer area, and the released second buffer area can be called when the next task is executed to the application buffer area. At this time, the execution status of the task is updated to target device copy completion, and the application second buffer area can be released.

[0083] The determination and release of the second buffer area can refer to the description of the above step S302, which will not be repeated here. The fourth state may refer to the state corresponding to the sub-source data being copied from the second buffer area to the target area of ​​the target device (such as the target device copying completion state in the above embodiment 1).

[0084] In addition, each time the data in the second buffer area is copied to the target area, it can be determined that the corresponding data in the second buffer area has been copied out, so as to guide the release of the DMA channel in the source device. Wherein, the data in the second buffer area has been copied (that is, completely copied to the target area), and the corresponding third thread can be released and can be used when the next task is executed.

[0085] like Figure 4 As shown, it is a schematic diagram of data interaction between a source device and a target device provided in the second embodiment of the present application, wherein the Host is the source device and the Device is the target device, and a physically continuous intermediate buffer area is constructed between the Host and the Device through PCIe. First, the Host Copy Worker is responsible for copying the tasks in the task queue to the intermediate buffer area on the Host side. Then, after the DMA Worker finds that there is a task in the intermediate buffer area on the Host side, it applies for an intermediate buffer area on the Device side, and then copies the data in the intermediate buffer area on the Host side to the intermediate buffer area on the Device side through the DMA channel. Finally, the Device Copy Worker is responsible for copying the data in the intermediate buffer area of ​​the Device to the target address. Among them, the HostCopy Worker, DMA Worker and Device Copy Worker are all controlled by the source device.

[0086] For example, in the Linux system, due to system limitations, the maximum size of the intermediate cache area that can be applied for at one time is 4M. You can apply for multiple blocks, but considering the number and size of the cache areas, you can apply for 1M*4.

[0087] exist Figure 4In the example, each Task is divided into three steps: Host Copy Worker, DMA Worker, and Device CopyWorker. Two Workers are used to work in parallel for each step, that is, two Tasks are executed at the same time. After the Worker starts working, it will go to the task queue to obtain the Task that meets the conditions and apply for the Task's resources (i.e., the intermediate buffer area, DMA channel, etc.). After completing the Task, the Task status is updated. In order to ensure that Workers can work in a pipeline, N intermediate buffer areas on both sides need to be allocated (N>=4). After all tasks are completed, the data copy is completed.

[0088] like Figure 5 As shown, a schematic diagram showing the relationship between time and thread in task execution provided by Example 2 of the present application is provided, wherein the above-mentioned three steps of Host Copy Worker, DMA Worker and Device Copy Worker are defined as thread 1, thread 2 and thread 3. For the same task, after being processed by the previous thread, it enters the processing of the next thread. For example, for the four tasks of Task A, Task B, Task C, and Task D, at time 1, all tasks can be processed by thread 1. At this time, thread 1 takes a Task A for processing. When threads 2 and 3 take the tasks, all tasks have not yet reached this stage, so they do not meet the requirements. At time 2, Task A can be processed by thread 2, and thread 1 takes the remaining task Task B from the task queue. From the perspective of the pipeline, at time 3, the three threads are all working, which is equivalent to completing a task in one time period. Each task is maintained in combination with the state of the task, so that the task can be processed in parallel and pipelined, and the resources of multiple DMA channels and multiple intermediate buffers are effectively utilized.

[0089] Still with the above Figure 4 For example, for the three types of workers, Host Copy Worker, DMA Worker and Device CopyWorker, each is responsible for different tasks, namely Host copy, PCIe DMA copy and Device copy. The task status is divided into 7 states according to the completed steps: initialization state, Host copy in progress, Host copy completed, DMA copy in progress, DMA copy completed, Device copy in progress and Device copy completed. Host copy and Device copy are mainly data copies between user data areas and intermediate buffer areas. N intermediate buffer areas need to be dynamically managed. After being applied for by Workers, they are in an occupied state and wait for the data to be copied from the intermediate buffer areas before the intermediate buffer areas are released.

[0090] If the underlying driver of PCIe has 4 DMA channels, DMA copies are directly copied from the intermediate buffer area of ​​the Host to the intermediate buffer area of ​​the Device. A DMA channel needs to be applied for. The most suitable channel is selected for transmission based on ch_mask and the amount of data requests received by the DMA channel. The channel maintains a request queue and makes a selection based on the amount of requests in the queue to achieve load balancing and optimal transmission efficiency.

[0091] The principle of application and release is that the thread in need is responsible for application, and after completing certain conditions, the last thread that has finished using it is responsible for release. In thread 1, the task of the source device is copied to the intermediate buffer area of ​​the source device (it is necessary to apply for the intermediate buffer area of ​​the source device), and the address information of the intermediate buffer area is saved in the task information so that thread 2 can use it. At this time, thread 1 completes the copy. In thread 2, DMA copy is copied from the intermediate buffer area of ​​the source device to the intermediate buffer area of ​​the target device (the intermediate buffer area of ​​the target device is applied for according to the intermediate buffer area of ​​the source device), and the address information of the intermediate buffer area of ​​the target device is saved in the task information. At this time, thread 2 completes the copy. Since the sub-source data of the source device has been copied to the intermediate buffer area of ​​the target device, there is no need to use the corresponding intermediate buffer area of ​​the source device, and it can be released. In thread 3, Device copy copies the data of the intermediate buffer area of ​​the target device to the target address. At this time, thread 3 completes the copy, and the execution status of the task is updated to the target device copy completed. There is no need to continue to use the intermediate buffer area of ​​the target device, and it can be released. In addition, the DMA channel corresponding to the intermediate buffer area can be released.

[0092] Optionally, after the third thread is used to update the task state of the third task to the fourth state, the method further includes:

[0093] The third thread is used to remove the task corresponding to the third task in the fourth state from the task queue.

[0094] Among them, for tasks whose copy operations have been completed, they can be removed from the task queue.

[0095] The embodiment of the present application uses each idle first thread to select a first task in a first state from the task queue, and uses the first thread to copy the first task to a target first cache area, and changes the task state of the first task to a second state after the copying is completed, and releases the first thread until there is no task in the first state in the task queue; uses each idle second thread to select a second task in a second state from the target first cache area, and copies the second task to a target second cache area on the target device side through an applied DMA channel, and changes the task state of the second task to a third state after the copying is completed, releases the target first cache area where the second task is located and the second thread, until there is no task in the second state in the target first cache area. ; Use each idle third thread to select a third task in the third state from the target second cache area, and use the third thread to copy the third task to the target area of ​​the target device, and after the copying is completed, use the third thread to update the task state of the third task to the fourth state, until there is no task in the third state in the target second cache area, decompose the source data to obtain multiple tasks, call corresponding operations to copy the tasks to the cache area of ​​the source device, then copy from the cache area of ​​the source device to the cache area of ​​the target device, and then copy from the cache area of ​​the target device to the target area, based on different threads to complete the sequential transmission of sub-source data, the transmission process is independently executed and does not affect each other, and parallelization can be implemented to improve the transmission efficiency, which effectively improves the utilization rate of the cache area and the DMA channel.

[0096] Corresponding to the PCIe-based data transmission method in the first embodiment above, Figure 6 The structural block diagram of the PCIe-based data transmission device provided in the third embodiment of the present application is shown. The data transmission device is applied to the source device, and the computer device corresponding to the source device is connected to the target device to perform data interactive transmission. The source device is connected to the target device through the DMA channel constructed by PCIe. The number of DMA channels is related to the underlying driver of PCIe and the system architecture of the source device and the target device, and can be specifically configured according to the needs.

[0097] This data transfer method relies on the interaction between the source device and the target device. If the data transfer method runs in the source device, see Figure 6 , the data transmission device comprises:

[0098] The task decomposition module 61 is used to decompose the source data into at least one sub-source data according to the capacity of the first buffer area in the source device;

[0099] A task queue forming module 62, configured to form a task queue according to at least one sub-source data, wherein the task queue includes at least one task, the at least one task corresponds to at least one sub-source data in a one-to-one manner, and each task includes each sub-source data and configuration information of each sub-source data;

[0100] The data transmission module 63 is used to transmit the tasks in the task queue to the target device based on the configuration information of at least one sub-source data.

[0101] Optionally, the task decomposition module 61 includes:

[0102] A decomposition number determining unit, used to determine the decomposition number according to the data volume of the source data and the capacity of the first buffer area;

[0103] A storage address acquisition unit, used to acquire the storage address of source data;

[0104] A data decomposition unit, used to evenly decompose the storage address using the decomposition number to obtain at least one source address and a target address corresponding to each source address;

[0105] The sub-source data determining unit is used to determine that the data between each source address and the corresponding target address is the corresponding sub-source data, so as to obtain at least one sub-source data.

[0106] Optionally, the configuration information includes at least one of a task status, a data size, a first buffer area application information and a second buffer area application information, the task status is used to record the first state, the second state and the third state of the sub-source data in the task, the data size is used to guide the application of the DMA channel, the first buffer area application information is used to record the information of the first buffer area storing the sub-source data, and the second buffer area application information is used to record the information of the second buffer area storing the sub-source data.

[0107] Optionally, the data transmission module 63 includes:

[0108] A first transmission unit is used to select a first task in a first state from a task queue using each idle first thread, and use the first thread to copy the sub-source data in the first task to a target first buffer area, and change the task state of the first task to a second state after the copying is completed, and release the first thread until there is no task in the first state in the task queue, and the target first buffer area is a buffer area corresponding to the first buffer area information in the configuration information included in the first task;

[0109] The second transmission unit is used to select a second task in a second state from the task queue using each idle second thread, and copy the sub-source data stored in the first buffer area corresponding to the first buffer area information in the second task to a second buffer area on the target device side through an applied DMA channel, and after the copying is completed, change the task state of the second task to a third state, release the corresponding first buffer area and the second thread, until there is no sub-source data in the second state in the task queue;

[0110] The third transmission unit is used to use each idle third thread to select a third task in a third state from the target second cache area, and use the third thread to copy the third task to the target area of ​​the target device, and after the copying is completed, use the third thread to update the task state of the third task to the fourth state until there is no task in the third state in the target second cache area, and release the target second cache area corresponding to the third task and the DMA channel corresponding to the third task.

[0111] Optionally, the data transmission device further includes:

[0112] The task queue updating module is used to remove the task corresponding to the sub-source data in the fourth state from the task queue after the task state of the third task is updated to the fourth state by the third thread.

[0113] It should be noted that the information interaction, execution process, etc. between the above-mentioned modules, units, and sub-units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0114] Figure 7 This is a schematic diagram of the structure of a computer device provided in Example 4 of the present application. Figure 7 As shown, the computer device of this embodiment includes: at least one processor ( Figure 7 Only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps in any of the above-mentioned data transmission method embodiments are implemented.

[0115] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 7 The above is only an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a PCIe interface, a display screen, and an input device.

[0116] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0117] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of an operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments may also be an external storage device of a computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on a computer device. Further, the memory may also include both an internal storage unit of a computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of computer programs, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0118] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiment when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0119] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the steps in the above-mentioned method embodiment can be implemented when the computer device executes it.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0122] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data transmission method based on PCIe, characterized in that: The data transmission method is run on a source device, the source device is connected to a target device via a DMA channel constructed by PCIe, the source device stores source data to be transmitted, and the data transmission method includes: Decomposing the source data into at least one sub-source data according to the capacity of the first buffer area in the source device; According to the at least one sub-source data, a task queue is formed, wherein the task queue includes at least one task, the at least one task corresponds to the at least one sub-source data one by one, and each of the tasks includes each of the sub-source data and configuration information of each of the sub-source data; Based on the configuration information of the at least one sub-source data, the tasks in the task queue are transmitted to the target device.

2. The data transmission method according to claim 1, characterized in that: The step of decomposing the source data into at least one sub-source data according to the capacity of the first buffer area in the source device comprises: Determining the decomposition number according to the data volume of the source data and the capacity of the first buffer area; Obtaining a storage address of the source data; Using the decomposition number to evenly decompose the storage address, obtaining at least one source address and a target address corresponding to each source address; The data between each of the source addresses and the target address corresponding to each of the source addresses is determined as a sub-source data to obtain at least one sub-source data.

3. The data transmission method according to claim 1 or 2, characterized in that: The configuration information includes at least one of a task status, a data size, a first buffer area application information and a second buffer area application information. The task status is used to record the first state, the second state and the third state of the sub-source data in the task. The data size is used to guide the application of a DMA channel. The first buffer area application information is used to record the information of the first buffer area storing the sub-source data. The second buffer area application information is used to record the information of the second buffer area storing the sub-source data.

4. The data transmission method according to claim 3, characterized in that: The step of transmitting the task in the task queue to the target device based on the configuration information of the at least one sub-source data comprises: Using each idle first thread, selecting a first task in a first state from the task queue, and using the first thread to copy the first task to a target first cache area, and changing the task state of the first task to a second state after the copying is completed, releasing the first thread, until there is no task in the first state in the task queue, and the target first cache area is the cache area corresponding to the first cache area information in the configuration information included in the first task; Using each idle second thread, selecting a second task in the second state from the target first buffer area, and copying the second task to a target second buffer area on the target device side through an applied DMA channel, changing the task state of the second task to a third state after the copying is completed, releasing the target first buffer area where the second task is located and the second thread, until there is no task in the second state in the target first buffer area; Using each idle third thread, select a third task in the third state from the target second cache area, and use the third thread to copy the third task to the target area of ​​the target device, and after the copying is completed, use the third thread to update the task state of the third task to the fourth state until there is no task in the third state in the target second cache area, and release the target second cache area corresponding to the third task and the DMA channel corresponding to the third task.

5. The data transmission method according to claim 4, characterized in that: After the third thread is used to update the task state of the third task to the fourth state, the method further includes: The third thread is used to remove the task corresponding to the third task in the fourth state from the task queue.

6. A data transmission device based on PCIe, characterized in that: The data transmission device is operated on a source device, the source device is connected to a target device via a DMA channel constructed by PCIe, the source device stores source data to be transmitted, and the data transmission device includes: A task decomposition module, configured to decompose the source data into at least one sub-source data according to the capacity of the first buffer area in the source device; A task queue forming module, used for forming a task queue according to the at least one sub-source data, wherein the task queue includes at least one task, the at least one task corresponds to the at least one sub-source data one by one, and each of the tasks includes each of the sub-source data and configuration information of each of the sub-source data; A data transmission module is used to transmit the tasks in the task queue to the target device based on the configuration information of the at least one sub-source data.

7. A computer device, characterized in that: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the data transmission method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the data transmission method according to any one of claims 1 to 5 is implemented.

Images