Parallel GPU asynchronous flow management method and device

Abstract

The embodiments of this specification provide a parallel GPU asynchronous stream management method and device, wherein the parallel GPU asynchronous stream management method includes: obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result. By obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result, multiple CUDA streams can be initiated to the GPU for synchronous execution at the same time, and multiple tasks can be executed in parallel, making full use of GPU resources and improving efficiency.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of data processing technology, and in particular to a parallel GPU asynchronous flow management method. Background Art

[0002] The sampled data of high-speed digital signals before demodulation can reach several GBytes to hundreds of GBytes. When using GPU to complete data demodulation, the data needs to be copied from the host memory to the specified input position of the GPU shared cache in the sampling order, and then the GPU kernel function is executed to store the result to the specified output position of the GPU shared cache. Finally, the result is copied from the specified output position of the GPU shared cache to the host memory to complete the entire demodulation process. Usually, the sampled data is sliced ​​according to a certain size, and each slice is a fixed size (such as 10MByte). The main program schedules the GPU to process each slice of data in the order of reception. Since the GPU itself is a parallel processing device, when multiple processes are initiated for simultaneous processing, it is impossible to ensure that the first scheduled execution process must end first. Therefore, in order to ensure that the processing process follows the timing of data reception, the next process must be started after the previous process is completed, which results in the GPU process being scheduled serially. Under the working principle of GPU, the steps of input data copying, kernel function execution, and output data copying do not interfere with each other and can be executed synchronously and in parallel. If the above process is adopted, these three steps can only be processed serially, which seriously wastes GPU resources. The GPU also has a high time cost when applying for shared memory, especially the first time application, which can take up to ten to dozens of milliseconds. If each scheduling requires a shared memory application and release operation, it will also seriously affect the GPU performance. Therefore, a better solution is urgently needed. Summary of the invention

[0003] In view of this, the embodiments of this specification provide a parallel GPU asynchronous flow management method. One or more embodiments of this specification also relate to a parallel GPU asynchronous flow management device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a parallel GPU asynchronous flow management method is provided, comprising:

[0005] Get sampling data and determine the number of idle flows;

[0006] Release GPU task scheduling based on sampling data and the number of idle streams, and determine the scheduling results;

[0007] Update the number of idle flows based on the scheduling result, and determine the updated number of idle flows;

[0008] Stream synchronization is performed based on the updated number of idle streams, and the synchronization result is determined.

[0009] In a possible implementation, GPU task scheduling is issued based on the sampled data and the number of idle flows, and a scheduling result is determined, including:

[0010] When the number of idle streams is not 0, determining the idle stream offset position;

[0011] GPU tasks are scheduled based on the idle stream offset position to determine the scheduling result.

[0012] In a possible implementation, GPU task scheduling is performed based on the idle stream offset position, and a scheduling result is determined, including:

[0013] Determine a first target CUDA stream based on the idle stream offset position;

[0014] determining a target input block and a target output block based on the first target CUDA stream;

[0015] GPU tasks are scheduled based on the target input block and the target output block, and a scheduling result is determined.

[0016] In a possible implementation, updating the number of idle flows based on the scheduling result, and determining to update the number of idle flows includes:

[0017] When the scheduling result is completed, the number of idle flows is subtracted by 1 to determine the updated number of idle flows; and the idle flow offset position is updated.

[0018] In a possible implementation, updating the idle stream offset position includes:

[0019] Determine first order information of a first target CUDA stream;

[0020] If the first sequence information is the last CUDA stream, the idle stream offset position is pointed to the first CUDA stream;

[0021] If the first sequence information is not the last CUDA stream, the idle stream offset position is pointed to the next CUDA stream.

[0022] In a possible implementation, performing stream synchronization based on updating the number of idle streams and determining a synchronization result includes:

[0023] Determine a flow number threshold, and determine a task status based on the updated idle flow number and the flow number threshold;

[0024] Determine a second target CUDA stream based on the task state and the workflow offset position;

[0025] Perform stream synchronization based on the second target CUDA stream and determine a synchronization result.

[0026] In a possible implementation, after performing stream synchronization based on updating the number of idle streams and determining a synchronization result, the method further includes:

[0027] The number of updated idle streams is increased by 1, and the second order information of the second target CUDA stream is determined;

[0028] If the second sequence information is the last CUDA stream, the workflow offset position is pointed to the first CUDA stream;

[0029] If the second sequence information is not the last CUDA stream, the workflow offset position is pointed to the next CUDA stream.

[0030] According to a second aspect of an embodiment of this specification, a parallel GPU asynchronous flow management device is provided, comprising:

[0031] A data acquisition module is configured to acquire sampled data and determine the number of idle flows;

[0032] A scheduling result module is configured to issue GPU task scheduling based on the sampling data and the number of idle flows, and determine the scheduling result;

[0033] A data updating module is configured to update the number of idle flows based on the scheduling result, and determine to update the number of idle flows;

[0034] The data synchronization module is configured to perform stream synchronization based on updating the number of idle streams and determine the synchronization result.

[0035] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0036] Memory and processor;

[0037] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned parallel GPU asynchronous flow management method are implemented.

[0038] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned parallel GPU asynchronous flow management method are implemented.

[0039] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned parallel GPU asynchronous stream management method.

[0040] The embodiments of this specification provide a parallel GPU asynchronous stream management method and device, wherein the parallel GPU asynchronous stream management method includes: obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result. By obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result, multiple CUDA streams can be initiated to the GPU for synchronous execution at the same time, and multiple tasks can be executed in parallel, making full use of GPU resources and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a parallel GPU asynchronous flow management method provided by an embodiment of this specification;

[0042] Figure 2 It is a schematic diagram of the principle of a parallel GPU asynchronous flow management method provided by an embodiment of this specification;

[0043] Figure 3 It is a schematic diagram of the scheduling and synchronization principle of a parallel GPU asynchronous flow management method provided by an embodiment of this specification;

[0044] Figure 4 It is a structural diagram of a parallel GPU asynchronous flow management device provided by an embodiment of this specification;

[0045] Figure 5 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0046] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0047] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0048] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0049] First, the terms involved in one or more embodiments of this specification are explained.

[0050] a) GPU shared memory: The secondary cache used to store data in the GPU has the characteristics of large capacity and low speed.

[0051] b) CUDA kernel function: an operation process executed on the GPU.

[0052] c) CUDA stream: It is a series of asynchronous CUDA operation sets. In the embodiments of this specification, each data slice forms a CUDA stream for a complete operation process in the GPU. A complete CUDA stream includes three independent steps: copying data from the host to the GPU shared memory; executing the demodulation kernel function to obtain the demodulation result and storing it in the GPU shared memory; copying the result from the GPU shared memory to the host. Secondly, each CUDA stream will be bound to a data input block and a data output block. This "CUDA stream" is a non-empty stream, also known as an asynchronous stream.

[0053] d) Data slice: Sampling refers to the process of continuously receiving continuous bit stream data to the host. Therefore, the data obtained by sampling is a continuous bit stream data. This data is the data before GPU demodulation. The bit stream data needs to be sliced. Each independent slice is called a data slice. Each CUDA stream needs to complete the processing of a data slice.

[0054] e) Stream queue: A collection of multiple independent "CUDA streams". Each "CUDA stream" in the collection is scheduled and run independently without interfering with each other. The number of "CUDA streams" in the "stream queue" indicates the amount of execution parallelism.

[0055] f) Stream scheduling module: When there are idle streams, specify one of the idle CUDA streams to process the sampled data slices in sequence.

[0056] g) Stream synchronization thread: It is used to detect the execution status of CUDA streams and whether the first scheduled CUDA stream among all currently working CUDA streams has been completed. In order to ensure the processing sequence of data slices, the stream synchronization thread uses blocking waiting to synchronize.

[0057] h) Result data processing module: processes the demodulation results after the CUDA stream is executed. After the "stream synchronization thread" completes the synchronization, this module will be triggered to process the demodulation results generated by the current CUDA stream.

[0058] i) Number of Idle Streams: indicates how many idle CUDA streams there are in the "stream queue" at this time. Only when there are idle CUDA streams can the "stream scheduling module" initiate a new CUDA stream scheduling. The initial value of "Number of Idle Streams" is the total size of the "stream queue".

[0059] j) First idle stream offset position: indicates the first idle "CUDA stream" that can be scheduled in the current "stream queue". When the "number of idle streams" is not 0, the "stream scheduling module" determines the next schedulable CUDA stream through this offset position.

[0060] k) The first workflow offset position: indicates the first scheduled and working CUDA stream in the current stream queue. When the number of idle streams is less than the size of the stream queue, the stream synchronization module determines the CUDA stream that needs to be synchronized through this offset.

[0061] In this specification, a parallel GPU asynchronous flow management method is provided. This specification also relates to a parallel GPU asynchronous flow management device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0062] See also Figure 1 , Figure 1 A flowchart of a parallel GPU asynchronous flow management method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0063] Step 101: Obtain sampling data and determine the number of idle flows.

[0064] In practical applications, see Figure 2 , according to the demodulation data code rate, the current GPU processing performance and the design principle of the demodulation kernel function, the "data slice / input block" size, "output block" size and "stream queue" size are determined. The sampling "data slice" size is recorded as IBSize, which is also the "input block" size; the "output block" size is recorded as OBSize; and the "stream queue" size is recorded as N.

[0065] Furthermore, in order to reduce the efficiency reduction caused by frequent application and release of GPU resources during the process, it is necessary to apply for and initialize the required GPU resources first: apply for N "CUDA streams" of type cudaStream_t through the cudaStreamCreate function, which are recorded as stream 1 to stream N; apply for N "input blocks" through the cudaMalloc function, which are recorded as input block 1 to input block N; apply for N "output blocks" through the cudaMalloc function, which are recorded as output block 1 to output block N. In the task scheduling process, only the corresponding idle resources need to be used each time, and there is no need to apply and release them repeatedly. After all sample data demodulation is completed, the GPU resources are recovered by releasing resources uniformly. At the same time, set the "number of idle streams" to N, indicating that all current "CUDA streams" are idle streams; set the "first idle stream offset position" to 0, indicating that the first "CUDA stream" in the "stream queue" is the "CUDA stream" used for the first scheduling; set the "first workflow offset position" to 0, indicating that the first "CUDA stream" in the "stream queue" is the "CUDA stream" used for the first synchronization.

[0066] The embodiments of this specification apply for GPU resources in advance, including "CUDA stream", "input block" shared memory and "output block" shared memory, establish a "stream queue" and reuse it according to scheduling rules, without the need to apply and release each time a "data slice" demodulation is executed, thereby reducing GPU performance loss.

[0067] Step 102: Publish GPU task scheduling based on the sampling data and the number of idle flows, and determine the scheduling result.

[0068] In one possible implementation, GPU task scheduling is issued based on sampling data and the number of idle streams, and a scheduling result is determined, including: when the number of idle streams is not 0, determining the idle stream offset position; and performing GPU task scheduling based on the idle stream offset position to determine the scheduling result.

[0069] In actual applications, after receiving the sampled data and forming an independent "data slice", the "stream scheduling module" will determine whether the "idle stream number" is 0. If the "idle stream number" is not 0, it will enter the monitoring mode and monitor the value change of the "idle stream number" until the "idle stream number" is not 0. If the "idle stream number" is not 0, it means that there are currently idle CUDA streams available, that is, new GPU task scheduling can be released immediately.

[0070] In one possible implementation, GPU task scheduling is performed based on the idle stream offset position to determine a scheduling result, including: determining a first target CUDA stream based on the idle stream offset position; determining a target input block and a target output block based on the first target CUDA stream; and performing GPU task scheduling based on the target input block and the target output block to determine a scheduling result.

[0071] In actual applications, the first available idle stream is specified by the "first idle stream offset position", and the "stream scheduling module" selects the "CUDA stream" specified by the "first idle stream offset position", uses the "input block" and "output block" with the same offset position, and initiates "copy in", "kernel function", and "copy out" requests. After the request is completed, the GPU will automatically execute as required until all work is completed.

[0072] Step 103: Update the number of idle flows based on the scheduling result and determine the updated number of idle flows.

[0073] In a possible implementation, the number of idle flows is updated based on the scheduling result, and determining the updated number of idle flows includes: when the scheduling result is completed, subtracting 1 from the number of idle flows to determine the updated number of idle flows; and updating the idle flow offset position.

[0074] In actual applications, after the "stream scheduling module" completes a new stream scheduling, it is also necessary to reduce the "number of idle streams" by 1. And the idle stream offset position needs to be updated. The details are as follows.

[0075] In one possible implementation, updating the idle stream offset position includes: determining the first sequence information of the first target CUDA stream; if the first sequence information is the last CUDA stream, pointing the idle stream offset position to the first CUDA stream; if the first sequence information is not the last CUDA stream, pointing the idle stream offset position to the next CUDA stream.

[0076] In actual applications, after the "number of idle streams" is reduced by 1, the "first idle stream offset position" is moved backward to point to the next "CUDA stream". If the current "first idle stream offset position" points to the last "CUDA stream" of the "stream queue", the "first idle stream offset position" is pointed to the first "CUDA stream" of the "stream queue", and the "stream queue" is used as a circular queue during the scheduling process. After the "stream scheduling module" completes a task scheduling, it continues to process new sampled "data slices" according to the above steps until all data processing is completed.

[0077] Step 104: Perform stream synchronization based on the updated number of idle streams and determine the synchronization result.

[0078] In one possible implementation, stream synchronization is performed based on updating the number of idle streams to determine the synchronization result, including: determining a stream number threshold, determining a task state based on the updated number of idle streams and the stream number threshold; determining a second target CUDA stream based on the task state and the workflow offset position; performing stream synchronization based on the second target CUDA stream to determine the synchronization result.

[0079] In actual applications, after the "stream synchronization thread" is started, it monitors the "number of idle streams" in real time. If the "number of idle streams" is N (the total size of the "stream queue"), it will continue to monitor until the "number of idle streams" is less than N. If the "number of idle streams" is less than N, it means that there are tasks being executed on the GPU, and the "CUDA stream" in the first work is specified by the "first work stream offset position". The "stream synchronization thread" selects the "CUDA stream" specified by the "first work stream offset position" and uses the cudaStreamSynchronize function to synchronize this "CUDA stream" until the synchronization is completed. After the synchronization is completed, the "result data processing module" is notified to process the demodulation result data.

[0080] In one possible implementation, after performing stream synchronization based on the updated idle stream number and determining the synchronization result, it also includes: adding 1 to the updated idle stream number and determining the second sequence information of the second target CUDA stream; if the second sequence information is the last CUDA stream, pointing the workflow offset position to the first CUDA stream; if the second sequence information is not the last CUDA stream, pointing the workflow offset position to the next CUDA stream.

[0081] In actual applications, the "stream synchronization thread" also needs to add 1 to the "number of idle streams" and move the "first workflow offset position" backward to point to the next "CUDA stream". If the current "first workflow offset position" points to the last "CUDA stream" in the "stream queue", then point the "first workflow offset position" to the first "CUDA stream" in the "stream queue", and use the "stream queue" as a circular queue during the synchronization process. After the "stream synchronization thread" completes a synchronization, it continues to perform synchronization operations according to the above steps until it ends. The "stream synchronization thread" will always select the first scheduled and working "CUDA stream" in the "stream queue" for synchronization. In this way, it can be guaranteed that the result return of the "CUDA stream" can follow the data sampling timing and will not cause data disorder.

[0082] It should be noted that the "stream scheduling module" can initiate up to N tasks at the same time, that is, the GPU can process up to N "CUDA streams" in parallel. The "stream synchronization thread" will synchronize in the order of task scheduling to determine whether the task has been completed. When the first scheduled task in the tasks being executed in the GPU is completed, the "stream synchronization thread" will immediately notify the "stream scheduling module" through the "idle stream number", and the "stream scheduling module" will initiate a new scheduling process. The workflow of the "stream scheduling module" and the "stream synchronization thread" is as follows: Figure 3As shown. The embodiment of this specification can simultaneously initiate N (the size of the "stream queue") "CUDA streams" to the GPU for synchronous execution, and execute multiple tasks in parallel to fully utilize GPU resources. And based on the working principle of CUDA asynchronous streams, this method of parallel task scheduling can automatically balance GPU input data copying, GPU kernel function execution, and GPU output data copying, making full use of the GPU's stream processor and shared memory IO resources.

[0083] The embodiments of this specification provide a parallel GPU asynchronous stream management method and device, wherein the parallel GPU asynchronous stream management method includes: obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result. By obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result, multiple CUDA streams can be initiated to the GPU for synchronous execution at the same time, and multiple tasks can be executed in parallel, making full use of GPU resources and improving efficiency.

[0084] Corresponding to the above method embodiment, this specification also provides a parallel GPU asynchronous flow management device embodiment, Figure 4 FIG. 1 shows a schematic diagram of a parallel GPU asynchronous flow management device provided by an embodiment of the present specification. Figure 4 As shown, the device comprises:

[0085] The data acquisition module 401 is configured to acquire sampled data and determine the number of idle flows;

[0086] The scheduling result module 402 is configured to issue GPU task scheduling based on the sampled data and the number of idle flows, and determine the scheduling result;

[0087] The data updating module 403 is configured to update the number of idle flows based on the scheduling result and determine to update the number of idle flows;

[0088] The data synchronization module 404 is configured to perform stream synchronization based on the updated number of idle streams and determine a synchronization result.

[0089] In a possible implementation, the scheduling result module 402 is further configured to:

[0090] When the number of idle streams is not 0, determining the idle stream offset position;

[0091] GPU tasks are scheduled based on the idle stream offset position to determine the scheduling result.

[0092] In a possible implementation, the scheduling result module 402 is further configured to:

[0093] Determine a first target CUDA stream based on the idle stream offset position;

[0094] determining a target input block and a target output block based on the first target CUDA stream;

[0095] GPU tasks are scheduled based on the target input block and the target output block, and a scheduling result is determined.

[0096] In a possible implementation, the data updating module 403 is further configured to:

[0097] When the scheduling result is completed, the number of idle flows is subtracted by 1 to determine the updated number of idle flows; and the idle flow offset position is updated.

[0098] In a possible implementation, the data updating module 403 is further configured to:

[0099] Determine first order information of a first target CUDA stream;

[0100] If the first sequence information is the last CUDA stream, the idle stream offset position is pointed to the first CUDA stream;

[0101] If the first sequence information is not the last CUDA stream, the idle stream offset position is pointed to the next CUDA stream.

[0102] In a possible implementation, the data synchronization module 404 is further configured to:

[0103] Determine a flow number threshold, and determine a task status based on the updated idle flow number and the flow number threshold;

[0104] Determine a second target CUDA stream based on the task state and the workflow offset position;

[0105] Perform stream synchronization based on the second target CUDA stream and determine a synchronization result.

[0106] In a possible implementation, the data synchronization module 404 is further configured to:

[0107] The number of updated idle streams is increased by 1, and the second order information of the second target CUDA stream is determined;

[0108] If the second sequence information is the last CUDA stream, the workflow offset position is pointed to the first CUDA stream;

[0109] If the second sequence information is not the last CUDA stream, the workflow offset position is pointed to the next CUDA stream.

[0110] The embodiments of this specification provide a parallel GPU asynchronous stream management method and device, wherein the parallel GPU asynchronous stream management device includes: obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result. By obtaining sampling data to determine the number of idle streams; issuing GPU task scheduling based on the sampling data and the number of idle streams to determine the scheduling result; updating the number of idle streams based on the scheduling result to determine the updated number of idle streams; performing stream synchronization based on the updated number of idle streams to determine the synchronization result, multiple CUDA streams can be initiated to the GPU for synchronous execution at the same time, and multiple tasks can be executed in parallel, making full use of GPU resources and improving efficiency.

[0111] The above is a schematic scheme of a parallel GPU asynchronous flow management device of this embodiment. It should be noted that the technical scheme of the parallel GPU asynchronous flow management device and the technical scheme of the parallel GPU asynchronous flow management method described above belong to the same concept, and the details of the technical scheme of the parallel GPU asynchronous flow management device that are not described in detail can all be referred to the description of the technical scheme of the parallel GPU asynchronous flow management method described above.

[0112] Figure 5 The structure block diagram of a computing device 500 provided according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0113] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0114] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0115] The computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 500 may also be a mobile or stationary server.

[0116] The processor 520 is used to execute the following computer executable instructions, which implement the steps of the above-mentioned parallel GPU asynchronous flow management method when executed by the processor. The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned parallel GPU asynchronous flow management method belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can all be referred to the description of the technical scheme of the above-mentioned parallel GPU asynchronous flow management method.

[0117] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned parallel GPU asynchronous stream management method.

[0118] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned parallel GPU asynchronous flow management method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned parallel GPU asynchronous flow management method.

[0119] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned parallel GPU asynchronous stream management method.

[0120] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned parallel GPU asynchronous stream management method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above-mentioned parallel GPU asynchronous stream management method.

[0121] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0122] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0123] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0124] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0125] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A parallel GPU asynchronous flow management method, characterized in that: include: Get sampling data and determine the number of idle flows; Issue GPU task scheduling based on the sampled data and the number of idle flows, and determine a scheduling result; Update the number of idle flows based on the scheduling result, and determine to update the number of idle flows; Performing stream synchronization based on the updated number of idle streams, and determining a synchronization result; The issuing of GPU task scheduling based on the sampled data and the number of idle flows and determining a scheduling result includes: When the number of idle streams is not 0, determining an idle stream offset position; Perform GPU task scheduling based on the idle stream offset position and determine a scheduling result; The GPU task scheduling based on the idle stream offset position and determining the scheduling result includes: Determine a first target CUDA stream based on the idle stream offset position; Determine a target input block and a target output block based on the first target CUDA stream; Perform GPU task scheduling based on the target input block and the target output block, and determine a scheduling result; The updating of the number of idle flows based on the scheduling result and determining the updated number of idle flows comprises: When the scheduling result is completed, subtract 1 from the number of idle flows to determine the updated number of idle flows; and update the idle flow offset position; The updating of the idle stream offset position comprises: Determine first sequence information of the first target CUDA stream; If the first sequence information is the last CUDA stream, the idle stream offset position is pointed to the first CUDA stream; If the first sequence information is not the last CUDA stream, the idle stream offset position is pointed to the next CUDA stream.

2. The method according to claim 1, characterized in that The performing stream synchronization based on the updating of the idle stream number and determining the synchronization result comprises: Determine a flow number threshold, and determine a task state based on the updated idle flow number and the flow number threshold; Determine a second target CUDA stream based on the task state and the workflow offset position; Perform stream synchronization based on the second target CUDA stream and determine a synchronization result.

3. The method according to claim 2, characterized in that After performing stream synchronization based on the updating of the idle stream number and determining the synchronization result, the method further includes: Add 1 to the updated idle stream number, and determine the second sequence information of the second target CUDA stream; If the second sequence information is the last CUDA stream, the workflow offset position is pointed to the first CUDA stream; If the second sequence information is not the last CUDA stream, the workflow offset position is pointed to the next CUDA stream.

4. A parallel GPU asynchronous flow management device, used to implement the steps of the parallel GPU asynchronous flow management method according to any one of claims 1 to 3, characterized in that: include: A data acquisition module is configured to acquire sampled data and determine the number of idle flows; A scheduling result module is configured to issue GPU task scheduling based on the sampled data and the number of idle flows, and determine a scheduling result; A data updating module, configured to update the number of idle flows based on the scheduling result, and determine to update the number of idle flows; The data synchronization module is configured to perform stream synchronization based on the updated number of idle streams and determine a synchronization result.

5. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the parallel GPU asynchronous stream management method described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the parallel GPU asynchronous stream management method according to any one of claims 1 to 3.

Images