Radar signal processing implementation method based on CPU and GPU isomerism

Abstract

The invention discloses a radar signal processing implementation method based on CPU and GPU isomerism, and belongs to the technical field of radar signal processing. A multi-module asynchronous multi-stream parallel architecture is constructed through a CPU and a GPU, the throughput time of the whole radar signal processing flow is determined by the maximum processing time of a single module, a management thread and an external interaction thread are started at the CPU end, a communication thread is started at the CPU end, echo data receiving and external terminal control are achieved, and the multi-module asynchronous multi-stream parallel architecture is achieved. The kernel functions of all modules of all GPU ends are scheduled through a management thread, CPU thread scheduling in one-to-one correspondence with the kernel functions of the modules of the GPU ends is started after echo data are written into a GPU video memory through the management thread, and pipeline parallel processing of a plurality of processing modules is achieved through control over the processing thread of each module and the kernel functions of all the modules of the GPU ends. The multi-stage pipeline parallel execution of a radar signal processing algorithm is realized, the throughput time of a single module is prolonged, the data transmission delay is effectively reduced, the resource utilization rate is optimized, and the real-time requirement of radar signal processing is met.

Description

Technical Field

[0001] The present invention relates to a method for implementing radar signal processing based on heterogeneous CPU and GPU, and belongs to the technical field of radar signal processing. Background Art

[0002] In recent years, the complexity of radar signal processing algorithms has increased significantly with the improvement of target detection accuracy, anti-interference capabilities, and multi-tasking processing requirements. Traditional FPGA+DSP-based radar systems have long development cycles, poor scalability, and limited floating-point computing capabilities, making it difficult to meet the real-time requirements of radar. Existing radar processing systems based on heterogeneous CPUs and GPUs use the CPU as a control and scheduling device, responsible for serial logic function implementation, external communication, and scheduling GPU kernel function execution. The GPU's super floating-point computing capabilities and large-scale parallel thread architecture are used to undertake high-density computing tasks, providing a new solution for radar signal processing. However, existing radar processing systems based on heterogeneous CPUs and GPUs still face the following problems in actual signal processing applications: a) Insufficient collaborative optimization of CPU and GPU heterogeneous architectures results in data transmission efficiency and task scheduling strategies between the CPU and GPU affecting the overall performance of the radar signal processing system. b) The pipeline parallelism of heterogeneous CPU and GPU architectures has not been fully exploited: The current serial processing method of multi-module pipelines results in weak data throughput capacity of the processing flow, which is not conducive to efficient processing of radar signals in multiple scenarios; c) CPU and GPU heterogeneous architectures have weak adaptability to non-uniform scenarios: Under complex processing timing, the common processing flows of CPU and GPU heterogeneous architectures are difficult to meet real-time processing requirements. Summary of the Invention

[0003] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and provide a method for fully utilizing CPU and GPU heterogeneous hardware platforms to build a modular heterogeneous parallel architecture. Through collaborative task scheduling between the CPU and GPU, multi-stage pipeline parallel execution of radar signal processing algorithms is achieved, effectively reducing data transmission delays and optimizing resource utilization. The method also achieves efficient acceleration and real-time optimization of radar signal processing algorithms, greatly improving the data throughput capacity of a single processing module, and resolving the problem of uneven processing data caused by multiple time-series switching of radars, which leads to the inability to process certain scenarios in real time. The method is particularly suitable for multi-module pipeline processing in non-uniform and complex scenarios of phased array early warning radars. The method is a relatively complete radar signal processing implementation method based on CPU and GPU heterogeneity that solves the problems existing in the current CPU and GPU heterogeneous architectures in the actual application of radar signal processing. Technical Solution

[0004] The present invention achieves the above-mentioned purpose through the following technical solutions: A radar signal processing implementation method based on a heterogeneous CPU and GPU is disclosed. The CPU and GPU are integrated into a computing board. The CPU model is ft2000+ / 64; the GPU model is Iluvatar MR-V50. The CPU is the host side, serving as a control and scheduling device; the GPU is the device side, processing high-density computing tasks in the system through parallel acceleration. The radar signal processing process is a multi-module pipeline operation. Generally, due to the limited video memory resources between the GPU and CPU, to avoid multiple data transfers between the CPU and GPU, the GPU-side algorithm in the CPU / GPU heterogeneous mode is often implemented sequentially. As a result, the throughput time of each module depends on the execution time of the entire signal processing process. That is, each module must complete the entire signal processing process before processing the second packet of data. The throughput time is the time interval between each module processing two adjacent data packets.

[0005] Radar signal processing is a real-time processing process involving a multi-module pipeline. In practice, due to the diversity of radar tasks, the length of processed data, time intervals, and even the processing flow vary significantly. Consequently, the processing time of each module fluctuates significantly. According to typical processing flows, the throughput of each module depends on the execution time of the entire flow. This performance cannot be met in scenarios with shorter time intervals. To address this issue, the present invention proposes a multi-module asynchronous multi-stream architecture that replaces the throughput of each module with its own processing time, thereby significantly improving the processing capability of the entire flow.

[0006] Its features include: building a multi-module asynchronous multi-stream parallel architecture through the CPU and GPU heterogeneous hardware architecture, implementing multi-threaded control on the CPU side using mutex locks and message variables, and implementing asynchronous parallelism of kernel functions on the GPU side through multi-stream and event scheduling. The processing flow of multiple modules is decomposed so that the throughput time of the entire radar signal processing process is determined by the maximum processing time of a single module, thereby improving the radar signal multi-module processing capability in data non-uniform scenarios; combining the CPU and GPU heterogeneous hardware platforms to build a multi-module asynchronous multi-stream architecture: starting a management thread and an external interaction thread on the CPU side, and modularly encapsulating the signal processing functions. Each module is controlled by a thread. Starting a communication thread on the CPU side realizes the reception of echo data and external terminal control, and simultaneously sends the video data in the process of processing; scheduling the kernel functions of all modules on the GPU side through the management thread, and starting the CPU thread scheduling corresponding to the kernel functions of the GPU modules after writing the echo data to the GPU memory through the management thread. Through the processing thread of each module and the kernel function control of all modules on the GPU side, pipeline parallel processing of multiple processing modules is realized; Multiple processing modules are pipelined and processed in parallel. Each module has the following interfaces: Datain: Input data Dataout: output data Dim: Thread grid parameters Stream: Event_in: preceding event Event_out: post-event Mtx: Mutex Cv: Condition variable Sig_serial: handles resident number Turn: array, indicating the processing progress of each resident Multiple processing threads are triggered by polling of the management thread, and the management thread inputs the above interface information of the module after the processing thread is triggered; Dim is the thread block size of the GPU kernel function; Steam is the processing stream corresponding to the current process; Event_in is the processing status of the previous module in the pipeline processing of the current module; Event_out: The current module notifies the next module of its current status in the pipeline processing; The Cv condition variable is used to determine whether the current module meets the conditions for processing input data; The Mtx mutex is used to prevent competition in the Turn array when multiple threads access it. Turn: array, indicating the processing progress of each residence; Multiple processing threads are triggered by polling of the management thread, and the management thread inputs the above interface information of the module after the processing thread is triggered; Dim is the thread block size of the GPU kernel function; Steam is the processing stream corresponding to the current process; Event_in is the processing status of the previous module in the pipeline processing of the current module; Event_out: The current module notifies the next module of its current status in the pipeline processing; The Cv condition variable is used to determine whether the current module meets the conditions for processing input data; The Mtx mutex is used to prevent competition in the Turn array when multiple threads access it. Multiple processing threads are triggered by polling of the management thread. After the processing thread is triggered, the module's interface information is input. The module completes the initialization of events and flows during the startup phase. The Turn array is an array maintained by the entire processing flow that records the resident number of each module that has completed resident processing. The size of the Turn array is set to 10, and the input resident number is calculated through modulo to achieve the effect of array recycling. The modules use the Turn array and events to control the processing data of each module. If it is not the turn to process, the control is returned to the CPU.

[0007] The beneficial effects of the present invention compared with the prior art are: This CPU / GPU heterogeneous radar signal processing implementation method uses the CPU and GPU to build a multi-module asynchronous multi-stream parallel architecture. The CPU and GPU collaborate on task scheduling to achieve multi-stage pipeline parallel execution of the radar signal processing algorithm, reducing data transmission latency and optimizing resource utilization. The method also determines the throughput time of the entire processing flow by the maximum processing time of a single module, ensuring that the radar signal processing algorithm completes data processing for the current cycle within a time interval less than the echo data interval. It also significantly enhances the adaptability of multi-module radar signal processing in scenarios with complex processing sequences and data inhomogeneity. It also improves data transmission efficiency between the CPU and GPU and optimizes task scheduling strategies, effectively increasing the data throughput of a single module and meeting the real-time requirements of radar signal processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is an overall architecture diagram of a radar signal processing implementation method based on CPU and GPU heterogeneity of the present invention; Figure 2 This is a workflow diagram of a radar signal processing implementation method based on CPU and GPU heterogeneity of the present invention; Figure 3 This is a rendering of the implementation effect of asynchronous stream parallel processing of multiple modules of a radar signal processing implementation method based on CPU and GPU heterogeneity of the present invention; Figure 4 This is the execution logic diagram of the CPU+GPU collaborative work in the radar signal processing process in the conventional CPU+GPU heterogeneous mode; Figure 5 This is a diagram showing the working principle of CPU+GPU in actual engineering applications of radar signal processing in conventional CPU+GPU heterogeneous mode; Figure 6 This is a diagram showing the implementation of multi-stream serial processing of multiple modules in the conventional CPU+GPU heterogeneous mode; Figure 7 A comparison chart of the hardware differences between CPU and GPU; Figure 8-11 This is the sample code for the multi-module asynchronous multi-stream specific scheduling intermediate process of the radar signal processing implementation method based on CPU and GPU heterogeneity of the present invention. DETAILED DESCRIPTION

[0009] The design concept of this invention is based on the following: With the rapid development of radar technology, the design requirements for radar's target detection accuracy, anti-interference capabilities, multi-tasking, and real-time critical functions are becoming increasingly stringent. As a result, the complexity of radar signal processing algorithms has doubled. Traditional FPGA+DSP-based radar systems have long development cycles, poor scalability, and limited floating-point computing capabilities, making it difficult to meet the real-time requirements of modern radar signal processing. Simultaneously, the rapid development and market entry of GPU (Graphics Processing Unit) technology has provided a new solution for radar signal processing with its superior floating-point computing power and massively parallel thread architecture. GPUs, leveraging the CUDA (Compute Unified Architecture) programming model, efficiently support a hybrid computing model combining thread-level, data-level, and task-level parallelism, significantly improving the execution efficiency of key radar signal processing algorithms such as pulse compression, moving target detection (MTD), and constant false alarm rate (CFAR).

[0010] However, conventional radar signal processing systems based on heterogeneous CPUs and GPUs still face the following challenges: a) Insufficient collaborative optimization of heterogeneous architecture: The data transmission efficiency and task scheduling strategy between the CPU and GPU affect the overall performance.

[0011] b) Pipeline parallelism is not fully exploited: Serial processing of multi-module pipelines reduces the data throughput of a single module.

[0012] c) Weak adaptability to non-uniform scenarios: Under complex processing time sequences, common processing flows are difficult to meet real-time processing requirements.

[0013] Therefore, in response to the above-mentioned existing technical problems to be solved, the present invention proposes a radar signal processing implementation method based on a CPU and GPU heterogeneous hardware platform. The core invention purpose is to build a modular heterogeneous parallel architecture using the CPU and GPU. Through collaborative task scheduling between the CPU and GPU, this method realizes multi-stage pipeline parallel execution of radar signal processing algorithms, reduces data transmission latency, and optimizes resource utilization.

[0014] The following is a detailed description of the specific implementation method of the radar signal processing method based on CPU and GPU heterogeneity (see the attached figure). Figure 1-11 ): A radar signal processing implementation method based on CPU and GPU heterogeneity is proposed. Based on the back-end signal processing design process of a certain type of radar, software development optimization is performed on a high-performance computing board that integrates a CPU and GPU. A high-performance development framework suitable for the characteristics of the radar signal processing process is proposed. Specifically, a multi-module asynchronous multi-stream architecture is constructed using a CPU and GPU heterogeneous hardware platform. The hardware platform of the multi-module asynchronous multi-stream architecture is as follows: a) CPU parameters Model: ft2000+ / 64 Kernel version: 4.19.90-25.10.v2101.ky10.aarch64.

[0015] Number of cores: 64.

[0016] b) GPU parameters Model: Iluvatar MR-V50 (Tianshu Zhikai 50) Global available memory size: 17179869184 (bytes).

[0017] The available shared memory size for a thread block is 131072 (bytes).

[0018] The number of registers available for a thread block of 32 is 262144.

[0019] The maximum number of threads in a thread block is 4096. The number of threads in the warp is 64.

[0020] The maximum continuous memory size supported by memory copy is 2147483647 (bytes).

[0021] Maximum number of threads in each dimension of a thread block: 4096, 4096, 256.

[0022] Maximum depth of each grid dimension: 2147483647, 65535, 65535.

[0023] c) Software environment: Operating system: Linux Software compiler: clang++ cuda version: 10.2 The differences in the hardware structure of CPU and GPU are as follows (see Figure 7 ): The GPU has many execution units inside, each of which can run multiple threads. This special hardware structure determines that the GPU is good at solving the high-density computing tasks it is responsible for, but is not good at applying to logically complex tasks. In order to support the use of CPU+GPU heterogeneous systems to execute applications, NVIDIA specially designed the CUDA programming model to implement programming on both the CPU and GPU sides. In the CUDA programming model, the CPU acts as a control and scheduling device and becomes the host side (Host), responsible for serial logical function implementation, external communication and scheduling GPU kernel function execution. The GPU is regarded as a coprocessor and becomes the device side (Device), responsible for high-density computing tasks in the processing system through parallel acceleration.

[0024] By using specific function type qualifiers, you can specify whether the function runs on the CPU or GPU. Functions declared with _host_ can only run on the CPU, functions declared with _device_ can only run on the GPU, and functions qualified with _global_ are kernel functions in the CUDA program, which are called by the CPU and run on the GPU.

[0025] In the conventional CPU+GPU heterogeneous mode, the design and implementation of the radar signal processing flow requires the collaboration of the CPU and GPU. The common execution logic of the CPU+GPU heterogeneous mode (see Figure 4 ),like Figure 4 As shown in the figure, the radar signal processing process is a multi-module pipeline operation. To save data transmission bandwidth between the CPU and GPU and give full play to the advantages of GPU high-performance computing, the algorithm implementation on the GPU side generally adopts a sequential implementation method, that is, a resident data is collected on the CPU side, then copied to the GPU memory, the entire signal processing flow is completed sequentially on the GPU side, and then the results are copied to the CPU side. The specific steps under normal circumstances are as follows: 1) Define variables on the CPU side, pre-allocate variables and video memory space, and pre-allocate the memory and interface for receiving echo data and control messages on the CPU based on the relevant variables and parameters of the radar's overall timing design.

[0026] 2) The CPU starts external interaction threads including: echo data receiving thread, control message receiving thread and A display output thread, and transmits the received echo data to the GPU memory through PCIE.

[0027] 3) The CPU starts the kernel function, and the GPU executes the kernel function for parallel processing, passing the processing of each module to the next module in turn until all modules are processed.

[0028] 4) The processing result data is transferred from the GPU to the CPU, and the CPU sends the processing result to the next subsystem, thus completing the currently resident processing task.

[0029] In the conventional CPU+GPU heterogeneous mode, the working principle of radar signal processing in engineering applications (see Figure 5 ).

[0030] In the conventional CPU+GPU heterogeneous mode, in the radar signal processing process, all CUDA-implemented kernel functions are explicitly or implicitly run in a stream. Generally, if a stream is not explicitly specified, the kernel function runs on the null stream by default. On the other hand, the stream in which the kernel function runs can be explicitly specified to overlap different CUDA operations. To start the kernel function in a non-default stream, a stream identifier needs to be provided as the fourth parameter in the kernel execution configuration: kernel_name<<<grid, block, shareMemSize, stream> >>(argument list); In this way, the effect of multi-stream serial processing of multiple modules is as follows Figure 6 As shown (see Figure 6 ).

[0031] When each module is assigned to a different stream, multiple modules can be parallelized in the time dimension, such as Figure 3 As shown (see Figure 3 ).

[0032] A radar signal processing implementation method based on CPU and GPU heterogeneity first improves the processing architecture. Using the CPU and GPU, a multi-module asynchronous multi-stream parallel architecture is constructed. This allows the throughput of the entire radar signal processing process to be determined by the maximum processing time of a single module. This improves the throughput of the entire processing process, solves the problem of insufficient processing performance in certain timing scenarios, and implements multi-module radar signal processing in scenarios with non-uniform data. Construct a multi-module asynchronous multi-stream architecture: Start a management thread and an external interaction thread on the CPU side. The external interaction thread receives echo data and controls external terminals, while sending the video data in the process. Encapsulate the signal processing functions in a modular manner, with each module controlled by a thread. The management thread schedules the kernel functions of all modules on the GPU side. After the management thread writes the echo data to the GPU memory, it starts the CPU thread scheduling corresponding to the kernel function of the GPU module. Through the processing thread of each module and the kernel function control of all modules on the GPU side, pipeline parallel processing of multiple processing modules is achieved. The internal function modules on the GPU side are controlled by conditional variables and events. On the one hand, this ensures that the entire processing flow is still pipelined, and on the other hand, it ensures that computing resources are fully utilized and not wasted.

[0033] The throughput time of each module in the multi-module asynchronous multi-stream architecture can be improved by improving the processing time of the entire process to the execution time of the current module.

[0034] Throughput time: The time difference between each module's sequential execution of two adjacent data packets is defined as the module's throughput time. In common processing architectures, the throughput efficiency of a single module depends on the execution time of the entire processing flow. Each module must process the second packet of data after the entire processing flow is completed. This model reduces the throughput capacity of the entire processing flow when data is uneven and the processing time of each module varies greatly.

[0035] See Table 1: When the sampling rate of echo data is 5M, the cycle time is 800us, and the pulse width is 200us, the processing time of common modules of radar signal processing is shown in Table 1.

[0036] Table 1 Processing time of each signal processing module

[0037] As shown in Table 1, the processing time of the entire signal processing process under the current timing is 1225us, which is longer than the cycle time of 800us. Therefore, the data processing of the current cycle cannot be completed within the echo data interval time, which requires further improvement.

[0038] The present invention provides a radar signal processing implementation method based on CPU and GPU heterogeneity. The method fully utilizes the multi-stream capability of the GPU for design, realizes multi-module asynchronous multi-stream processing of radar signal processing, and changes the throughput time of the entire processing flow to be determined by the maximum processing time of a single module, thereby realizing multi-module processing of radar signals in data non-uniform scenarios.

[0039] The interface of the functional module and the multi-module asynchronous multi-stream implementation architecture were designed in combination with the functional characteristics of the signal processing backend.

[0040] The overall multi-module asynchronous multi-stream architecture is as follows: the CPU starts a communication thread to receive echo data and external terminal control, and at the same time sends the video data in the process of processing. The other part is different from the conventional processing flow. Through one thread, the kernel functions of all modules on the GPU side are scheduled. Instead, the management thread is used to write the echo data to the GPU memory and then start the CPU thread scheduling corresponding to the kernel function of the GPU side module. Through the processing thread of each module and the control of the GPU side kernel function, the pipeline parallel processing of multiple processing modules is realized (see Figure 1 ).

[0041] To adapt to the pipeline parallel processing of multiple processing modules, each module is designed to have the following interfaces: Datain: Input data Dataout: output data Dim: Thread grid parameters Stream: Event_in: preceding event Event_out: post-event Mtx: Mutex Cv: Condition variable Sig_serial: handles resident number Turn: array, indicating the processing progress of each resident Multiple processing threads are triggered by polling of the management thread, and the management thread inputs the above interface information of the module after the processing thread is triggered; Dim is the thread block size of the GPU kernel function; Steam is the processing stream corresponding to the current process; Event_in is the processing status of the previous module in the pipeline processing of the current module; Event_out: The current module notifies the next module of its current status in the pipeline processing; The Cv condition variable is used to determine whether the current module meets the conditions for processing input data; The Mtx mutex is used to prevent competition in the Turn array when multiple threads access it. Multiple processing threads are triggered by polling of the management thread. After the processing thread is triggered, the module inputs the above interface information. The module completes the initialization of events and flows during the startup phase (see Figure 8 ).

[0042] First, determine the status of the preceding event to determine whether the module of the previous pipeline has been processed. If it is not completed, wait. If it is completed, the condition variable waits for the resident processing progress in the Turn array to be executed sequentially in this module. The Turn array involves multi-threaded access, so add a mutex lock before use (see Figure 9 ).

[0043] After confirming that the current input information meets the processing conditions, the function module calls the kernel function calculated on the GPU through the grid information and flow parameters (see Figure 10 ).

[0044] After the current module is processed, it is determined whether there is a post-event. If there is no post-event, it means the process ends and the result is returned. If there is a post-event, it is recorded, indicating that the next module can be executed, and the Turn array is updated at the same time (see Figure 11 ).

[0045] If the program stop signal is not received, the current processing thread remains silent and waits for the next trigger of the management thread (see Figure 2 ).

[0046] The Turn array maintains an array for the entire processing flow, recording the resident numbers of each module's completed processing. During the actual use of the CPU-GPU heterogeneous radar signal processing implementation method of the present invention, the size of this Turn array is set to 10, and the input resident numbers are calculated through modulo calculation to achieve the effect of array recycling. The modules use this Turn array and events to control the processing data of each module. If it is not their turn to process, control is returned to the CPU end, effectively saving CPU and GPU computing resources and time resources.

[0047] The above processing scenario was repeated using the method implemented by the present invention, and the test results are shown in Table 2: Table 2 Processing time of each module after implementation of this method

[0048] As shown in Table 2, after the implementation of this method, the processing time of each module increased accordingly due to the addition of scheduling events. The time of the entire processing process increased from 1225us to 1340us, an increase of 9%. However, before the implementation of this method, each module had to complete the entire process of processing the current data before responding to the next packet of data. After the implementation of this method, each module only needs to complete its own module processing before responding to the next packet of data. Therefore, the data throughput of the entire process is reduced from the previous full-process processing time of 1225us to the full-process processing time of 500us, i.e., the throughput time, which is the most time-consuming pulse compression module. This reduces the full-process processing time by 59%, and solves the problem that the previous full-process processing time of 1225us does not meet the dwell interval of 800us. It greatly improves the radar's processing capability for data unevenness caused by multiple time series switching.

[0049] This CPU / GPU heterogeneous radar signal processing implementation method uses the CPU and GPU to build a modular heterogeneous parallel architecture. Through collaborative task scheduling between the CPU and GPU, it achieves multi-stage pipeline parallel execution of radar signal processing algorithms, improving the throughput time of individual modules, effectively reducing data transmission latency, and optimizing resource utilization, fully meeting the real-time requirements of radar signal processing.

[0050] The above description is only a preferred embodiment of the present invention. The above examples do not impose any form of limitation on the essential content of the present invention. After reading this specification, ordinary technicians in the relevant technical field make any simple modifications or deformations to the above specific implementation methods based on the technical essence of the present invention, as well as equivalent embodiments that may be changed or modified into equivalent changes using the technical content disclosed above, all fall within the scope of the technical solution of the present invention and do not deviate from the essence and scope of the present invention.

Claims

1. A radar signal processing implementation method based on a heterogeneous CPU and GPU architecture. The CPU and GPU are integrated into a computing board. The CPU model is ft2000+ / 64; the GPU model is Iluvatar MR-V50. The CPU is the host-side, serving as a control and scheduling device; the GPU is the device-side, accelerating the processing of high-density computing tasks in the system through parallelization. The radar signal processing process is a multi-module pipeline operation. In the heterogeneous CPU and GPU model, the GPU-side algorithm is mostly implemented sequentially. The throughput time of each module depends on the execution time of the entire signal processing process. That is, each module must complete the entire signal processing process before processing the second packet of data. The throughput time is the time interval between each module processing two adjacent data packets. Its characteristics are: A multi-module asynchronous multi-stream parallel architecture is built through the heterogeneous CPU and GPU hardware architecture. Multi-threaded control is implemented on the CPU side using mutex locks and message variables. On the GPU side, asynchronous parallel kernel functions are implemented through multi-stream and event scheduling. The processing flow of multiple modules is decomposed, and the throughput time of the entire radar signal processing flow is determined by the maximum processing time of a single module. This improves the multi-module processing capability of radar signals in scenarios with non-uniform data. Construct a multi-module asynchronous multi-stream architecture: start a management thread and an external interaction thread on the CPU side, and modularize the signal processing functions. Each module is controlled by a thread. Start a communication thread on the CPU side to receive echo data and control external terminals, and send the video data in the process. The kernel functions of all modules on the GPU side are scheduled through the management thread. After the echo data is written to the GPU memory through the management thread, the CPU thread scheduling corresponding to the kernel function of the GPU module is started. Through the processing thread of each module and the kernel function control of all modules on the GPU side, pipeline parallel processing of multiple processing modules is realized. The functional modules on the GPU side are controlled by separate processing threads through conditional variables and time parameters. On the one hand, pipeline processing of multiple modules is realized, and on the other hand, GPU resource scheduling is optimized to avoid waste of computing resources. Multiple processing modules are pipelined and processed in parallel. Each module has the following interfaces: Datain: Input data Dataout: output data Dim: Thread grid parameters Stream: Event_in: preceding event Event_out: post-event Mtx: Mutex Cv: Condition variable Sig_serial: handles resident number Turn: array, indicating the processing progress of each resident Multiple processing threads are triggered by polling of the management thread, and the management thread inputs the above interface information of the module after the processing thread is triggered; Dim is the thread block size of the GPU kernel function; Steam is the processing stream corresponding to the current process; Event_in is the processing status of the previous module in the pipeline processing of the current module; Event_out: The current module notifies the next module of its current status in the pipeline processing; The Cv condition variable is used to determine whether the current module meets the conditions for processing input data; The Mtx mutex is used to prevent competition in the Turn array when multiple threads access it. The Turn array is an array maintained by the entire processing flow that records the resident number of each module that has completed resident processing. The size of the Turn array is set to 10, and the input resident number is calculated through modulo to achieve the effect of array recycling. The modules use the Turn array and events to control the processing data of each module. If it is not the turn to process, the control is returned to the CPU.

Images