FPGA-based YOLOv3 convolutional neural network virtualization design method

Abstract

The invention discloses a YOLOv3 convolutional neural network virtualization design method based on an FPGA. The method comprises the specific steps of network structure and parameter tuning and shaping, neural network model disassembly and analysis, 'neural network operator module 'type design, virtualization design module determination, operator module construction and acceleration implementation. Compared with the prior art, the method has the advantages that the AI application development difficulty of the FPGA is reduced, and software engineers who do not know FPGA development, simulation and debugging too much can perform AI application development by utilizing the FPGA; the development efficiency of the FPGA is effectively improved, so that a top-layer AI application development user does not need to pay much attention to the implementation mode and details of a hardware logic circuit of the FPGA, and AI application development is realized in a shorter time; the flexibility of algorithm implementation is improved, and the parallelism degree of hardware acceleration can be adjusted according to CPU and FPGA resource conditions and algorithm real-time requirements; the method can be popularized to hardware acceleration application of various convolutional neural network models.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a design method for virtualizing a YOLOv3 convolutional neural network on an FPGA. Background Art

[0002] 1. Advantages and disadvantages of CPU, ASIC, and GPU in AI applications

[0003] As a general-purpose processor, CPU has extremely high flexibility and ease of use. For example, ARM processors can be programmed in high-level languages ​​such as C / C++, can complete various complex operations and be compatible with various scenarios, and have low deployment and operation costs. Compared with ASIC and FPGA, CPU has higher power consumption. In terms of performance, CPU is not designed for computing acceleration, so its performance is lower than GPU, ASIC and FPGA.

[0004] Therefore, the mainstream research directions for the acceleration design of AI algorithms include GPU (Graphics Processing Unit) acceleration, ASIC (Application Specific Integrated Circuit) acceleration and FPGA (Field Programmable Gate Array) acceleration.

[0005] The advantages of ASIC dedicated chips are high throughput, low latency and low power consumption, but it is not suitable for deep learning because deep learning computing tasks are flexible and require continuous iteration and improvement, while ASIC R&D and tape-out are expensive and have a long cycle. Most importantly, the logic cannot be adjusted dynamically.

[0006] Due to its high parallelism, high main frequency, and high bandwidth, the advantage of GPU lies in the parallel computing and floating-point computing capabilities of large amounts of data. GPU-accelerated deep learning algorithms have been widely used and have achieved remarkable results. For deep learning algorithms, GPU is a good choice in terms of acceleration performance. However, the power consumption of GPU is very high, not only much higher than that of CPU, but also dozens or even hundreds of times that of FPGA. The high power consumption requirement makes it only usable in high-performance computing clusters. Therefore, there is an urgent need for a high-performance, low-power processing platform. In this context, FPGA becomes a more ideal choice. In addition, compared with GPU, FPGA not only has the characteristics of data parallelism, but also has the characteristics of pipeline parallelism. Therefore, for streaming computing tasks, compared with GPU, FPGA has a natural advantage in latency.

[0007] 2. Advantages of FPGA in AI applications

[0008] Field Programmable Gate Array is abbreviated as FPGA in English. Different hardware circuit designs can be downloaded repeatedly to FPGA to realize different hardware designs and functions. FPGA can even dynamically change the circuit structure and function at the scene of use. Therefore, flexibility is one of the most important characteristics and advantages of FPGA.

[0009] At present, AI technology has been applied to all aspects of social production and life, and the combination of AI+FPGA has received more and more attention for the following reasons: First, AI applications have put forward higher and higher requirements on the computing power of hardware platforms, and at the same time, the hardware platform needs to take into account a series of requirements such as high energy efficiency, low latency, high throughput, and scalability, and FPGA meets all of the above requirements; secondly, AI algorithms are applied to various scenarios, and the requirements for training and reasoning tasks are also different, and FPGA with the characteristics of "strong flexibility" can better meet this demand; finally, since AI algorithms are in a state of continuous iteration and endless emergence, it is inevitable that AI applications need to be frequently upgraded, and the "field programmable" feature of FPGA can well meet this demand.

[0010] Therefore, FPGA is very suitable for this kind of AI application hardware implementation platform that requires high performance, low energy consumption, low latency, strong flexibility, small batch, and continuous iteration.

[0011] 3. Traditional FPGA implementation process

[0012] In the traditional FPGA development model, users usually use hardware description language (HDL) to model application scenarios and build business logic. Then, through specific FPGA development tools, the hardware model is directly mapped to the FPGA, and EDA tools automatically complete operations such as layout and routing and timing optimization, and finally generate an image file that can run on the FPGA. The advantage of using this method to directly develop the FPGA is that the required hardware logic resources can be used arbitrarily, such as the LUT lookup table on the FPGA, RAM, DSP, and various I / O modules and interfaces, such as PCIe and Ethernet controllers. At the same time, users can make targeted adjustments and optimizations to the design memory at the hardware level, so generally better system performance will be obtained.

[0013] 4. Difficulties in applying FPGA to AI

[0014] Although FPGA has many advantages for AI applications, it is not easy to implement AI applications using FPGA. The main difficulties are as follows:

[0015] Difficulty 1: Compared with CPU and GPU, FPGA development is more difficult. This is because it requires high skills from developers. Developers need to have a good understanding of the structure and internal resources of FPGA, master the hardware description language to model the design, and be proficient in the development, simulation and debugging tools of the target FPGA device. Especially in actual engineering projects, it often takes several years to master the above skills. The steep learning curve makes it impossible for software engineers to independently complete the development of AI applications for FPGA.

[0016] Difficulty 2: AI algorithms based on deep learning are generally built for GPU computing, but the FPGA hardware platform is far from the GPU in terms of computing and underlying structure, which makes it extremely difficult to implement deep learning networks on FPGA. In computers, various deep learning frameworks provide convenience for algorithm writing, such as TensorFlow, PyTorch and other frameworks. For example, the YOLOv3 algorithm only requires more than a thousand lines of code to be implemented in the above deep learning framework, but there is no deep learning framework that can provide support in FPGA. Tens of thousands of lines of code need to be written to initially implement the algorithm function, and the uncertainty of the number of convolution kernels and channels is also a challenge for FPGA implementation. In theory, each convolution layer needs to reprogram the FPGA, but this means a huge workload. In the case where the network structure parameters change with depth, how to efficiently implement the FPGA hardware of the algorithm is also a problem that needs to be solved. Summary of the invention

[0017] The present invention aims to solve the above technical problems and provide a design method for virtualization of YOLOv3 convolutional neural network based on FPGA.

[0018] In order to solve the above technical problems, the technical solution provided by the present invention is: a design method for virtualization of YOLOv3 convolutional neural network based on FPGA, including an AI application layer and a hardware layer, and an abstract layer arranged therebetween, wherein the AI ​​application layer is mainly a YOLOv3 algorithm model, the abstract layer is composed of various neural network operator modules, and the hardware layer is composed of FPGA, CPU, and GPU hardware resources; the method for implementing the convolutional neural network algorithm based on FPGA specifically includes the following steps:

[0019] Step 1: Use deep learning frameworks such as TensorFlow, Pytorch, and PaddlePaddle to optimize the network structure and parameters of the YOLOv3 algorithm based on GPU or CPU.

[0020] Step 2: Conduct an in-depth and detailed disassembly and analysis of the neural network model of the YOLOv3 algorithm that has been initially tuned and finalized. Through calculation, clarify the "neural network model calculation amount" and "neural network model access bandwidth" required for the operation of the YOLOv3 algorithm;

[0021] At the same time, we can accurately identify the "key computing bottlenecks" and make rigorous and accurate calculations of AI application acceleration effects.

[0022] Step 3: Using the "neural network operator module" design method, the convolution operator, which accounts for about 90% of the total computing workload, is implemented in the FPGA, and the neural network layer combination and scheduling work is completed by the CPU, etc.

[0023] Step 4: Analyze the convolutional neural network structure, select the frequently used "Convolution Conv + Batch Normalization BN + Activation ReLU" combination CBL module for FPGA virtualization design, and use it as the main basic unit to implement most of the YOLOv3 calculation work;

[0024] Step 5: Use the DSP and LUT resources in the FPGA to build matrix convolution operator, BN operator, and activation function LeakyReLU operator modules. The matrix convolution operator and BN operator are mainly composed of several DSPs. The adder and multiplier in the DSP are used to realize hardware acceleration of matrix multiplication and batch normalization. The LeakyReLU operator is mainly realized by the lookup table LUT to realize the hardware acceleration of activation operation. The above operators are serially combined into a virtualized CBL module. The CBL module is used as the main basic unit to realize the Res module and ResN module, and realize the hardware acceleration of most convolution calculations in the YOLOv3 model. Other tasks that are not suitable for hardware acceleration are realized by the CPU running software.

[0025] Furthermore, the tuning of the grid structure includes adjusting the number of network layers, the connection method between each layer, the convolution kernel size, the number of convolution kernels, and the number of channels to find the architecture that best suits the specific tasks and data sets of the YOLOv3 algorithm;

[0026] The parameter tuning is to use various optimization algorithms to update the weights and bias parameters in the network to minimize the loss function and improve the performance and accuracy of the model, where the optimization algorithms are stochastic gradient descent, Adagrad, Adadelta, etc.

[0027] The training described above enables the model to achieve better performance and generalization ability on the validation set or test set.

[0028] Furthermore, the "neural network operator module" design method allows the FPGA to focus on processing operators such as convolution calculations that have large computational complexity and are suitable for parallel processing by reasonably allocating tasks, while leaving the neural network layer combination and scheduling work that requires flexible scheduling and complex logic processing to the CPU.

[0029] Furthermore, the virtualization design of the CBL module can effectively improve the design and implementation efficiency of the YOLOv3 algorithm on the FPGA.

[0030] Furthermore, the operator modules and combinations constructed using FPGA resources achieve hardware acceleration of key parts in the YOLOv3 model and optimize system performance.

[0031] Furthermore, the design method is applicable to other algorithms that require extensive use of matrix multiplication.

[0032] The advantages of the present invention compared with the prior art are:

[0033] 1. The method of the present invention can effectively reduce the difficulty of FPGA AI application development, so that software engineers who are not familiar with FPGA development, simulation and debugging can also use FPGA to develop AI applications;

[0034] 2. The method described in the present invention can effectively improve the development efficiency of FPGA, so that top-level AI application development users do not need to pay too much attention to the implementation method and details of the hardware logic circuit of FPGA, and realize AI application development in a shorter time.

[0035] 3. The method of the present invention can greatly improve the flexibility of algorithm implementation, and can adjust the parallelism of hardware acceleration according to the CPU and FPGA resources and the real-time requirements of the algorithm. That is, when FPGA resources are limited and the algorithm running time limit is relatively loose, the main goal can be "using limited resources to realize model operation"; when FPGA resources are relatively abundant and the algorithm real-time requirements are high, the main goal can be "making full use of the parallel operation characteristics of FPGA circuits to perform algorithm hardware acceleration". In short, it can be understood that when space is limited, time is used to exchange space; when time is limited, space is used to exchange time.

[0036] 4. The method described in the present invention can be extended to hardware acceleration applications of various convolutional neural network models (or models that require a large number of matrix multiplications). BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is an FPGA virtualization layered diagram of AI application of the design method of FPGA-based YOLOv3 convolutional neural network virtualization of the present invention.

[0038] Figure 2 It is a structural schematic diagram of the design method of FPGA-based YOLOv3 convolutional neural network virtualization of the present invention.

[0039] Figure 3 It is a schematic diagram of the YOLOv3 network structure of the YOLOv3 convolutional neural network virtualization based on FPGA of the present invention. DETAILED DESCRIPTION

[0040] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0043] like Figure 1 As shown, it includes an AI application layer and a hardware layer, as well as an abstract layer arranged between the two. The AI ​​application layer is mainly a YOLOv3 algorithm model, the abstract layer is composed of various neural network operator modules, and the hardware layer is composed of FPGA, CPU, and GPU hardware resources;

[0044] The abstraction layer is like a bridge, connecting AI applications and FPGA underlying circuits. At the top AI application layer, the user's application and business logic are first mapped to the abstraction layer through a specific virtualization compilation tool, thus avoiding direct operation of the FPGA hardware. The abstraction layer can have functions such as task scheduling, resource allocation and management. The FPGA at the hardware layer only needs to implement and execute the tasks assigned by the abstraction layer. The hardware layer includes a complete heterogeneous computing system, such as FPGA, CPU, GPU, etc. For example, taking the FPGA-based YOLOv3 algorithm model as an example, the FPGA virtualization layering of AI applications is implemented.

[0045] like Figure 2 is a schematic diagram of the design method. Figure 3 This is the network structure diagram of the convolutional neural network YOLOv3. All the dotted boxes are virtual function modules implemented using virtual modularization. For the basic implementation method, see Figure 2 .

[0046] The method for implementing the convolutional neural network algorithm based on FPGA specifically includes the following steps:

[0047] (I) Network structure and parameter tuning

[0048] 1. Choose the right deep learning framework

[0049] Among deep learning frameworks such as TensorFlow, Pytorch, PaddlePaddle, etc., choose one as the development basis based on project requirements and team technical familiarity.

[0050] 2. Data Preparation

[0051] Collect and organize image data sets for training and verification, and perform preprocessing work such as data cleaning and labeling.

[0052] 3. Network structure design and adjustment

[0053] Based on the basic architecture of YOLOv3, the number of network layers, convolution kernel size, number of channels, etc. are adjusted and optimized according to specific application scenarios and data characteristics.

[0054] 4. Parameter Tuning

[0055] Use optimization algorithms such as stochastic gradient descent (SGD), Adagrad, and Adadelta to iteratively update network parameters such as weights and biases to minimize the loss function and improve the accuracy of the model.

[0056] 5. Model evaluation and finalization

[0057] Evaluate the trained model on the validation set and determine whether the model has achieved the expected results by calculating indicators such as accuracy, recall, and F1 value. If so, determine the network structure and parameters of the model to prepare for subsequent FPGA implementation.

[0058] In this process, repeated experiments and verifications are performed to ensure that the performance and accuracy of the algorithm meet the ideal expectations. Only when the results meet expectations can the subsequent work be carried out. By operating in these mature frameworks, you can make full use of their rich tools and resources to ensure the rationality and effectiveness of network structure and parameters. This kind of pre-optimization work can lay a solid foundation for subsequent development on FPGA, avoiding repeated modifications and adjustments in the FPGA development stage due to imperfections in the early stage, thereby saving a lot of resources and avoiding unnecessary waste of time.

[0059] (II) Disassembly and analysis of neural network model

[0060] 1. Model conversion

[0061] Convert the model trained in the deep learning framework into a common intermediate representation format, such as ONNX format, for subsequent analysis.

[0062] 2. Computational Amount Analysis

[0063] By counting and analyzing the computational workload of operations such as convolution, pooling, and full connection in each layer of the model, the computationally intensive parts can be determined.

[0064] 3. Bandwidth demand analysis

[0065] Analyze the data read and write operations of the model during operation and calculate the required memory bandwidth.

[0066] 4. Identification of key computing bottlenecks

[0067] Combined with the computational workload and bandwidth requirements, as well as the resource characteristics of the FPGA chip, identify the key computational bottlenecks that may affect the algorithm performance, such as a large number of matrix multiplication operations and data transmission delays.

[0068] In this process, we scientifically and rationally divide the parts that are suitable for FPGA computing acceleration and the parts that are not suitable. For the suitable parts, we make full use of the powerful performance of FPGA to achieve efficient implementation; for the computing tasks that are not suitable for FPGA acceleration, we transfer them to other computing resources, such as CPU, to handle them.

[0069] (III) “Neural Network Operator Module” Design

[0070] 1. Operator Partitioning

[0071] The convolution calculation, pooling calculation, activation function calculation, etc. in the YOLOv3 algorithm are divided into different operators.

[0072] 2. Division of work between hardware implementation and software scheduling

[0073] It is determined that operators such as convolution calculations, which account for about 90% of the total computing workload, will be implemented in FPGA, using the parallel processing capabilities of FPGA for hardware acceleration. The combination and scheduling of neural network layers will be implemented by the CPU running software, giving full play to the flexibility and control capabilities of the CPU.

[0074] 3. Interface definition

[0075] Define the data interaction interface between FPGA and CPU, including data format, transmission protocol, etc., to ensure that the two can work together efficiently; if using a chip such as SoPC, SoC or MPSoC that integrates CPU and FPGA, you can use the on-chip bus and transmission protocol.

[0076] By making full use of the high-speed parallel computing capabilities of FPGA and the flexible control capabilities of CPU, the two work together to greatly improve the efficiency and effectiveness of the entire development process. Through this clever division of labor and cooperation, the technical difficulty of implementing the FPGA-based YOLOv3 algorithm can be significantly reduced, and the development efficiency can be greatly improved. Even software engineers can complete the entire development work independently and efficiently without a deep hardware knowledge background.

[0077] (IV) Determine the virtualization design module

[0078] 1. Convolutional Neural Network Structure Analysis

[0079] Conduct an in-depth analysis of the convolutional neural network structure of YOLOv3 to understand the connection relationship and data flow between each layer.

[0080] 2. High frequency module screening

[0081] Through the operation statistics and analysis of the model, we can find out the modules that are used frequently, such as the CBL module of the combination of "convolution Conv + batch normalization BN + activation ReLU".

[0082] 3. Selection of virtual design objects

[0083] The selected high-frequency modules are determined as objects for FPGA virtualization design to improve resource utilization and computing efficiency.

[0084] As a key module frequently used in the YOLOv3 algorithm, the CBL module is specially designed for FPGA virtualization, which can be optimized and customized according to its characteristics. By implementing hardware acceleration of the CBL module on FPGA, the calculation speed and efficiency of the module can be greatly improved, thereby driving the performance improvement of the entire YOLOv3 algorithm on FPGA and achieving more efficient target detection and processing.

[0085] (V) Constructing operator modules and achieving acceleration

[0086] 1. Resource utilization planning

[0087] According to the selected virtualization module and computing requirements, select the FPGA and plan the use of resources such as DSP (digital signal processing), LUT (lookup table), BRAM (block random access memory) in the FPGA.

[0088] 2. Matrix convolution operator construction

[0089] Use the DSP resources in the FPGA to build a matrix convolution operator. By properly configuring the multipliers and adders, efficient matrix multiplication operations can be achieved, completing the hardware acceleration of the convolution calculation.

[0090] 3. BN operator construction

[0091] DSP resources are also used to build a batch normalization (BN) operator to achieve data normalization processing and improve the stability and generalization ability of the model.

[0092] 4. Construction of activation function operator

[0093] For activation functions, such as LeakyReLU, LUT is used to achieve hardware acceleration and reduce calculation time.

[0094] 5. Operator Serial Combination

[0095] The matrix convolution operator, BN operator, and activation function LeakyReLU operator are serially combined into a virtualized CBL module to form a complete computing unit.

[0096] 6. Functional module implementation

[0097] Taking the CBL module as the basic unit, we further implement functional modules such as the Res module and the ResN module to complete the hardware acceleration of most convolution calculations in the YOLOv3 model.

[0098] 7. Software and hardware collaboration

[0099] By running the software on the CPU, the combination and scheduling of the various layers of the neural network and the calculations that are not suitable for hardware acceleration are realized, and the hardware acceleration module in the FPGA is coordinated to complete the efficient execution of the YOLOv3 algorithm.

[0100] By cleverly utilizing the DSP, LUT and other resources in the FPGA to build the matrix convolution operator, BN operator, activation function LeakyReLU operator and other modules, and forming a complete functional module in a specific serial combination mode, it is possible to perform targeted hardware acceleration on the key calculation parts in the YOLOv3 model. This hardware acceleration can significantly reduce the calculation time and increase the data processing speed, thereby optimizing the performance of the entire system and achieving faster and more accurate target detection and recognition.

[0101] (IV) Application scenarios and actual cases

[0102] 1. Smart security

[0103] In the video surveillance system, abnormal behaviors, intrusion events, etc. are detected and identified in real time. For example, the surveillance system of a large shopping mall uses the FPGA-based YOLOv3 algorithm, which can quickly and accurately identify abnormal situations such as not wearing masks and crowds, and issue alarms in time to ensure the safety and order of the mall.

[0104] 2. Autonomous driving

[0105] It can quickly and accurately identify vehicles, pedestrians, traffic signs and other targets on the road, providing a reliable basis for autonomous driving decisions. An autonomous driving car R&D team uses this technology to enable the vehicle to detect surrounding obstacles and traffic signals in real time in complex traffic environments, significantly improving the safety and reliability of autonomous driving.

[0106] 3. Industrial testing

[0107] Perform quality inspections on products on the production line, such as defect detection, size measurement, etc. An electronics manufacturing company deployed the FPGA-based YOLOv3 algorithm on its circuit board production line, achieving rapid detection of tiny defects on circuit boards and improving product qualification and production efficiency.

[0108] 4. Medical Image Analysis

[0109] Assist doctors to analyze medical images, such as tumor detection, lesion location, etc. The medical imaging diagnosis system of a certain hospital uses this algorithm to help doctors find lesion areas more quickly, improve the accuracy and efficiency of diagnosis, and buy precious time for patients' treatment.

[0110] This technical solution is a YOLOv3 algorithm implementation method based on FPGA. Through the steps of network structure and parameter tuning and finalization, neural network model disassembly and analysis, "neural network operator module" design, determination of virtualization design module, construction of operator module and acceleration, the YOLOv3 algorithm can be efficiently operated on FPGA. This method has the advantages of performance improvement, reduced development difficulty, resource optimization, flexibility and scalability, and has broad application prospects in the fields of intelligent security, autonomous driving, industrial detection, medical image analysis, etc.

[0111] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A design method for FPGA-based YOLOv3 convolutional neural network virtualization includes an AI application layer and a hardware layer, and an abstract layer arranged between the two. The AI ​​application layer is mainly a YOLOv3 algorithm model, the abstract layer is composed of various neural network operator modules, and the hardware layer is composed of FPGA, CPU, and GPU hardware resources. It is characterized by: The method for implementing the convolutional neural network algorithm based on FPGA specifically includes the following steps: Step 1: In the deep learning framework of TensorFlow, PyTorch, PaddlePaddle, MindSpore, Caffe, MXNet, Theano, Keras, CNTK, Sonnet, Torch, Darknet, Chainer, DL4J, optimize the network structure and parameters of the YOLOv3 algorithm based on GPU or CPU; Step 2: Conduct an in-depth and detailed disassembly and analysis of the neural network model of the YOLOv3 algorithm that has been initially tuned and finalized. Through calculation, clarify the "neural network model calculation amount" and "neural network model access bandwidth" required for the operation of the YOLOv3 algorithm; At the same time, it can accurately identify the "key computing bottlenecks" and make rigorous and accurate calculations of AI application acceleration effects; Step 3: Using the "neural network operator module" design method, the convolution operator, which accounts for about 90% of the total computing workload, is implemented in the FPGA, and the neural network layer combination and scheduling work is completed by the CPU, etc. Step 4: Analyze the convolutional neural network structure, select the frequently used "Convolution Conv + Batch Normalization BN + Activation ReLU" combination CBL module for FPGA virtualization design, and use it as the main basic unit to implement most of the YOLOv3 calculation work; Step 5: Use the DSP and LUT resources in the FPGA to build matrix convolution operator, BN operator, and activation function LeakyReLU operator modules. The matrix convolution operator and BN operator are mainly composed of several DSPs. The adder and multiplier in the DSP are used to realize hardware acceleration of matrix multiplication and batch normalization. The LeakyReLU operator is mainly realized by the lookup table LUT to realize the hardware acceleration of activation operation. The above operators are serially combined into a virtualized CBL module. The CBL module is used as the main basic unit to realize the Res module and ResN module, and realize the hardware acceleration of most convolution calculations in the YOLOv3 model. Other tasks that are not suitable for hardware acceleration are realized by the CPU running software.

2. The design method of YOLOv3 convolutional neural network virtualization based on FPGA according to claim 1, characterized in that: The tuning of the grid structure includes adjusting the number of network layers, the connection method between each layer, the size of the convolution kernel, the number of convolution kernels, and the number of channels.

3. The design method of YOLOv3 convolutional neural network virtualization based on FPGA according to claim 1, characterized in that: The "neural network operator module" design method allocates tasks reasonably, allowing the FPGA to focus on processing operators such as convolution calculations that have large computational complexity and are suitable for parallel processing, while leaving the neural network layer combination and scheduling work that requires flexible scheduling and complex logic processing to the CPU.

4. The design method of YOLOv3 convolutional neural network virtualization based on FPGA according to claim 1, characterized in that: The virtualization design of the CBL module can effectively improve the design and implementation efficiency of the YOLOv3 algorithm on FPGA.

5. The design method of FPGA-based YOLOv3 convolutional neural network virtualization according to claim 1, characterized in that: The operator modules and combinations constructed using FPGA resources achieve hardware acceleration of key parts in the YOLOv3 model and optimize system performance.

6. The design method of FPGA-based YOLOv3 convolutional neural network virtualization according to claim 1, characterized in that: The design method described is applicable to other algorithms that can also use the resources in the FPGA for hardware acceleration.

Images