Road tunnel detection system based on YOLOv6 and FPGA, electronic equipment and storage medium
By implementing the YOLOv6-based road tunnel detection system on FPGA, the problem of time-consuming and inconsistent manual detection in the prior art is solved, efficient and accurate road tunnel detection is achieved, and detection complexity and power consumption are reduced.
Patent Information
- Application Number
- CN202411952793.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-11
AI Technical Summary
In the prior art, road tunnel detection relies on manual visual inspection, which is time-consuming and labor-intensive, and the results are easily affected by the experience and subjective judgment of the inspector, making it difficult to ensure the consistency and accuracy of the detection.
The road tunnel detection system based on YOLOv6 and FPGA is adopted, and the training set of road tunnel images is obtained through the training module and the YOLOv6 model is trained, the model weight is quantified, and the model accelerator is implemented on the FPGA, convolution, pooling and activation processing is performed, and feature map data, detected tunnel bounding box information and category probability are output.
The inference process of the network model is realized on hardware, significantly reducing storage space, improving computing speed and power consumption, improving detection accuracy and efficiency, and reducing the complexity of network model implementation.
Smart Images
Figure CN119810551A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of road detection, and in particular to a road pit detection system, electronic equipment and storage medium based on YOLOv6 and FPGA. Background Art
[0002] The detection and analysis of road pits is a crucial link in modern road maintenance and management. With the acceleration of global urbanization, the health of road networks directly affects people's quality of life and regional economic development. Road pits are not only early signs of road aging, but also affect road safety and comfort. Therefore, timely and accurate detection of road pits is crucial to prevent their further deterioration, which can effectively extend the service life of roads, reduce maintenance costs, and ensure traffic safety. In the past, the detection of road pits mainly relied on manual visual inspection. This method is not only time-consuming and labor-intensive, but also the results are easily affected by the experience and subjective judgment of the inspectors, making it difficult to ensure the consistency and accuracy of the detection. With the development of computer vision and deep learning technology, the automatic detection technology of road pits based on image processing has become a hot topic of research. In particular, the emergence of the YOLO (You Only Look Once) series of algorithms has greatly improved the efficiency and accuracy of road pit detection.
[0003] As a fast target detection method, the YOLO algorithm has been successfully applied in many fields, including road pit detection. Its core advantage is that it can identify the target object and its position in the image in a single detection process. Compared with traditional target detection algorithms, YOLO significantly improves the detection speed while maintaining a high accuracy. However, the structure of the neural network algorithm is very complex, and the pure software algorithm implementation is very complex, occupies a large storage space, and has limited detection efficiency. Summary of the invention
[0004] In order to solve the above problems existing in the prior art, the present invention provides a road pit detection system, electronic device and storage medium based on YOLOv6 and FPGA. The technical problem to be solved by the present invention is achieved by the following technical solutions:
[0005] A first aspect of an embodiment of the present invention provides a road pit detection system based on YOLOv6 and FPGA, including:
[0006] A training module is used to obtain a training set of road tunnel images and train the YOLOv6 model to obtain a target YOLOv6 model;
[0007] A quantization module, used to quantize the weights of the target YOLOv6 model to obtain quantized data;
[0008] A model accelerator, configured to perform convolution, pooling and activation processing on the road image data to be detected according to the quantized data cached in the weight cache module, and output feature map data, bounding box information of the detected pit and category probability;
[0009] The model accelerator uses the hardware description language Verilog to describe the logic circuits of convolution, pooling and activation processing; the feature map data output after the road image data to be detected is processed once in the model accelerator is input into the model accelerator again for processing until the last processing is completed.
[0010] In one embodiment of the present invention, quantizing the weights of the target YOLOv6 model to obtain quantized data includes:
[0011] The weights of the target YOLOv6 model are quantized to int8 type to obtain quantized data; wherein the value range of the quantized data is [-128, 127].
[0012] In one embodiment of the present invention, the model accelerator includes: a computing module, the computing module includes: a convolution module, a maximum pooling module, an addition module and an activation function module;
[0013] The convolution module is used to perform convolution operations of the ConvModule module, RepVGGBlock module, RepStageBlock module and SPPFBottleneck module in the target YOLOv6 model;
[0014] The maximum pooling module is used to perform the maximum pooling operation of the SPPFBottleneck module in the target YOLOv6 model;
[0015] The activation function module is used to perform activation operations on the ConvModule module and the RepVGGBlock module in the target YOLOv6 model;
[0016] The addition module is used to perform the addition operation in the SPPFBottleneck module in the target YOLOv6 model.
[0017] In one embodiment of the present invention, when performing a convolution operation in a ConvModule module in the target YOLOv6 model, the convolution module is used to calculate a fusion result;
[0018] The calculation formula of the fusion result is:
[0019] BN(Conv(x))=W fused (x)+Bfused
[0020] in, x is the input, b is the convolution layer bias, mean is the mean of a batch input, is the standard deviation of the current batch, γ is the scaling parameter used to adjust the scale of the standardized value, and β is the translation parameter used to adjust the offset of the standardized value.
[0021] In one embodiment of the present invention, when executing the convolution operation of the RepVGGBlock module in the target YOLOv6 model, the convolution module is used to place the weight of Conv1×1 at the center position of Conv3×3, fill other positions with zero, and directly add the bias; or to set the convolution kernel parameters of the front channel to 1 and the convolution kernel parameters of the remaining channels to 0.
[0022] In one embodiment of the present invention, the model accelerator further includes: a row cache module;
[0023] The line buffer module is used to buffer three lines of the road image data to be detected, and input the road image data to be detected into the calculation module.
[0024] In one embodiment of the present invention, the model accelerator further comprises: a layer control module;
[0025] The layer control module is used to input the quantized data in the weight cache module into the calculation module.
[0026] In one embodiment of the present invention, it further includes: an output cache module;
[0027] The output cache module is used to cache feature map data, bounding box information and category probabilities of detected tunnels.
[0028] A second aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a road pit detection system based on YOLOv6 and FPGA provided by the first aspect of an embodiment of the present invention is implemented.
[0029] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a road pit detection system based on YOLOv6 and FPGA provided by the first aspect of an embodiment of the present invention is implemented.
[0030] Beneficial effects of the present invention:
[0031] The present invention implements each layer of the target YOLOv6 model on the FPGA, that is, completes the reasoning process of the network model on hardware. This includes realizing the functions of various network layers such as convolutional layer, pooling layer, activation function, etc. The hardware description language Verilog is used to describe the logic circuit of each layer and integrate it into the FPGA. In this process, the weight of the target YOLOv6 model is quantized, which can significantly reduce the storage space occupied, and more efficient integer operations can be used on the FPGA to improve the calculation speed and reduce power consumption. The reasoning process can be completed by reusing the model accelerator, which reduces the complexity of the network model implementation.
[0032] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.
[0033] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0035] Figure 1 A schematic block diagram of a road pit detection system based on YOLOv6 and FPGA provided in an embodiment of the present invention;
[0036] Figure 2 A schematic diagram of the network structure of the target YOLOv6 model provided by an embodiment of the present invention;
[0037] Figure 3 A detection flow diagram of a road pit detection system based on YOLOv6 and FPGA provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0038] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.
[0039] like Figure 1 As shown, a first aspect of an embodiment of the present invention provides a road pit detection system based on YOLOv6 and FPGA, comprising:
[0040] The training module 11 is used to obtain a training set of road tunnel images and train the YOLOv6 model to obtain a target YOLOv6 model.
[0041] The quantization module 12 is used to quantize the weights of the target YOLOv6 model to obtain quantized data.
[0042] The model accelerator 13 is used to perform convolution, pooling and activation processing on the road image data to be detected according to the quantized data in the weight cache module, and output feature map data, bounding box information of the detected pit and category probability.
[0043] Among them, the model accelerator uses the hardware description language Verilog to describe the logic circuits of convolution, pooling and activation processing; the feature map data output after the road image data to be detected is processed once in the model accelerator is input into the model accelerator again for processing until the last processing is completed.
[0044] In this embodiment, by implementing each layer of the target YOLOv6 model on the FPGA, the reasoning process of the network model is completed on the hardware. This includes implementing the functions of various network layers such as convolutional layers, pooling layers, and activation functions. The hardware description language Verilog is used to describe the logic circuit of each layer and integrate it into the FPGA. In this process, the weights of the target YOLOv6 model are quantized, which can significantly reduce the storage space occupied, and more efficient integer operations can be used on the FPGA to increase the calculation speed and reduce power consumption. The reasoning process can be completed by reusing the model accelerator, reducing the complexity of the network model implementation.
[0045] A second aspect of an embodiment of the present invention provides a road pit detection system based on YOLOv6 and FPGA, comprising:
[0046] The training module is used to obtain a training set of road tunnel images and train the YOLOv6 model to obtain a target YOLOv6 model.
[0047] The quantization module is used to quantize the weights of the target YOLOv6 model to obtain quantized data. Specifically, the quantization module quantizes the weights of the target YOLOv6 model to int8 type to obtain quantized data; wherein the value range of the quantized data is [-128, 127].
[0048] The model accelerator is used to perform convolution, pooling and activation processing on the road image data to be detected according to the quantized data in the weight cache module, and output feature map data, bounding box information of the detected pit and category probability. The weight cache module of the model accelerator is used to cache the quantized data.
[0049] Furthermore, the model accelerator includes a weight cache module and a computing module, the computing module including: a convolution module, a maximum pooling module, an addition module and an activation function module;
[0050] Convolution module, used to perform convolution operations of ConvModule module, RepVGGBlock module, RepStageBlock module and SPPFBottleneck module in the target YOLOv6 model.
[0051] Specifically, when the convolution operation in the ConvModule module in the target YOLOv6 model is performed, the convolution module is used to calculate the fusion result.
[0052] When performing the convolution operation of the RepVGGBlock module in the target YOLOv6 model, the convolution module is used to place the weights of Conv1×1 at the center of Conv3×3 and fill other positions with zero, and the bias is directly added; or it is used to set the convolution kernel parameters of the front channel to 1 and the convolution kernel parameters of the remaining channels to 0.
[0053] The maximum pooling module is used to perform the maximum pooling operation of the SPPFBottleneck module in the target YOLOv6 model.
[0054] The activation function module is used to perform activation operations on the ConvModule module and RepVGGBlock module in the target YOLOv6 model.
[0055] The addition module is used to perform the addition operation of the SPPFBottleneck module in the target YOLOv6 model.
[0056] Furthermore, the model accelerator also includes a row cache module and a layer control module.
[0057] The line buffer module is used to buffer three lines of road image data to be detected, and input the road image data to be detected into the calculation module.
[0058] The layer control module is used to input the quantized data in the weight cache module into the calculation module.
[0059] In this embodiment, an implementation and application process of a road pit detection system based on YOLOv6 and FPGA includes the following steps:
[0060] Step 21: Before training the YOLOv6 model, training set data collection is required.
[0061] During data collection, the traditional training data set is expanded to collect road tunnel image data in more different scenarios, different weather conditions (such as sunny, rainy, cloudy), different time periods (day, night), and different road types (highways, urban roads, rural roads, etc.).
[0062] Specifically, the training set data collection method uses self-photographing and open source data sets. On sunny days, the collected images are well-lit and clearly show the road surface and possible pits; on rainy days, the impact of rain on the pits and the slippery state of the road surface are captured; on cloudy days, the performance of road pits under soft light. And collect online to ensure that image data of different scenes, different weather conditions, different time periods, and different road types are collected. Tongji University UDTIRI (Urban Digital Twins Intelligent Road Inspections) road pothole open source data set is used. After the data collection is completed, the image size is processed to be consistent, and the images without marked pit locations are manually annotated. Finally, the sorted data set is divided into a training set (1,000 images containing 883 road potholes), a validation set (200 images containing 250 road potholes), and a test set (400 images containing 369 road potholes). The image size is unified to 640*640.
[0063] Before starting model training, you need to check the training set to see if the image size is 640*640 and the pixel value is between 0 and 255. And randomly flip the training set images horizontally and vertically to increase data diversity. The environment is set up on the RTX4090 environment, using the basic environment of python version 3.8 and torch version 1.8.0 to build the target detection model, using the cosine annealing learning rate scheduler, setting the initial value of the learning rate to 0.01 and the final value to 0.01; using the stochastic gradient descent optimizer for optimization, the batch size is set to 32, and the number of training rounds is 400.
[0064] Step 22, the training module obtains a training set of road tunnel images and trains the YOLOv6 model to obtain a target YOLOv6 model. The network structure of the target YOLOv6 model is as follows: Figure 2 shown.
[0065] After collecting enough image data, the model training phase begins. The collected data is preprocessed, including image cropping, scaling, normalization and other preprocessing operations to ensure data consistency and standardization. The YOLOv6-s model is trained using the preprocessed data set, and the model parameters are continuously adjusted to enable it to better learn the characteristics of road pits and improve the accuracy of detection. Appropriate hyperparameters need to be set during the training process, such as learning rate, batch size, number of training rounds, etc.
[0066] Specifically, the preprocessed training set data is input into the model for training. During the training process, the validation set is used to evaluate the model every certain number of rounds, and the accuracy is stabilized at 94%. The test set is used for testing, and the expected requirements are met, and then the target YOLOv6 model is obtained.
[0067] After training is completed, it is necessary to extract features from the input image data. The model first automatically extracts low-level features of the image, such as edges, textures, etc., through multiple convolutional layers and pooling layers. Then, more abstract features that can effectively represent road pits are extracted through high-level feature extraction layers. These features are stored as feature vectors for subsequent recognition and positioning.
[0068] After feature extraction, the detection system needs to be deployed on FPGA (Field Programmable Gate Array) to detect road pits in actual applications. Install and debug the hardware equipment to ensure that the detection system can operate normally. At the same time, the stability and reliability of the hardware equipment need to be considered to ensure the performance and safety of the system in long-term operation.
[0069] Step 23, the quantization module obtains the weight of the target YOLOv6 model, quantizes the weight of the target YOLOv6 model, and obtains quantized data.
[0070] Specifically, to complete the hardware deployment, the weights obtained from model training need to be quantized first. The quantization module quantizes the weights of the target YOLOv6 model to int8 type to obtain quantized data. The value range of the quantized data is [-128, 127]. The quantized data can significantly reduce the storage space occupied, and more efficient integer operations can be used on the FPGA to increase the computing speed and reduce power consumption. The weights of the quantized model file need to be extracted to obtain the values on each convolution kernel. The specific process is: first load and save the quantized model file, and access the model's state dictionary from it, extract the model's state dictionary and access the weights of each layer, and save these quantized weights.
[0071] In this embodiment, after quantization is completed, pure hardware is performed on ZYNQ7020, and each layer of the network is implemented on the FPGA, that is, the reasoning process of the network model is completed on the hardware. Building each layer of the network on ZYNQ7020 requires hardware design based on the structure and parameters of the YOLOv6-s model. This includes implementing the functions of various network layers such as convolutional layers, pooling layers, and activation functions. Use the hardware description language Verilog to describe the logic circuit of each layer and integrate it into the FPGA.
[0072] Step 24: Build a model accelerator.
[0073] Implementation of calculation module:
[0074] Implement the ConvModule module, which consists of a 2d convolution module, a BN layer, and an activation layer. Research has found that the convolution layer and the BN layer can be fused during the inference process, which can reduce the hardware workload, reduce the number of layers, and speed up the network inference performance. Specifically:
[0075] The convolutional layer formula is: Conv(x) = W(x) + b
[0076] The formula for the BN layer is:
[0077] Substitute the result of the convolutional layer into the BN formula:
[0078]
[0079] Further simplified to:
[0080]
[0081] It can be seen that this is a convolutional layer, but the weight takes into account the parameters of the BN layer; the final fusion result is:
[0082] BN(Conv(x))=W fused (x)+B fused
[0083] in, x is the input, b is the convolution layer bias, mean is the mean of a batch input, is the standard deviation of the current batch, γ is the scaling parameter used to adjust the scale of the standardized value, and β is the translation parameter used to adjust the offset of the standardized value.
[0084] When the ConvModule module in the target YOLOv6 model that fuses the convolution layer and the BN layer is implemented on the FPGA, only the convolution and activation processing need to be implemented.
[0085] When implementing the RepVGGBlock module, both RepVGGBlock modules are multi-channel models. To facilitate reasoning, they are converted into single-channel models, that is, 3×3 convolution can be fused with 1×1 convolution. For example, if the input feature map size is 5×5, the convolution kernel is 3×3, the step size is 1, and the padding is 0, a 3×3 output feature map is finally obtained; and if the input feature map size is 3×3, the convolution kernel is 1×1, the step size is 1, and the padding is 0, a 3×3 output feature map is also obtained. Due to the additivity principle of convolution, Conv1×1 and Conv3×3 can be merged without affecting the reasoning results and reducing the paths. After merging, we only need to place the weights of Conv1×1 at the center of Conv3×3, fill other positions with zero, and directly add the bias. Another path has no convolution layer, and the input is directly equal to the output. This can be achieved by setting the convolution kernel parameter of the current channel to 1 and the convolution kernel parameters of the remaining channels to 0. This path is equivalent to Conv1×1, and then converted to Conv3×3.
[0086] Therefore, after the convolutional layers are merged, the Conv layer and the BN layer are fused in the ConvModule module part, so that when the hardware is implemented, the RepVGGBlock part only needs to complete Conv3×3 and the activation layer ReLU / SiLU.
[0087] In the target YOLOv6 model, the RepStageBlock module contains multiple RepVGGBlock modules. Combined with the above analysis, the RepStageBlock module is implemented in the same way as the RepVGGBlock module. The SPPFBottleneck module contains the ConvModule module. The SPPFBottleneck module is implemented in the same way as the ConvModule module, and the addition operation in the SPPFBottleneck module is implemented accordingly through the addition module.
[0088] The weight cache module, row cache module and layer control module are designed, and after completion, the entire hardware system is integrated.
[0089] Step 25, after the hardware deployment is completed, the road pit is identified and located. After receiving the road image to be detected from the image acquisition device, the image is first pre-processed in real time, including denoising, contrast enhancement and other operations to improve the image quality.
[0090] Step 26, Figure 3As shown, the preprocessed road image to be detected is cached in the image cache module, and the image cache module inputs the cached data into the model accelerator for processing. The model accelerator includes: a calculation module, a row cache module, a layer control module, and a weight cache module. The weight cache module is used to cache quantized data.
[0091] Specifically, step 26 includes step 261-step 262:
[0092] Step 261 , the image buffer module inputs the preprocessed image of the road to be detected into the line buffer module, the line buffer module buffers three lines of image data of the road to be detected, and inputs the image data of the road to be detected into the calculation module.
[0093] Step 262: The layer control module inputs the quantized data in the weight cache module into the calculation module.
[0094] Step 263: The computing module performs convolution, pooling and activation processing on the road image data to be detected according to the quantized data, and outputs feature map data, bounding box information of the detected pit and category probability.
[0095] Specifically, the computing module includes: a convolution module, a maximum pooling module, an addition module and an activation function module.
[0096] Convolution module, used to perform convolution operations of ConvModule module, RepVGGBlock module, RepStageBlock module and SPPFBottleneck module in the target YOLOv6 model.
[0097] The maximum pooling module is used to perform the maximum pooling operation of the SPPFBottleneck module in the target YOLOv6 model.
[0098] The activation function module is used to perform activation operations on the ConvModule module and RepVGGBlock module in the target YOLOv6 model.
[0099] The addition module is used to perform the addition operation in the SPPFBottleneck module in the target YOLOv6 model.
[0100] According to the implementation principle of the above network model module, when the model accelerator needs to implement the function of the ConvModule module, the convolution module performs the convolution operation to calculate the fusion result:
[0101] BN(Conv(x))=W fused (x)+B fused
[0102] Then the activation function module processes the fusion results.
[0103] When the model accelerator needs to implement the function of the RepVGGBlock module, the convolution module is used to place the weights of Conv1×1 at the center of Conv3×3 and fill other positions with zero, and the bias is directly added; or it is used to set the convolution kernel parameters of the front channel to 1 and the convolution kernel parameters of the remaining channels to 0; then the activation function module processes the results output by the convolution module.
[0104] The RepStageBlock module includes multiple RepVGGBlock modules. When the model accelerator needs to implement the function of the RepStageBlock module, the implementation of each RepVGGBlock module is the same as the implementation of the function of the RepVGGBlock module by the model accelerator.
[0105] The SPPFBottleneck module includes multiple ConvModule modules, multiple maximum pooling operations and addition operations. When the model accelerator needs to implement the functions of the SPPFBottleneck module, the implementation of each ConvModule module is the same as the implementation of the ConvModule module by the model accelerator, and then combined with the maximum pooling module and the addition module to implement the corresponding processing.
[0106] Step 27, after step 261-step 263 is executed once, a layer of calculation of three lines of road image data to be detected is completed. According to the implementation principle of the target YOLOv6 model, multiple layers of calculation are required. The structure of each layer of the detection system of this embodiment is the same. When performing each layer of calculation, the corresponding module can be reused. After the three lines of road image data to be detected are calculated once, the output result is used as the input of the current model accelerator for the next calculation until the last calculation is completed, and the result is cached in the output cache module. The output cache module is used to cache the final feature map data, the bounding box information of the detected pit and the category probability.
[0107] Step 28, by post-processing the bounding box information, such as non-maximum suppression, removing the results of repeated detection, the final tunnel position and range are determined. The detected tunnel information is output to the relevant traffic management department or road maintenance personnel. The results can be sent to the monitoring center through wireless communication, or displayed on the display screen on site. The traffic management department can set warning signs in time according to the detection results, arrange road maintenance, and ensure road traffic safety.
[0108] A third aspect of an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a road pit detection system based on YOLOv6 and FPGA provided in the above-mentioned embodiment of the present invention is implemented.
[0109] A fourth aspect of an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a road pit detection system based on YOLOv6 and FPGA provided in the above-mentioned embodiment of the present invention is implemented.
[0110] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0111] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware devices.
[0112] The system provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.
[0113] As for the device / electronic device embodiment, since it is basically similar to the system embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the system embodiment.
[0114] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0117] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A road pit detection system based on YOLOv6 and FPGA, characterized in that: include: A training module is used to obtain a training set of road tunnel images and train the YOLOv6 model to obtain a target YOLOv6 model; A quantization module, used to quantize the weights of the target YOLOv6 model to obtain quantized data; A model accelerator, configured to perform convolution, pooling and activation processing on the road image data to be detected according to the quantized data cached in the weight cache module, and output feature map data, bounding box information of the detected pit and category probability; The model accelerator uses the hardware description language Verilog to describe the logic circuits of convolution, pooling and activation processing; the feature map data output after the road image data to be detected is processed once in the model accelerator is input into the model accelerator again for processing until the last processing is completed.
2. The system according to claim 1, characterized in that The step of quantizing the weight of the target YOLOv6 model to obtain quantized data includes: The weights of the target YOLOv6 model are quantized to int8 type to obtain quantized data; wherein the value range of the quantized data is [-128, 127].
3. The system according to claim 1, characterized in that The model accelerator comprises: a computing module, wherein the computing module comprises: a convolution module, a maximum pooling module, an addition module and an activation function module; The convolution module is used to perform convolution operations of the ConvModule module, RepVGGBlock module, RepStageBlock module and SPPFBottleneck module in the target YOLOv6 model; The maximum pooling module is used to perform the maximum pooling operation of the SPPFBottleneck module in the target YOLOv6 model; The activation function module is used to perform activation operations on the ConvModule module and the RepVGGBlock module in the target YOLOv6 model; The addition module is used to perform the addition operation in the SPPFBottleneck module in the target YOLOv6 model.
4. The system according to claim 3, characterized in that When performing the convolution operation in the ConvModule module in the target YOLOv6 model, the convolution module is used to calculate the fusion result; The calculation formula of the fusion result is: BN(Conv(x))=W fused (x)+B fused in, x is the input, b is the convolution layer bias, mean is the mean of a batch input, is the standard deviation of the current batch, γ is the scaling parameter used to adjust the scale of the standardized value, and β is the translation parameter used to adjust the offset of the standardized value.
5. The system according to claim 3, characterized in that When performing the convolution operation of the RepVGGBlock module in the target YOLOv6 model, the convolution module is used to place the weight of Conv1×1 at the center of Conv3×3, fill other positions with zero, and directly add the bias; or to set the convolution kernel parameters of the front channel to 1 and the convolution kernel parameters of the remaining channels to 0.
6. The system according to claim 3, characterized in that The model accelerator also includes: a row cache module; The line buffer module is used to buffer three lines of the road image data to be detected, and input the road image data to be detected into the calculation module.
7. The system according to claim 3, characterized in that The model accelerator also includes: a layer control module; The layer control module is used to input the quantized data in the weight cache module into the calculation module.
8. The system according to claim 3, characterized in that Also includes: Output cache module; The output cache module is used to cache feature map data, bounding box information and category probabilities of detected tunnels.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the road pit detection system based on YOLOv6 and FPGA is implemented as described in any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the road pit detection system based on YOLOv6 and FPGA according to any one of claims 1 to 7 is implemented.