A GPU-based vehicle video detection method

Abstract

This invention provides a GPU-based vehicle video detection method, belonging to the field of vehicle image recognition technology. The method includes real-time video stream decoding, format and resolution conversion, and target detection model training and detection. By using a GPU for video decoding and detection, the method can meet the demand for real-time, concurrent vehicle detection of large amounts of video images.

Description

Technical Field

[0001] The present invention relates to the fields of vehicle image recognition and video image decoding, and in particular to a GPU-based vehicle video detection method. Background Art

[0002] With the advancement of science and technology, urban road traffic systems are becoming increasingly developed. Vehicle detection algorithms based on real-time video have become the mainstream means of vehicle detection in current intelligent transportation systems. How to complete vehicle detection in real time, efficiently, and accurately is a current research hotspot.

[0003] In the field of vehicle detection, in addition to more traditional machine learning algorithms such as SVM classifiers, target detection algorithms based on deep learning are widely adopted by various systems due to their high recognition accuracy, especially their ability to maintain a high recognition rate in complex scenarios.

[0004] Currently, deep learning-based object detection algorithm frameworks include the R-CNN series (R-CNN, Fast R-CNN, Faster R-CNN), the YOLO series (YOLO v1, YOLO v2, YOLO v3, YOLO v4), SSD, and Cascade R-CNN. These frameworks significantly improve vehicle detection compared to previous approaches that manually designed features and then used classifiers like SVM. However, they also significantly increase the hardware and computing resources required.

[0005] The aforementioned deep learning algorithm frameworks can leverage the powerful parallel processing capabilities of GPUs for vehicle detection. However, for video stream detection, this often requires using video processing frameworks such as OpenCV to decode the video stream into frame-by-frame images before performing detection. The FFmpeg framework, on the other hand, relies on the CPU for video decoding, which has relatively limited parallel processing capabilities. Parallel vehicle detection on large numbers of video streams requires a CPU-GPU collaboration, with the CPU performing video decoding first and the GPU performing image detection. However, CPU computing resource constraints often limit the number of concurrent detections and hinder the full utilization of GPU resources.

[0006] A GPU, short for Graphic Processing Unit, is a microprocessor specialized for processing graphics operations. Compared to a CPU, a GPU has more processing units and greater memory bandwidth, resulting in superior computing power and suitability for large-scale computations.

[0007] VPF, short for Video Processing Framework, is a set of C++ libraries and Python bindings that provide full hardware acceleration for video processing tasks such as decoding, encoding, transcoding, and GPU-accelerated color space and pixel format conversion. Compared to OpenCV, VPF offloads all decoding steps to the GPU, making CPU processing more efficient.

[0008] The Yolo algorithm, whose full name is You Only Look Once: Unified, Real-Time Object Detection, uses only a single CNN network to directly predict the category and location of different targets. The algorithm is fast and can process real-time video streams with a latency of less than 25 milliseconds.

[0009] The YOLO-v4 algorithm builds on the existing YOLO object detection architecture and utilizes the most advanced optimization strategies in recent years in the CNN field. It offers varying degrees of optimization across various aspects, including data processing, backbone networks, network training, activation functions, and loss functions. The YOLOv4-tiny architecture is a streamlined version of YOLOv4, a lightweight model with only 6 million parameters, one-tenth the original. This significantly improves detection speed. Summary of the Invention

[0010] In order to solve the above technical problems, the present invention provides a method for vehicle recognition using a GPU for video decoding based on the VPF framework and combined with a Yolov4-tiny lightweight network model. The method can meet the needs of simultaneously processing a large number of video streams for parallel vehicle detection, has high recognition accuracy and fast detection speed, and has low CPU computing resource usage. At the same time, GPU resources can be fully utilized, which can meet the needs of real-time and concurrent vehicle detection of a large number of video images in today's intelligent road traffic systems.

[0011] The technical solution of the present invention is:

[0012] A GPU-based vehicle video detection method.

[0013] The steps include:

[0014] (1) Use the PyNvDecoder method in the PyNvCodec module in the VPF framework to decode the original video stream output by the surveillance camera to obtain a frame of image data; the original image resolution is required to be no less than 1920*1080.

[0015] (2) Use the PySurfaceConverter method in the PyNvCodec module in the VPF framework to convert the image data in step (1) into YUV420 data format;

[0016] (3) Use the PySurfaceResizer method in the PyNvCodec module in the VPF framework to modify the image size converted in step (2) to a resolution of 1280*720;

[0017] (4) Using the PySurfaceConverter method in the PyNvCodec module in the VPF framework, the image data in step (3) is converted into RGB data format;

[0018] (5) Annotate the position and type of the vehicles in the image in step (4), incorporate them into the convolutional neural network model for target detection training, and finally output a target detection model for vehicle identification;

[0019] (6) Load the target detection model and perform target detection and recognition on the image taken in step (4). The recognition resolution is 416*416 and the confidence threshold is set to 0.45. Identify and obtain the vehicle position and corresponding vehicle type in the image.

[0020] Further,

[0021] In step (1), the video is decoded using the PyNvDecoder in the PyNvCodec module in the VPF framework. This method uses the GPU for recognition decoding.

[0022] Further,

[0023] In step (2), the format conversion is performed using the PySurfaceConverter method in the PyNvCodec module in the VPF framework, which uses the GPU for processing.

[0024] Further,

[0025] In step (3), the image resolution is adjusted using the PySurfaceResizer method in the PyNvCodec module in the VPF framework. This method uses the GPU for processing.

[0026] Further,

[0027] In step (4), the format conversion is performed using the PySurfaceConverter method in the PyNvCodec module in the VPF framework, which uses the GPU for processing.

[0028] Further,

[0029] The labeling and training actions described in step 5) are only performed when there is no target detection model or the model needs to be optimized.

[0030] The Yolov4-tiny detection network based on Darknet is used for training. 4800 vehicle sample images with a resolution of 1280*720 are used for training. The network image resolution is 608*608 during training, and the number of training rounds is 200,000.

[0031] Further,

[0032] In step (6), the recognition resolution is 416*416, and the confidence threshold is set to 0.45. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] The present invention is a GPU-based vehicle video detection algorithm, and its technical solution includes the following steps:

[0035] (1) Use the PyNvDecoder method in the PyNvCodec module in the VPF framework to decode the original video stream output by the surveillance camera to obtain a frame of image data. The original image resolution must be no less than 1920*1080.

[0036] (2) Use the PySurfaceConverter method in the PyNvCodec module in the VPF framework to convert the image data in step (1) into YUV420 data format.

[0037] (3) Use the PySurfaceResizer method in the PyNvCodec module in the VPF framework to modify the image size converted in step (2) to a resolution of 1280*720.

[0038] (4) Use the PySurfaceConverter method in the PyNvCodec module in the VPF framework to convert the image data in step (3) into RGB data format.

[0039] (5) The position and type of the vehicles in the image in step (4) are annotated and incorporated into the convolutional neural network model for target detection training, and finally a target detection model for vehicle identification is output. The annotation and training actions of this step are only performed when there is no target detection model or the model needs to be optimized. The present invention uses the Yolov4-tiny detection network implemented based on Darknet for training. Yolov4-tiny is a simplified version of Yolo3. Its advantages are simple network, small amount of calculation, and can be run on mobile or device ends. 4800 vehicle sample pictures with a resolution of 1280*720 are used for training, and the vehicles in the pictures are annotated, where different types of vehicles are annotated separately to distinguish vehicle models. The network image resolution during training is 608*608, and the number of training rounds is 200,000 rounds.

[0040] (6) Load the target detection model and perform target detection and recognition on the image taken in step (4). The recognition resolution is 416*416 and the confidence threshold is set to 0.45. Identify and obtain the vehicle position and corresponding vehicle type in the image.

[0041] The above description is only a preferred embodiment of the present invention and is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A GPU-based vehicle video detection method, characterized in that: The steps include: (1) Use the PyNvDecoder method in the PyNvCodec module in the VPF framework to decode the original video stream output by the surveillance camera to obtain a frame of image data; the original image resolution must be no less than 1920*1080; (2) Use the PySurfaceConverter method in the PyNvCodec module in the VPF framework to convert the image data in step (1) into YUV420 data format; (3) Use the PySurfaceResizer method in the PyNvCodec module in the VPF framework to modify the image size converted in step (2) to a resolution of 1280*720; (4) Use the PySurfaceConverter method in the PyNvCodec module in the VPF framework to convert the image data in step (3) into RGB data format; (5) Label the location and type of the vehicles in the image in step (4), incorporate them into the convolutional neural network model for target detection training, and finally output a target detection model for vehicle identification; (6) Load the target detection model and perform target detection and recognition on the image taken in step (4). The recognition resolution is 416*416 and the confidence threshold is set to 0.

45. Identify and obtain the vehicle position and corresponding vehicle type in the image; In step (1), the PyNvDecoder in the PyNvCodec module in the VPF framework is used to perform video decoding. This method uses the GPU for recognition decoding. In step (2), the format conversion is performed using the PySurfaceConverter method in the PyNvCodec module in the VPF framework, which uses the GPU for processing; In step (3), the image resolution is adjusted using the PySurfaceResizer method in the PyNvCodec module in the VPF framework, which uses the GPU for processing; In step (4), the format conversion is performed using the PySurfaceConverter method in the PyNvCodec module in the VPF framework, which uses the GPU for processing; The labeling and training actions described in step 5) are only performed when there is no target detection model or the model needs to be optimized.

2. The method according to claim 1, characterized in that In step (5), the Yolov4-tiny detection network based on Darknet is used for training. 4800 vehicle sample images with a resolution of 1280*720 are used for training. The network image resolution during training is 608*608, and the number of training rounds is 200,000.

3. The method according to claim 1, characterized in that In step (6), the recognition resolution is 416*416, and the confidence threshold is set to 0.45.