Vehicle high beam detection method and system

Abstract

The invention discloses a vehicle high beam detection method and system, and relates to the technical field of computer vision, a YOLOv8 network is improved by designing a feature fusion global attention module (CGM), a convolution feature recombination module (CCM) and a small target detection module (CSTC), and a vehicle high beam data set is adopted to train an improved YOLOv8 target detection model, so that the detection accuracy of a vehicle high beam is improved. A model adaptive to an edge computing hardware platform is obtained through format conversion and quantization; the core number matched with the hardware resource specification is adopted in an edge calculation hardware platform, collaborative reasoning is carried out on the quantized model based on night road vehicle images obtained in real time, and a real-time vehicle high beam detection result is rapidly obtained. Through targeted optimization of the architecture and deployment mode of the vehicle high beam detection model, the speed and performance of the model deployed in edge computing equipment are effectively improved, and real-time robust vehicle high beam detection is realized based on edge computing.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic safety, and in particular to a vehicle high beam detection method and system. Background Art

[0002] According to the statistical analysis of traffic accident data from the Ministry of Public Security, about 30% to 40% of nighttime traffic accidents are caused by illegal use of high beams. Patrolling on the road by police is inefficient.

[0003] In order to significantly improve the accuracy and efficiency of vehicle high beam detection, deep learning technology has become a powerful means. In the field of deep learning target detection, there are two main types of algorithms: two-stage detection algorithms and one-stage detection algorithms. Two-stage algorithms, such as spatial pyramid pooling convolutional network (SPP-NET) and faster region-based convolutional neural network (Faster R-CNN), have bottlenecks in detection speed because they need to generate candidate regions before detection, and it is difficult to meet the needs of real-time detection.

[0004] In contrast, one-stage detection algorithms, such as Only Look Once (YOLO) and Single-shot Multibox Detector (SSD), achieve end-to-end fast object detection by directly regressing the location information and category probability of objects.

[0005] With the rise of edge computing, vehicle high beam detection has ushered in new development opportunities. Edge computing brings data processing and computing closer to the source of data generation, thereby reducing data transmission latency. Researchers have successfully deployed the YOLOV5 algorithm for target detection on edge computing chips such as Rockchip, Huawei HiSilicon, and NVIDIA. At the same time, some scholars have optimized algorithms such as YOLOV5, such as using the mobilenetv1 network to replace the backbone of the original YOLOV5 network. Although the edge computing speed has been improved, the algorithm accuracy has been reduced. In addition, the backbone network of YOLOV7 is combined with the weighted bidirectional feature pyramid structure (BiFPN) to effectively solve the problem of information loss during the sampling process.

[0006] Although the edge computing speed is improved by replacing the backbone of the original YOLOV5 network with the mobilenetv1 network, this is at the expense of algorithm accuracy. In some application scenarios with high accuracy requirements, this loss of accuracy may be unacceptable. Although the YOLO series of algorithms perform well in the field of target detection, it is still a challenge to detect small targets. In particular, in the detection of small-sized targets such as vehicle high beams, further optimization may be required to improve detection performance. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art of low accuracy in vehicle high beam detection, the main purpose of the present invention is to provide a vehicle high beam detection method and system to improve the accuracy, real-time and adaptability of vehicle high beam detection, especially in an edge computing environment, while ensuring the detection capability of small targets and optimizing the computational efficiency of the model to adapt to resource-constrained devices, so as to respond in real time and improve the generalization capability and robustness of the vehicle high beam detection system.

[0008] To achieve the above object, the present invention adopts the following technical solution, a vehicle high beam detection method, comprising the following steps:

[0009] An improved target detection model is constructed based on the YOLOv8 network. The improved target detection model introduces a feature fusion global attention module CGM into the backbone network of the original YOLOv8 network, and introduces a convolutional feature recombination module CCM into the neck network of the original YOLOv8, as well as a small target detection module CSTC formed by stacking a convolutional feature recombination module, a connection layer, and a feature fusion module;

[0010] Acquire a plurality of preprocessed vehicle high beam image data sets, and train the improved target detection model using the preprocessed high beam image data sets to obtain a trained target detection model;

[0011] The high beam image of the vehicle to be tested is obtained as the input of the trained target detection model, and feature maps of different scales are constructed through the backbone network processing of the feature fusion global attention module CGM. The feature maps of different scales are processed through the neck network of the convolutional feature recombination module CCM and the small target detection module CSTC to obtain optimized recombined feature maps, and the optimized recombined feature maps are sent to the YOLO detection layer to obtain the vehicle high beam detection result.

[0012] The feature fusion global attention module CGM includes a feature fusion module C2f and an attention mechanism GAM;

[0013] The convolutional feature reconstruction module CCM includes a 1*1 convolution and upsampling operator CARAFE.

[0014] The method of acquiring a plurality of vehicle high beam images and then preprocessing them to obtain a preprocessed high beam image dataset includes the following steps:

[0015] Acquire multiple vehicle high beam images, perform enhancement processing, and obtain multiple enhanced vehicle high beam images, wherein the enhancement processing includes: one or more of random rotation, scaling, cropping, brightness adjustment, and adding noise;

[0016] An image annotation tool is used to annotate the enhanced vehicle high beam image, and the vehicle high beam image is converted into a YOLO data set format including target position coordinates and target categories to obtain the vehicle high beam data set.

[0017] A vehicle high beam detection system based on edge computing, comprising:

[0018] A data acquisition module acquires multiple vehicle high beam images and then performs preprocessing to obtain a preprocessed high beam image data set;

[0019] A detection model construction module is used to construct a target detection model based on the YOLOv8 network, wherein a feature fusion global attention module CGM is introduced into the backbone network of the YOLOv8 network, and a convolutional feature recombination module CCM and a small target detection module CSTC are introduced into the neck network of the YOLOv8 to obtain an improved target detection model; the improved target detection model is trained using the preprocessed high beam image data set to obtain a trained target detection model;

[0020] The detection result acquisition module obtains the high beam image of the vehicle to be tested as the input of the trained target detection model, and constructs feature maps of different scales through the backbone network processing of the feature fusion global attention module CGM; the feature maps of different scales are processed by the neck network of the convolution feature recombination module CCM and the small target detection module CSTC to obtain the optimized recombined feature map, and the optimized recombined feature map is sent to the YOLO detection layer to obtain the vehicle high beam detection result.

[0021] The data acquisition module includes an image acquisition device, an edge computing hardware platform and an image output device; the image acquisition device is used to acquire a real-time video stream containing a vehicle high beam image, and transmit the real-time video stream to the edge computing hardware platform;

[0022] The edge computing hardware platform performs vehicle high beam detection on the nighttime road vehicle image contained in the real-time video stream, and outputs a vehicle high beam detection result, wherein the vehicle high beam detection result includes: location information and category probability information of the vehicle high beam;

[0023] The image output device is used to superimpose the vehicle high beam detection result onto the real-time video stream and output the real-time detection video stream, wherein the real-time detection video stream displays the position and category probability of the vehicle high beam.

[0024] Also includes:

[0025] The detection result acquisition module also includes a format conversion unit, which converts and quantizes the weight parameters of the trained target detection model to obtain a quantization model that matches the format of the edge computing hardware platform; obtains a high beam image of the vehicle to be tested, inputs the high beam image of the vehicle to be tested into the quantization model, and obtains a vehicle high beam detection result.

[0026] The weight parameters of the trained target detection model are formatted and quantized to obtain a quantized model that matches the format of the edge computing hardware platform, including:

[0027] Converting the weight parameters of the training network model into a weight file in an open neural network exchange format;

[0028] Call the configuration interface to obtain the preprocessing parameters of the neural network software development kit that matches the edge computing hardware platform, create an RKNN object to initialize the neural network software development kit; import the weight file into the neural network model development tool, call the creation interface, and create an initial RKNN model based on the RKNN object;

[0029] Calling the application program interface of the neural network model development tool, evaluating the memory usage of the initial RKNN model, and performing simulation reasoning until the initial RKNN model achieves the preset expected effect;

[0030] The initial RKNN model that achieves the expected effect is quantized through the 8-bit asymmetric quantization function of the neural network software development kit to obtain a quantized model in the RNKK format that matches the edge computing hardware platform format.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention designs a feature fusion global attention module CGM, a convolution feature reorganization module CCM and a small target detection module CSTC to improve the backbone network and the neck network of the target detection model. By introducing the feature fusion global attention module CGM, the convolution feature reorganization module CCM and the small target detection module CSTC in the YOLOv8 network, the model's detection capability of vehicle high beams is enhanced, especially the detection accuracy of small targets. The CSTC module is optimized specifically for small targets. By adding additional convolution layers and attention mechanisms, the detection effect of small-sized high beams is improved, and missed detections and false detections are reduced. The introduced CGM module strengthens the global context information of the feature map through feature fusion and global attention mechanisms, so that the model can better understand and distinguish different targets; CCM reduces the amount of calculation and improves the reasoning speed of the model through the reorganization and optimization of the feature map, which is suitable for real-time detection applications, increases the model's adaptability to various scenarios, and improves the generalization ability. The vehicle high beam detection system based on edge computing can collect, process and display detection results in real time, providing instant feedback for drivers or traffic monitoring systems. Through format conversion and quantization, the trained model is adapted to the edge computing hardware platform, optimizing the model's storage and computing requirements, making it suitable for deployment in resource-constrained edge environments; the quantized model reduces the model size and computational complexity, reducing the cost of deployment on edge devices while maintaining high detection performance. Accurately detecting vehicle high beams helps reduce traffic accidents caused by improper use of high beams and improves nighttime driving safety.

[0032] In summary, this technical solution not only improves the accuracy and efficiency of vehicle high beam detection, but also realizes real-time detection and low-cost deployment through edge computing, providing effective technical support for traffic safety management.

[0033] In the edge computing hardware platform, the number of cores that matches the hardware resource specifications of the edge computing hardware platform is used to perform collaborative reasoning on the deployed model to quickly obtain real-time detection results. Therefore, the speed and performance of model deployment in edge computing devices are effectively improved, and real-time and robust vehicle high-beam detection is achieved based on edge computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0035] Figure 1 is a flow chart of a vehicle high beam detection method based on edge computing provided by the present invention according to an exemplary embodiment;

[0036] Figure 2 is a structural diagram of an improved Y0L0v8-n model provided by the present invention according to an exemplary embodiment;

[0037] Figure 3 is a schematic diagram of a network module in an improved Y0L0v8-n model provided by the present invention according to an exemplary embodiment;

[0038] Figure 4 is a vehicle high beam detection result diagram obtained by using an improved Y0L0v8-n model according to an exemplary embodiment of the present invention;

[0039] Figure 5 is a structural diagram of a multi-threaded edge computing vehicle high beam detection system provided by the present invention according to an exemplary embodiment;

[0040] Figure 6 is a schematic diagram of a vehicle high beam detection method based on edge computing provided by the present invention according to an exemplary embodiment;

[0041] Figure 7 is a schematic diagram of a hardware structure of a computing device provided by the present invention according to an exemplary embodiment;

[0042] Figure 8 It is a schematic diagram of the process structure of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0044] The vehicle high beam detection method provided by the present invention includes several main processes, including data set preparation, model building and model deployment. Among them, the PC end device is responsible for model building, model conversion and other processes. After obtaining the improved model for vehicle high beam detection, it is transplanted to the edge computing hardware platform for deployment, and the edge computing hardware platform performs real-time vehicle high beam detection. Figure 8 A schematic diagram of a vehicle high beam detection method provided by the present invention is shown below in conjunction with Figure 8 The vehicle high beam detection method provided by the present invention is introduced. The method comprises the following steps:

[0045] Acquire multiple vehicle high beam images, and then preprocess them to obtain a preprocessed high beam image dataset;

[0046] A target detection model is constructed based on the YOLOv8 network, wherein a feature fusion global attention module CGM is introduced into the backbone network of the YOLOv8 network, and a convolutional feature recombination module CCM and a small target detection module CSTC are introduced into the neck network of the YOLOv8 to obtain an improved target detection model;

[0047] Using the preprocessed high beam image data set to train the improved target detection model to obtain a trained target detection model;

[0048] The high beam image of the vehicle to be tested is obtained, and the high beam image of the vehicle to be tested is input into the trained target detection model to obtain a vehicle high beam detection result.

[0049] Example 1

[0050] See also Figure 1 When the present invention is operated, it is implemented according to the following steps S1 to S6.

[0051] S1. The PC device constructs a vehicle high beam data set based on multiple training vehicle high beam images.

[0052] In the embodiment of the present invention, firstly, data set collection and processing are performed to construct a vehicle high beam data set for training the model. This step S1 includes the following steps S11 to S13.

[0053] S11. Extracting a plurality of high beam images of training vehicles from the video captured by the camera in the target site.

[0054] In some embodiments, the training vehicle high beam images are collected from target sites in multiple regions under various weather conditions to enhance the richness of the environmental background of the vehicle high beam images. Exemplarily, a video camera is used to record nighttime road vehicle videos in different regions and weather conditions, and multiple training vehicle high beam images are captured from the videos. Optionally, a target number of images, for example, 1,000 images, are selected from the captured multiple training vehicle high beam images. Optionally, the image is in RGB format. S12. The training vehicle high beam images are enhanced in the following manner.

[0055] One or more of randomly rotating, scaling, cropping, adjusting brightness, and adding noise is performed on the training vehicle high beam image.

[0056] In some embodiments, through the above enhancement process, a larger number of training vehicle high beam images can be obtained in the end. For example, 1000 images are selected during collection and 3000 images are obtained through enhancement.

[0057] S13. Use an image annotation tool to annotate the enhanced training vehicle high beam image, convert the training vehicle high beam image into a Y0L0 data set format containing target position coordinates and target categories, and obtain a vehicle high beam data set; divide the vehicle high beam data set into a training set, a validation set, and a test set for neural network training, verification, and testing according to a preset ratio.

[0058] In some embodiments, the enhanced training vehicle high beam images are manually labeled using the LabelImg tool, and the image data is converted into a Y0L0 dataset format containing target categories and target position coordinates as a vehicle high beam dataset for neural network training. Optionally, the vehicle high beam dataset is divided into a training set, a validation set, and a test set according to a preset ratio, such as 7:1:2.

[0059] S2. The PC device designs a feature fusion global attention module CGM in the backbone network of the target detection model YOLOv8, and designs a convolutional feature recombination module CCM and a small target detection module CSTC in the neck network to obtain an improved YOLOv8 network.

[0060] The target detection model adopted in the embodiment of the present invention is the YOLOv8-n model, and an improved target detection model is finally generated. Since the improved YOLOv8-n model has fewer parameters and faster calculation speed, it can be run on mobile computing devices or distributed edge servers and devices.

[0061] In some embodiments, this step S2 includes the following steps S21-S22.

[0062] S21. Build a YOLOv8-n network training project based on the PyTorch training framework.

[0063] S22. Based on the YOLOv8-n network training project, CGM is designed in the backbone network of the YOLOv8-n model, CCM and CSTC are designed in the neck network, and a network structure whose performance meets the preset performance target is searched through neural network search technology, and the improved YOLOv8 target detection model is generated based on the network structure.

[0064] Figure 2 is a structural diagram of an improved YOLOv8-n model provided by the present invention according to an exemplary embodiment, Figure 3 Schematic diagram of a network module in an improved YOLOv8-n model provided by the present invention according to an exemplary embodiment. Figure 2 and Figure 3 The improved YOLOv8-n model provided by the present invention is introduced.

[0065] refer to Figure 2 The improved YOLOv8-n model provided by the present invention includes: a backbone network, a neck network and a detection head.

[0066] The input is the vehicle high beam image to be trained (vehicle high beam dataset), and the adaptive anchor frame calculation, adaptive image scaling and data enhancement methods are used to preprocess the input image. Preprocessing can enrich the diversity of input images, improve the detection effect of small targets, and enhance the network model's ability to detect vehicle high beam targets.

[0067] A feature fusion global attention module is designed in the backbone network. It is a vehicle high beam detection-friendly network improved on the basis of the feature fusion module. It reduces information loss in the feature downsampling process and amplifies the global dimensional interaction characteristics. The feature fusion global attention module is composed of a stack of feature fusion modules and global attention modules. The feature fusion module contains a convolutional layer, a separation layer, a dark network bottleneck block, and a connection layer. The global attention module consists of two parts: a channel attention submodule (CAM) and a spatial attention submodule (SAM). Through CAM, the network can learn channel features that are more important for vehicle high beam recognition. Through SAM, the model can focus on the key areas of the feature map to extract more important features. The calculation formula of GAM is as follows: F1 is the input feature map, F2 is the intermediate feature map, F3 is the output feature map, and M C and M S They are channel attention feature map and spatial attention feature map respectively. Represents element-wise multiplication.

[0068]

[0069]

[0070] The convolutional feature reorganization module and the small target detection module are designed in the neck network. The convolutional feature reorganization module realizes the feature enhancement of upsampling. The small target detection module enables the network to generate a feature map with a smaller perception field of view to enhance the detection of small targets. The convolutional feature reorganization module and the small target detection module are two network modules that are friendly to vehicle high beam detection. The convolutional feature reorganization module is composed of a stack of point-by-point convolution and a content-aware feature reorganization module. The content-aware feature reorganization module consists of two parts: the kernel prediction module (Kernel PredictionModule) and the content-aware reassembly module (Content-awareReassemblyModule). The small target detection module is composed of a stack of three parts: the convolutional feature reorganization module, the connection layer, and the feature fusion module.

[0071] The kernel prediction module ψ is based on each target location area N(X l ,k up) content predicts its reconstruction kernel W l' , the calculation formula is as follows. X l is a position in the input feature map X, k up is the size of the reorganization kernel, k encoder is the size of the convolution.

[0072] W l' =ψ(N(X l ,k encoder ))

[0073] The content-aware reorganization module φ uses the predicted kernel W l' and the original target position N(X l ,k up )Reconstruct feature X l ", the calculation formula is as follows. The original target position is centered on l(i,j), l' is the corresponding target position, and r = k up / 2.

[0074]

[0075] See also Figure 3 , the feature extraction module includes convolution layer (Conv2d) - batch normalization layer (BatchNorm2d) - activation layer (SiLU). The spatial pyramid pooling module includes convolution layer (Conv) - maximum pooling layer (MaxPool2d) - connection layer (Concat) - convolution layer (Conv). The decoupling head includes feature extraction module and convolution layer (Conv2d).

[0076] See also Figure 2 The neck network and the detection head are used to enhance the network's feature extraction capability and further improve the network's detection capability for vehicle high beam images, including: fusing the features given by the backbone network to generate bounding boxes and predict categories. The small target detection module and the three subsequent feature fusion modules each output a feature map of different sizes, and the detection head performs decoupling operations on these four feature maps of different sizes to obtain prediction results.

[0077] S3. The PC device trains an improved target detection model based on the vehicle high beam data to obtain a training model that meets the preset training objectives.

[0078] In the embodiment of the present invention, the vehicle high beam data set of step S1 is sent to the improved YOLOv8-n model obtained in step S2 for training to obtain a training model.

[0079] Specifically, an improved YOLOv8-n model is used for training, an Early Stopping training mechanism is adopted, the Patience is set to 50, the number of training rounds is 500, the input image size is 640*640, and the model that meets the preset training goals during the training process is saved, where the preset training goal can be that the number of training rounds reaches the set number of training rounds, or that the recognition error of the model reaches a preset value.

[0080] Through the above steps S1 to S3, data set preparation, model improvement and model training are completed in sequence to obtain a model that meets the preset training objectives.

[0081] Next, through steps S4 and S5, the process of converting the model format and quickly porting it to the edge computing hardware device is introduced.

[0082] S4. The PC device converts and quantizes the weight parameters of the training model to obtain a quantized model that matches the format of the edge computing hardware platform.

[0083] The embodiment of the present invention uses Rockchip's RK3588S platform with built-in NPU (neural network processing unit) as the edge computing hardware platform for deploying the model. RK3588S supports AI acceleration and has a higher-performance CPU architecture and stronger GPU performance.

[0084] Among them, the neural network model development tool is RKNN-Toolkit2 provided by Rockchip, which is used to achieve rapid transplantation and application development of trained models.

[0085] In the embodiment of the present invention, step S4 includes:

[0086] S41. Format conversion: Convert the weight parameters of the training model into a weight file in the Open Neural Network Exchange (ONNX) format.

[0087] In some embodiments, before exporting the model in the open neural network exchange ONNX format, the exported operator version is pre-set. The open neural network exchange ONNX supports multiple operator versions, and each version corresponds to a different set of operations. Specifically, the exported operator version is pre-set according to the operator version supported by the edge computing hardware platform to be deployed. For example, the highest operator version supported by the RK3588S platform in the embodiment of the present invention is 12, so the operation set version (opsetversion) variable is set to 12.

[0088] S42. Quantization: Based on the quantization function of the neural network model development tool (Software Development Kit, SDk), the weight file is converted into a lightweight model in the RKNN (Rockchip Neural Network) format.

[0089] Specifically, first call the configuration interface to obtain the preprocessing parameters of the neural network model development tool (RKNN-Toolkit2) that matches the edge computing hardware platform, create an RKNN object to initialize the software development environment of RKNN-Toolkit2; import the weight file into RKNN-Toolkit2, call the build interface, and create the initial RKNN model based on the RKNN object: call the application programming interface (API) of RKNN-Toolkit2, evaluate the memory usage of the initial RKNN model, and perform simulated reasoning until the initial RKNN model achieves the preset expected effect: enable the 8-bit asymmetric quantization function of the neural network software development kit, quantize the initial RKNN model that achieves the expected effect, and export the lightweight model in RNKK format.

[0090] In an embodiment of the present invention, the PC-side device exports the weight parameters of the trained lightweight model, and uses RKNN Toolkit2 to quantize the model and convert the format. The model is quantized using an asymmetric 8-bit quantization method. After the model is quantized, the weight information of the model will be saved with lower precision. After deployment, the memory space occupied by the model can be reduced and the model reasoning speed can be accelerated. Finally, the model is converted to the RKNN format and can be quickly ported to the edge computing hardware platform.

[0091] In some embodiments, the edge computing hardware platform needs to be configured with software and hardware resources related to the model application and the PC. For example, the debian system and the tailored ARM-linux kernel are transplanted to the RK3588S platform in advance, and the firmware version supported by the NPU, the development environment and library files required by the deep learning network are installed. In terms of application software, a cross-compilation environment needs to be established between the PC and Opencv, Cmake-gui are installed, and the model file is transferred using MobaXterm.

[0092] S5. The PC device transmits the lightweight model to the edge computing hardware platform. The edge computing hardware platform uses a core number that matches the hardware resource specifications of the edge computing hardware platform. Based on the real-time acquired vehicle high beam image, the lightweight model is collaboratively inferred and the detection result of the vehicle high beam image is output. The vehicle high beam detection result includes: vehicle high beam location information and category probability information.

[0093] For example, the lightweight model in RKNN format received from the PC device is loaded and run on the RK3588S platform, and the model is inferred by calling the RKNN API interface. Specifically, according to the hardware resource specifications of RK3588S itself, the reasoning of the model is optimized in a targeted manner, and the multi-threaded collaborative computing solution is enabled according to the number of cores that match the hardware resource specifications, thereby improving the performance and computing speed of the model in the hardware platform, so that the subsequent edge computing hardware platform can perform rust recognition in real time at a higher frame rate.

[0094] Specifically, taking the edge computing hardware platform including multi-core NPU as an example, multi-core NPU can realize parallel computing based on multiple threads. According to the hardware resource specifications, the channel width used by NPU hardware DMA to transfer image data can be determined; according to the channel width, the image data processed by the model is divided into multiple parts, and for each part, data is transferred according to the channel width. Then, NPU can perform model inference on image data in parallel based on multiple threads.

[0095] In some embodiments, this step S5 includes:

[0096] 1. Obtain a real-time video stream to be detected, the real-time video stream including multiple frames of vehicle high beam images. Specifically, the real-time video stream can be obtained from an image acquisition device.

[0097] 2. Standardize, 8-bit asymmetric quantize and convert the real-time video stream in the original video format to obtain the video stream in the target format, which matches the hardware resource specifications of the edge computing platform.

[0098] In some embodiments, a third-party video image processing program can be used to complete the pre-processing of the image, which includes the above-mentioned standardization of the input video stream, asymmetric 8-bit quantization and format conversion. Specifically, NHWC represents the dimensional order of the image, where: N represents the batch size (Batch Size), that is, the number of images; H represents the height of the image (Height); W represents the width of the image (Width); C represents the number of channels of the image (Channels), and the data formats before and after pre-processing are: NHWC and NC1HWC2 respectively.

[0099] Among them, C1 and C2 divide the image channel into two parts (or more or fewer parts determined by different hardware resource specifications), and the image data corresponding to the two parts of the channel can be calculated in parallel in the NPU (edge ​​computing hardware platform). Based on this, the efficiency and performance of the edge computing hardware platform for model reasoning can be improved.

[0100] 3. Input the video stream in the target format into the edge computing platform, use the core number that matches the hardware resource specifications of the edge computing hardware platform, perform collaborative reasoning on the lightweight model based on the real-time acquired night road vehicle images to perform vehicle high beam detection, and obtain the vehicle high beam detection result, which includes: vehicle high beam location information and category probability information.

[0101] 4. Superimpose the vehicle high beam detection result onto the corresponding nighttime road vehicle image in the video stream, convert the superimposed video stream into the original video format, and output it to the image display device.

[0102] Among them, the above steps 2 and 4 can be implemented by CPU or GPU, and step 3 can be implemented by multi-core NPU.

[0103] Figure 4 is a vehicle high beam detection result diagram obtained by using an improved Y0L0v8-n model according to an exemplary embodiment of the present invention, see Figure 4 ,The figure uses a rectangular box to mark the position of the vehicle’s high beam lights, and the category probability information of the vehicle’s high beam lights is displayed above the rectangular box, for example, Figure 4 The probability of the car high beam (CarHighBeam) marked in the upper right sub-image is 0.9.

[0104] The present invention provides a vehicle high beam detection method, designs a feature fusion global attention module CGM, a convolution feature recombination module CCM and a small target detection module CSTC to improve the backbone network and neck network of the target detection model, adopts a core number matching the hardware resource specification of the edge computing hardware platform in the edge computing hardware platform, performs collaborative reasoning on the deployed model to quickly obtain a lightweight model, and the edge computing hardware platform can efficiently perform real-time vehicle high beam detection using the lightweight model. Therefore, the technical solution of the present invention effectively improves the speed and performance of model deployment in edge computing devices.

[0105] Furthermore, the technical solution of the present invention uses a variety of data set expansion technologies to expand the vehicle high beam data set, and then uses the multi-core capabilities of the NPU in the edge computing hardware platform to accelerate the reasoning of the model. In some embodiments, the lightweight model deployed in the edge computing hardware platform of the present invention can achieve a real-time vehicle high beam detection speed of 30 frames per second in real-time detection, meeting the real-time robust vehicle high beam detection requirements.

[0106] Example 2

[0107] According to the above-mentioned vehicle high beam detection method, the present invention provides a vehicle high beam detection system based on edge computing, which includes an image acquisition device, an edge computing hardware platform and an image output device.

[0108] The image acquisition device is used to acquire a real-time video stream containing a vehicle high beam image and transmit the real-time video stream to the edge computing hardware platform. The real-time video stream can be a local video input or a nighttime road vehicle video acquired in real time by an image acquisition device such as a camera. In some embodiments, the acquired real-time video stream is also pre-processed (refer to the aforementioned step 2) to facilitate input into the neural network model for calculation.

[0109] The edge computing hardware platform is used to detect the vehicle high beam image contained in the real-time video stream based on the lightweight model provided in Example 1, and output the vehicle high beam detection result, which includes: the location information and category probability information of the vehicle high beam. The model reasoning is completed by the NPU under the RK3588S platform, and the multi-threading method is used in the program design to use the NPU multi-core mode to improve the computing speed.

[0110] The image output device is used to superimpose the vehicle high beam detection result onto the real-time video stream and output the real-time detection video stream, which includes the position and category probability of the vehicle high beam. The output real-time detection video stream can display the position information and category probability information of the vehicle high beam in real time on an image display device, such as a display screen on a PC.

[0111] In some embodiments, the edge computing hardware platform implements multi-threaded model reasoning based on a multi-core neural network processing unit; the image acquisition device and the image output device use a central processing unit and / or an image processing unit to perform corresponding image processing.

[0112] The system provided by the present invention uses a trained lightweight model, is deployed on the RK3588S platform, and optimizes the algorithm operation scheme according to hardware resources to improve the operation speed. The processing flow of the system includes image data pre-processing, model reasoning, and output post-processing.

[0113] Figure 5This is a structural diagram of a multi-threaded edge computing vehicle high beam detection system provided by the present invention according to an exemplary embodiment. In the vehicle high beam recognition process, the image data, that is, the real-time video stream, is first pre-processed by an image acquisition device with integrated CPU / GPU. The pre-processing includes standardization, quantization and format conversion of the image data: Then, an edge computing hardware platform with integrated multi-core NPU, such as core0-2, performs multi-threaded model reasoning. During the reasoning process, RKNNAPI can be called for model reasoning: Finally, the vehicle high beam recognition result and image data are post-processed by an image output device with integrated CPU / GPU, including format conversion and dequantization, and sent to the image display device for display. Among them, the introduction of format conversion refers to the above step 2.

[0114] Example 3

[0115] In combination with the above-mentioned Embodiment 1 and Embodiment 2, the present invention provides a schematic diagram of a vehicle high beam detection method according to an exemplary embodiment. Figure 6 First, the vehicle high beam data, i.e., the vehicle high beam image, is collected through image acquisition devices such as cameras, and the collected data is enhanced to complete the data set preparation. In the model building stage, YOLOv8-n is improved; in the model deployment stage, the improved YOLOv8-n model is trained and then transplanted to the RK3588S platform. Based on this, the image data collected by the camera can be pre-processed and then transmitted to the RK3588S platform for vehicle high beam detection. The detection results can be transmitted to the image display device after post-processing to realize the presentation of real-time detection results.

[0116] Example 4

[0117] In order to better illustrate the performance of the improved lightweight YOLOv8-n model in the technical solution of the present invention, the present invention also provides an experimental analysis process and experimental data.

[0118] Specifically, the experimental analysis was conducted in the Ubuntu 22.04 operating system using Python. The training framework of the network model was Pytorch 2.1.1, the CPU was Intel (R) Xeon (R) Gold 5318Y @ 2.10 GHz, and the GPU was NVIDIA GeForce RTX 4090. Before model training, the relevant parameters such as the number of categories, category names, and training paths were initialized. The image input size was set to 640*640, the maximum number of iterations was set to 500, the number of images per training was set to 16, the weight decay coefficient was set to 0.0005, and the initial learning rate was set to 0.001.

[0119] Table 1 is a comparison of the parameters, operation speed and performance of different models. The models YOLOv6n, YOLOv7 and YOLOv8n are the initial models without any changes, and GCS-YOLOv8 is the improved model of the present invention. The third column parameter is the number of floating point operations per second (FLOPs), and the smaller the value, the lower the computational complexity. The fourth column parameter mAP@0.5 is the average accuracy of the model for target detection, and the higher the percentage, the more accurate it is.

[0120] Table 1 Parameters, operation speed and performance comparison of different models

[0121] Detection Model Parameter quantity / million FLOPs mAP@0.5 / % Model size / MB YOLOv6n 4.63 11.34 92.4 10.2 YOLOv7 36.48 103.2 90.6 73.0 YOLOv8n 3.00 8.1 92.0 6.0 GCS-YOLOv8 5.33 18.4 94.0 10.8

[0122] As can be seen from Table 1, the improved GCS-YOLOv8 model of the present invention achieves the highest average accuracy while maintaining a small number of parameters, low computational complexity and a small model size, achieving a balance between accuracy and computational complexity.

[0123] Figure 7 is a schematic diagram of the hardware structure of a computing device provided by an embodiment of the present invention, such as Figure 7 As shown, the computing device can be implemented as any one of the above-mentioned image acquisition devices, image output devices, PC devices or edge computing hardware platforms.

[0124] The computing device includes a processor 701, a memory 702, a bus 703, and a computer program stored in the memory 702 and executable on the processor 701. The processor 701 includes one or more processing cores. The memory 702 is connected to the processor 701 via the bus 703. The memory 702 is used to store program instructions. When the processor executes the computer program, all or part of the steps in the above method embodiment provided by the present invention are implemented.

[0125] As an executable solution, the computing device may be a computer unit, which may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer unit may include, but is not limited to, a processor and a memory. Those skilled in the art may understand that the composition structure of the computer unit is merely an example of a computer unit and does not constitute a limitation on the computer unit. The computer unit may include more or fewer components than the above, or a combination of certain components, or different components. For example, the computer unit may also include an input / output device, a network access device, a bus, etc., which is not limited in the embodiments of the present invention.

[0126] As an executable solution, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, and various interfaces and lines are used to connect the various parts of the entire computer unit.

[0127] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer unit by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory. For example, a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMc), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0128] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present invention are implemented.

[0129] If the module / unit integrated in the computer unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-0nly Memory), random access memory (RAM, Random Access Memory) and software development medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0130] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0131] The above embodiments are merely examples of the present invention and do not limit the protection scope of the present invention. All designs that are the same or similar to the present invention fall within the protection scope of the present invention.

Claims

1. A vehicle high beam detection method, characterized in that: The following steps are involved: An improved target detection model is constructed based on the YOLOv8 network. The improved target detection model introduces a feature fusion global attention module CGM into the backbone network of the original YOLOv8 network, and introduces a convolutional feature recombination module CCM into the neck network of the original YOLOv8, as well as a small target detection module CSTC formed by stacking a convolutional feature recombination module, a connection layer, and a feature fusion module; Acquire a plurality of preprocessed vehicle high beam image data sets, and train the improved target detection model using the preprocessed high beam image data sets to obtain a trained target detection model; The high beam image of the vehicle to be tested is obtained as the input of the trained target detection model, and feature maps of different scales are constructed through the backbone network processing of the feature fusion global attention module CGM. The feature maps of different scales are processed through the neck network of the convolutional feature recombination module CCM and the small target detection module CSTC to obtain optimized recombined feature maps, and the optimized recombined feature maps are sent to the YOLO detection layer to obtain the vehicle high beam detection result.

2. The vehicle high beam detection method according to claim 1, characterized in that: The feature fusion global attention module CGM includes a feature fusion module C2f and an attention mechanism GAM; The convolutional feature reconstruction module CCM includes a 1*1 convolution and upsampling operator CARAFE.

3. The vehicle high beam detection method according to claim 2, characterized in that: The construction of feature maps of different scales includes the following steps: The backbone network with feature fusion global attention module CGM is used to extract deep features of the high beam image of the vehicle to be tested, and the deep enhanced features of the image are obtained; The deep enhanced features of the image are sent to the feature pyramid network, and feature maps of different scales are constructed through upsampling and feature fusion.

4. The vehicle high beam detection method according to claim 2, characterized in that: The step of obtaining the optimized recombination characteristic graph comprises the following steps: The feature maps of different scales are sent to the neck network for fusion to obtain the fused feature map; Among them, the convolutional feature recombination module CCM introduced in the neck network reorganizes feature maps of different scales to obtain reorganized feature maps of different scales; the introduced small target detection module optimizes the reorganized feature maps of different scales to obtain the optimized reorganized feature maps.

5. The vehicle high beam detection method according to claim 2, characterized in that: The obtaining of the vehicle high beam detection result comprises the following steps: The optimized recombined feature map is sent to the YOLO detection layer, and the coordinates, category probability and confidence of the bounding box are generated by forward propagation to obtain the vehicle high beam detection result; wherein, the final detection result is screened out according to the confidence threshold; non-maximum suppression (NMS) is applied to merge overlapping bounding boxes, and the bounding box with the highest confidence is retained.

6. The vehicle high beam detection method according to claim 1, characterized in that: The method of acquiring a plurality of vehicle high beam images and then preprocessing them to obtain a preprocessed high beam image dataset includes the following steps: Acquire multiple vehicle high beam images, perform enhancement processing, and obtain multiple enhanced vehicle high beam images, wherein the enhancement processing includes: one or more of random rotation, scaling, cropping, brightness adjustment, and adding noise; An image annotation tool is used to annotate the enhanced vehicle high beam image, and the vehicle high beam image is converted into a YOLO data set format including target position coordinates and target categories to obtain the vehicle high beam data set.

7. A vehicle high beam detection system based on edge computing, characterized in that: include: A data acquisition module acquires multiple vehicle high beam images and then performs preprocessing to obtain a preprocessed high beam image data set; A detection model construction module is provided, which constructs a target detection model based on a YOLOv8 network, introduces a feature fusion global attention module CGM into the backbone network of the YOLOv8 network, introduces a convolutional feature reorganization module CCM into the neck network of the YOLOv8, and a small target detection module CSTC formed by stacking a convolutional feature reorganization module, a connection layer, and a feature fusion module, to obtain an improved target detection model; and trains the improved target detection model using a preprocessed high beam image data set to obtain a trained target detection model; The detection result acquisition module obtains the high beam image of the vehicle to be tested as the input of the trained target detection model, and constructs feature maps of different scales through the backbone network processing of the feature fusion global attention module CGM; the feature maps of different scales are processed by the neck network of the convolution feature recombination module CCM and the small target detection module CSTC to obtain the optimized recombined feature map, and the optimized recombined feature map is sent to the YOLO detection layer to obtain the vehicle high beam detection result.

8. The lightweight vehicle high beam detection system based on edge computing as claimed in claim 7, characterized in that: The data acquisition module includes an image acquisition device, an edge computing hardware platform and an image output device; The image acquisition device is used to acquire a real-time video stream containing a vehicle high-beam image, and transmit the real-time video stream to an edge computing hardware platform; The edge computing hardware platform performs vehicle high beam detection on the nighttime road vehicle image contained in the real-time video stream, and outputs a vehicle high beam detection result, wherein the vehicle high beam detection result includes: location information and category probability information of the vehicle high beam; The image output device is used to superimpose the vehicle high beam detection result onto the real-time video stream and output the real-time detection video stream, wherein the real-time detection video stream displays the position and category probability of the vehicle high beam.

9. The vehicle high beam detection system based on edge computing as claimed in claim 7, characterized in that: Also includes: The detection result acquisition module also includes: a format conversion unit; The format conversion unit performs format conversion and quantization on the weight parameters of the trained target detection model to obtain a quantization model that matches the format of the edge computing hardware platform; obtains a high beam image of the vehicle to be tested, inputs the high beam image of the vehicle to be tested into the quantization model, and obtains a vehicle high beam detection result.

10. The lightweight vehicle high beam detection system based on edge computing as claimed in claim 7, characterized in that: The obtaining of a quantization model matching the format of the edge computing hardware platform includes: Converting the weight parameters of the training network model into a weight file in an open neural network exchange format; Call the configuration interface to obtain the preprocessing parameters of the neural network software development kit that matches the edge computing hardware platform, create an RKNN object to initialize the neural network software development kit; import the weight file into the neural network model development tool, call the creation interface, and create an initial RKNN model based on the RKNN object; Calling the application program interface of the neural network model development tool, evaluating the memory usage of the initial RKNN model, and performing simulation reasoning until the initial RKNN model achieves the preset expected effect; The initial RKNN model that achieves the expected effect is quantized through the 8-bit asymmetric quantization function of the neural network software development kit to obtain a quantized model in the RNKK format that matches the edge computing hardware platform format.

Images