Road small target detection algorithm based on Real-ESRGAN and improved YOLOv5

Abstract

The invention discloses a road small target detection algorithm based on Real-ESRGAN and improved YOLOv5, and the algorithm comprises the steps: S1, preparing a data set, collecting data, and dividing the data into a training set, a verification set and a test set; s2, building a YOLOV5 algorithm model, inputting data and training the data to obtain a YOLOV5 algorithm model weight; the trained YOLOV5 algorithm model weight is used to detect whether a small road target appears, and if yes, a detected image frame is intercepted and stored; s3, performing super-resolution reconstruction on the detected image frame, and outputting a detection frame image after the super-resolution reconstruction; and S4, identifying the output detection frame image after super-resolution reconstruction, and updating a detection result after classification. By adopting the Real-ESRGAN and the improved YOLOv5 detection algorithm, the detection confidence and accuracy of the small road target are improved, and the false detection and missing detection conditions are reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of artificial intelligence, and in particular relates to a road small target detection algorithm based on Real-ESRGAN and improved YOLOv5. Background Art

[0002] Existing technologies have the main problems of low resolution and small pixel ratio when dealing with small road targets, resulting in high false detection rate and missed detection rate during the detection process. Target detectors trained on data sets mainly composed of larger objects may be difficult to generalize to the detection of small objects due to scale differences. In addition, road scenes are complex and changeable, including different lighting conditions, weather conditions, traffic conditions, etc., which poses higher challenges to small target detection algorithms.

[0003] At present, the road small target detection algorithm based on the traditional computer vision method is adopted. With the rapid development of computer vision technology, more and more researchers are focusing on the research of behavior recognition. At present, small target detection is mostly based on the mainstream target detection algorithm, and improvements are made in feature extraction, feature fusion, attention mechanism and detection head to improve the small target detection ability. For example, some researchers designed the Vehicle-YOLO model, which mainly changes the residual network on the network based on YOLOV3 to enhance the small target feature extraction ability; some researchers designed the Refined-YOLO model, which is based on the YOLOV4 network and improves the small target detection from three aspects: anchor box, post-processing algorithm and attention. Cui improved MobileNet and then combined it with the YOLO V3 target detection algorithm. The improved MobileNet was used as the feature extraction network to replace the Darknet53 in YOLO V3. The detection speed is faster than the original YOLO V3 algorithm, but the detection accuracy is not much improved. Some researchers proposed adding a coordinate attention (CA) module at the connection between Backbone and Neck, which can aggregate different position information of the image into the channel, so that the attention of the detected object can be improved. However, the impact of high-resolution images on the accuracy of lightweight networks and other issues need to be solved. Some researchers added bilinear interpolation upsampling to the feature fusion part, and inserted 5 convolutional block attention modules (CBAM) and small target detection layers to enhance the semantics and position information of small targets. However, it only achieved the detection of one small target category in the dataset. Some researchers proposed using the Ghost-Hards-wish Conv module to simplify the convolution operation, and introduced coordinate attention combined with weighted bidirectional feature pyramid (Bidirectional Feature Pyramid Network, BiFPN) to enhance the expression ability of small target features.

[0004] The above small target detection methods have some shortcomings: First, the small target detection algorithm based on traditional computer vision methods has high requirements for the environment, and requires a clear camera field of view and good lighting conditions to accurately capture the image information. In complex or harsh environments, the performance of the algorithm may decline. Therefore, high-resolution and clear images play an important role, but existing lightweight networks often face the problem of reduced accuracy when processing such images; although the small target information is enhanced through bilinear interpolation upsampling and convolutional attention mechanism, this method only realizes the detection of a small target category in the data set, limiting its application in multi-category small target detection.

[0005] Chinese patent document CN116503820A discloses a road vehicle type detection method and detection equipment, which collects vehicle data on the road through a camera, annotates the vehicle data, and constructs a vehicle data set; sends the vehicle data set to the neural network model, executes the training process, and parses the vehicle data in the vehicle data set; then calculates the loss value between the prior frame and the real annotated frame, and uses a validation set isolated from the training set to evaluate the current model; after multiple rounds of training, when the loss value tends to a preset threshold, the training is terminated; after obtaining the vehicle data prediction result, if the category is a tricycle or a two-wheeled vehicle, helmet detection and overload detection are performed. This technical solution is based on the improved target detection algorithm of YOLOv5, finds the feature relationship between pixels in a larger range in the feature layer, makes up for the limitation that the convolutional neural network can only learn small-range features, and improves the detection accuracy.

[0006] In contrast, the road small target detection method based on computer vision deep learning has better flexibility and generalization ability. Therefore, it is necessary to provide a road small target detection algorithm based on Real-ESRGAN and improved YOLOv5. The deep learning target detection algorithm can automatically learn and extract features, adapt to different environmental conditions, and can be trained and optimized end-to-end from a large amount of data, which makes the road small target detection algorithm based on deep learning have higher accuracy and robustness. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a road small target detection algorithm based on Real-ESRGAN and improved YOLOv5, which can automatically learn and extract features using a deep learning target detection algorithm, adapt to different environmental conditions, and perform end-to-end training and optimization from a large amount of data, thereby making the road small target detection algorithm based on deep learning have higher accuracy and robustness.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: the road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 specifically includes the following steps:

[0009] S1: Prepare the data set, collect data, and divide it into training set, validation set, and test set;

[0010] S2: Build the YOLOV5 algorithm model, input data and train it to obtain the YOLOV5 algorithm model weights; use the trained YOLOV5 algorithm model weights to detect whether there are small road targets, and if so, capture the detected image frame and save it;

[0011] S3: Perform super-resolution reconstruction on the detected image frame and output the super-resolution reconstructed detection frame image;

[0012] S4: Use the Long-CLIP zero-shot classification method to identify the output super-resolution reconstructed detection box image, and update the detection result after classification.

[0013] The present invention improves YOLOv5. In the detection of small targets, it maintains the computational complexity of efficient detection speed and improves the excellent detection accuracy, and also performs a lightweight design of the model, so as to detect more accurately. The method of combining YOLO v5 with Real ESRGAN is adopted to pass the target detection frame detected by the target, and super-resolution reconstruction is performed. The details of the image are restored by Real ESRGAN to improve the resolution, which greatly improves the clarity and recognizability of small targets in the image, and provides more powerful support for subsequent target recognition and classification. The improved YOLOv5 detection algorithm adopted by the algorithm of the present invention not only improves the confidence and accuracy of road small target detection, but also reduces the false detection and missed detection. On the basis of the traditional model, a strategy suitable for road small target detection is considered. Compared with the original single detector strategy, the detection accuracy of small targets is significantly improved, and even under challenging conditions such as illumination changes and occlusion, a high detection performance can be maintained.

[0014] Preferably, in step S1, image data is obtained by inputting an image sequence or video, and the image data is segmented using an image segmentation strategy, that is, the image is segmented into small blocks, and then divided into a training set, a validation set and a test set, and then input into a detector to enhance the image data using Mosaic.

[0015] Preferably, in step S3, the deep learning Real-ESRGAN super-resolution algorithm is used to perform super-resolution reconstruction, and the specific steps are:

[0016] S31: Simulate the actual degradation process, including smearing, downsampling, adding noise and JPEG compression, to generate low-resolution blurred images;

[0017] S32: Upscaling low-resolution images to high-definition through the generator;

[0018] S33: The discriminator determines whether the generated image is similar to the real high-definition image until the best match is achieved.

[0019] Preferably, the deep learning Real-ESRGAN super-resolution algorithm loss function in S3 adopts a combination of L1 loss, perceptual loss and GAN loss. By optimizing these loss functions, it is ensured that the generated image is as consistent as possible with the real image visually, avoiding the loss of image details or incorrect reconstruction.

[0020] Preferably, the generative model loss function expression in step S3 is:

[0021] L G =L percep +λL G +ηL1(1);

[0022] Among them, λ and η are hyperparameters, λ defaults to 0.1, and η defaults to 1; they are used to balance the contribution between different loss functions, where L percep is the perceptual loss, which measures the quality of the generated image by comparing the difference between the generated image and the real image. G is GANloss, which measures the adversarial process between the generator and the discriminator, L1 is L1 loss, which directly calculates the average of the absolute difference between the pixel values ​​of the generated image and the real image, and L percep The expression is:

[0023] L percep =∥φ(x i )-φ(y i )∥1(2);

[0024] Among them, φ is the VGG19 model, x i and i It represents the output of the generative model and the real data ground truth. The VGG19 model is used to obtain their feature representation in the network, minimizing the distance between the input image and the target image in the feature space. G The expression is:

[0025]

[0026] in, is the real label, which is a matrix of all 1s. D is the discriminant model. The generated result and the real label are used to calculate the binary cross entropy loss BCELoss. The expression of L1 is:

[0027] L1=mean∥G(x i )-y i ∥1(4);

[0028] Among them, G(x i ) represents the output of the generative model, y i represents the real data ground truth, mean represents the average value, and L1loss is calculated to measure the average absolute error between the model-generated results and the real results.

[0029] Through the above steps, Real-ESRGAN can achieve super-resolution reconstruction of small target images. In small target detection, the reconstruction results can be combined with subsequent classification results.

[0030] The small target detection training strategy using image segmentation is used to divide the high-resolution image into small blocks for training, and then input the large image for detection, which effectively improves the detection accuracy of small targets and reduces the missed detection rate; the original YOLOv5 network neck structure is replaced with SPPFCSCP, which can not only reduce the amount of calculation in the model, but also improve the accuracy of target detection; the GD (gather-and-distribute aggregation-distribution) mechanism method is used to detect small targets on the road, and the ghost shuffled convolution (GSConv) module that combines standard convolution and depthwise separable convolution and performs channel cleaning replaces the convolution module in GD (gather-and-distribution aggregation-distribution) to improve the network speed, lightweight, reduce the number of parameters and computational complexity; the NWD loss function is introduced for small target detection. This method uses the Gaussian distribution of the bounding box to judge the similarity between the predicted box and the real box; and the decoupled head DecoupledHead is introduced. The decoupled head processes the position and category information separately, reducing resource requirements and improving generalization ability.

[0031] Preferably, the data set in step S1 includes small road targets in different scenarios to be detected.

[0032] Preferably, the specific steps of using the Long-CLIP zero-sample classification method to identify and classify the output super-resolution reconstructed detection frame image in step S4 are:

[0033] S41 Data preprocessing: For each road small target category, the category label information is combined with the prompt word to be converted into text information, and then the cropped detection image is preprocessed to meet the input requirements of Long-CLIP;

[0034] S42 classifies the image in the detection frame: the multimodal classification model Long-CLIP is used to classify and identify the small road target image after cropping super-resolution. Long-CLIP will identify the category of the small road target based on the similarity between the image features and the text features of each category, and make category predictions;

[0035] S43 result update: According to the results output by the classifier, that is, the classification results predicted by the multimodal classification model Long-CLIP, if the category confidence predicted by a certain image is the highest among the candidate categories, but the absolute value is still at a low level, then this result can be regarded as an effective filtering mechanism for potential false detections (especially false detections of empty frames generated in the background area). It can eliminate many falsely detected empty frames generated in the background and update the detection results. After super-resolution reconstruction, a high-resolution target image is obtained, and then another corresponding stage is entered. At this time, the model will make a deeper judgment and analysis of the target based on the reconstructed image, and use the Long-CLIP technology to recognize the small target image combined with the category text; the feature tensors of the target image and the category text are input into the model, and the target detection results obtained by YOLO v5 are matched according to the maximum similarity to obtain the recognition result. According to different situations, it can be divided into 3 types of processing. If the category of small target recognition is consistent, it remains unchanged; if the image mistakenly detects the background as the detection category, after recognition, each category may not be high. At this time, the detection box can be deleted to reduce false detection; if the image is mistakenly detected as other categories, the category recognition of the detection box is updated to eliminate false detection.

[0036] Preferably, the preprocessing in step S41 includes image scaling and normalization operations.

[0037] Preferably, the multimodal classification model Long-CLIP in step S42 uses an unlabeled test data set to evaluate the multimodal classification model Long-CLIP to verify its generalization ability in different road scenarios.

[0038] Preferably, the evaluation indicators of step S42 include precision, recall, F1 score and detection speed. Precision refers to the ratio of the number of samples correctly predicted by the model to the number of samples determined as positive samples. Recall refers to the ratio of the number of correctly classified positive samples to the number of true positive samples. F1 score is the harmonic mean of precision and recall, which takes into account both the accuracy and recall of the classification model. The specific formula is as follows:

[0039]

[0040] Among them, TP represents the number of samples correctly predicted as positive samples, FP represents the number of negative samples in the sample that are incorrectly predicted as positive samples, and FN represents the number of positive samples in the sample that are incorrectly predicted as negative samples. When calculating this feature, the similarity calculation is not performed one by one, but can be performed in batches. Therefore, Long-CLIP has fast inference speed and high efficiency.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) High accuracy: YOLOv5 has high accuracy as a target detector and can quickly and accurately detect road targets in images or videos. By introducing the small target detection training strategy in the image segmentation strategy, the present invention can more effectively process small targets in road scenes, reducing false detections and missed detections, which is of great significance to road safety and environment;

[0043] (2) Super-resolution reconstruction: Combined with Real-ESRGAN to reconstruct small objects, the image clarity is significantly improved, providing higher quality input for subsequent analysis;

[0044] (3) Two-stage classification model: The two-stage classification model Long-CLIP is introduced to perform zero-shot classification. The prediction results are updated according to the classification results. For the blurred and targetless images that are misdetected, whether to retain the detection results is determined based on the category probability. This improves the robustness and accuracy of the overall detection system and reduces the false detection rate of small target detection.

[0045] (4) Multi-task learning: YOLOv5, Real-ESRGAN, and the two-stage classification model are integrated to form a multi-task learning framework. The modules collaborate with each other and share features, which improves the performance of the entire system.

[0046] (5) Universality: The road object detection method based on YOLOv5 and Real-ESRGAN can be applied to various types of cameras and scenes, and has strong universality and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the overall algorithm flow in the road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 of the present invention;

[0048] Figure 2 The flowchart of the Real-ESRGAN algorithm in the road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 of the present invention;

[0049] Figure 3 It is a schematic diagram of Real-ESRGAN reconstruction and recognition in the road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 of the present invention;

[0050] Figure 4 This is a network structure diagram of an improved YOLOv5 algorithm in a road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 of the present invention;

[0051] Figure 5This is the Long-CLIP network structure in the road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 of the present invention. DETAILED DESCRIPTION

[0052] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0053] Embodiment: The known technical contents underlying the present invention include the YOLOv5 target detection algorithm and the Real-ESRGAN image resolution enhancement model.

[0054] (1) YOLOv5 target detection algorithm:

[0055] YOLOv5 technical features: YOLOv5 is a single-stage target detection algorithm. This algorithm adds some new improvement ideas based on YOLOv4, which greatly improves its speed and accuracy. Compared with the YOLOv4 algorithm, the YOLOv5 target detection algorithm has the following improvements:

[0056] Input side: During the model training phase, some improvement ideas were proposed, mainly including Mosaic data enhancement and adaptive anchor box calculation.

[0057] Mosaic data enhancement was first introduced in YOLOv4. It randomly selects four different training images, and generates new training samples by randomly scaling, cropping, and arranging the four images to form a "mosaic" image. During the stitching process, the size and position of each image are randomly adjusted and flipped. This allows the model to be exposed to more background and target samples during training, increasing the richness and diversity of the data set. Mosaic data enhancement can improve the generalization ability and robustness of the model and reduce the overfitting problem.

[0058] Adaptive anchor box calculation is a method that dynamically adjusts the anchor box according to the distribution and size of the target in the image. Traditional target detection algorithms usually use fixed anchor boxes to predict the bounding box of the target. Adaptive anchor box calculation dynamically generates anchor boxes based on the distribution of targets in the training set, making them more adaptable to the size and aspect ratio of the target. By dynamically adjusting the anchor points during the training process, the accuracy of the predicted box is improved, reducing the work of manually adjusting the anchor points. Adaptive anchor box calculation is usually combined with clustering algorithms or statistical analysis methods to determine the optimal anchor box size and aspect ratio.

[0059] These techniques are widely used in object detection algorithms to increase data diversity, improve model adaptability and accuracy. Their use can make the model better adapt to various scenarios and targets, and achieve better performance in practical applications.

[0060] The improved YOLOv5 target detection algorithm has the following improvements compared to the YOLOv5 algorithm:

[0061] Neck network: The main improvement of Neck network is to upgrade SPPF to SPPFCSPC;

[0062] The target detection network often inserts some layers between the BackBone and the final Head output layer. Compared with the SPP (Spatial Pyramid Pooling) used by YOLOv4, Yolov5 uses an upgraded SPPF structure after the update. This structure is often used in target detection tasks to extract and fuse feature information of different scales to better detect multi-scale targets. The main idea of ​​SPPF is to achieve a similar effect to SPP in a simpler way by stacking the Max Pooling layer and the Convolutional Layer multiple times, while possibly reducing the amount of calculation and the amount of parameters.

[0063] SPPFCSPC combines the advantages of SPPF (Spatial Pyramid Pooling Faster) and CSPNet, and uses grouped convolution operations to process feature maps. It divides the input feature maps into several groups and performs independent convolution operations in each group. SPPFCSPC reduces the computational load and the number of parameters while maintaining high performance, with higher parameter efficiency and lower computational load.

[0064] Head output layer: The output layer mainly improves the GD (gather-and-distribute) mechanism and the loss function NWD during training.

[0065] In the Head output layer of YOLOv5, a set of predefined anchor boxes are used to predict the bounding box of the target. Each anchor box is usually associated with multiple grid cells, and the position and size of the target are predicted by regression. The size and aspect ratio of these anchor boxes are usually designed and adjusted according to the distribution of targets in the training set to accommodate targets of different scales and shapes.

[0066] Adaptive anchor box calculation is used to dynamically adjust the anchor points during the training process, which improves the accuracy of the prediction box and reduces the work of manually adjusting the anchor points.

[0067] GD (gather-and-distribute) is a mechanism proposed by Gold-YOLO to avoid the indirect fusion of information at different levels through continuous iterations. The GD mechanism is implemented through three modules: Information Alignment Module (FAM), Information Fusion Module (IFM) and Information Injection Module (Inject). Without significantly increasing the latency, the GD mechanism significantly enhances the information fusion capability of the Neck part and improves the model's ability to detect objects of different sizes. In order to balance the computational cost and model accuracy, GSConv (Group Shuffle Convolution) is used to replace the traditional convolution operation. GSConv is a core technology introduced to improve the performance of lightweight convolution models. It combines the advantages of standard convolution (SC) and depthwise separable convolution (DSC) by combining the shuffling mechanism.

[0068] During training, YOLOv5 improved the calculation method of the loss function and adopted the NWD (Normalized Wasserstein Distance) loss function. NWD is a loss function designed specifically for small target detection, and it shows significant advantages in dealing with small target detection tasks. IOU is calculated based on the intersection over union ratio, which is insensitive to the size of the target, which means that when dealing with small targets, even if there is a slight position deviation between the predicted box and the true box, it may cause a significant drop in the IoU value, thereby affecting the optimization of the model. By introducing the concept of Wasserstein distance, the NWD loss function overcomes the limitations of the traditional IoU (Intersection over Union) loss function in dealing with small targets, especially the sensitivity to position deviation and scale invariance.

[0069] Real-ESRGAN image resolution enhancement algorithm: Real-ESRGAN is an improved model based on ESRGAN, designed to handle complex and diverse degradation situations in real-world images. It can generate high-quality and detailed high-resolution images by combining multi-scale feature extraction with generative adversarial training. Its core design adopts the generative adversarial network (GAN) framework, which consists of two key components: the generator (G) and the discriminator (D). The task of the generator is to receive low-resolution inputs and perform feature extraction, information gain, and high-frequency detail reconstruction on them through a multi-layer convolutional neural network structure, and finally output a high-resolution image; the discriminator is responsible for distinguishing between real high-resolution images and high-resolution images generated by the generator. Through generative adversarial training, the generator is continuously improved to generate more realistic images; since the goal of Real-ESRGAN is to solve a much larger degradation space than ESRGAN, the original discriminator design in ESRGAN is no longer applicable. The VGG-style discriminator in ESRGAN is improved to a U-Net design with skip connections. At the same time, the U-Net structure and complex degradation also increase the instability of training. We adopt spectral normalization regularization to stabilize the training. In addition, we observe that spectral normalization is also helpful in alleviating over-sharpening and artifacts caused by GAN training.

[0070] A high-order data degradation process is given. That is, several typical degradation processes are spliced ​​together to model (including the sinc filter), so as to obtain a low-quality image that is closer to reality. Finally, the author adopts a second-order degradation process to achieve a good balance between simplicity and effectiveness.

[0071] Real-ESRGAN main features:

[0072] (1) Multi-scale feature extraction: Real-ESRGAN uses a multi-scale feature extraction module to better capture information of different scales in an image, thereby improving the detail expression of the image.

[0073] (2) Generative Adversarial Training: Generative Adversarial Networks (GANs) are used for training to make the generated high-resolution images more visually realistic and with richer details.

[0074] (3) Lightweight design: The model structure is optimized so that Real-ESRGAN can run efficiently even on devices with limited computing resources.

[0075] (4) Adaptive degradation processing: Real-ESRGAN can adaptively process different types of degradation to improve image quality.

[0076] like Figure 1 As shown in the figure, the road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 specifically includes the following steps:

[0077] S1: Prepare the data set, collect data, and divide it into training set, validation set, and test set. The data set includes small road targets in different scenarios to be detected;

[0078] In step S1, image data is obtained by inputting an image sequence or video, and the image data is segmented, that is, the image is segmented into small blocks, and then divided into a training set, a verification set and a test set, and then the image data is enhanced using Mosaic; the data set in step S1 includes small road targets in different scenes to be detected;

[0079] S2: Build the YOLOV5 algorithm model, input data and train it to obtain the YOLOV5 algorithm model weights; use the trained YOLOV5 algorithm model weights to detect whether there are small road targets, and if so, capture the detected image frame and save it;

[0080] S3: Perform super-resolution reconstruction on the detected image frame and output the super-resolution reconstructed detection frame image; Figure 4 As shown, Figure 4 SPPFCSCP: dual-channel spatial pyramid pooling; Decoupled-Detect: decoupled detection head; SimFusion_4in: adaptive pooling fusion; IFM: information fusion module; GSConv: ghost shuffle convolution; InjectionMultiSum_Auto_pool: information injection automatic pooling; PyramidPoolAgg: pyramid pooling fusion; AdvPoolFusion: adaptive advanced pooling fusion; TopBasicLayer: apply multiple top blocks and map the results to the specified number of output channels through the convolution layer;

[0081] like Figure 2 As shown, in step S3, the deep learning Real-ESRGAN super-resolution algorithm is used to perform super-resolution reconstruction, and the specific steps are:

[0082] S31: Simulate the actual degradation process, including smearing, downsampling, adding noise and JPEG compression, to generate low-resolution blurred images;

[0083] S32: Upscaling low-resolution images to high-definition through the generator;

[0084] S33: judging whether the generated image is similar to the real high-definition image through the discriminator until the best match is achieved;

[0085] The loss function of the deep learning Real-ESRGAN super-resolution algorithm in S3 uses a combination of L1 loss, perceptual loss, and GAN loss. By optimizing these loss functions, it is ensured that the generated image is as visually consistent as possible with the real image, avoiding the loss of image details or incorrect reconstruction;

[0086] The generative model loss function expression in step S3 is:

[0087] L G =L percep +λL G +ηL1(1);

[0088] Among them, λ and η are hyperparameters, λ defaults to 0.1, and η defaults to 1; they are used to balance the contribution between different loss functions, where L percep is the perceptual loss, which measures the quality of the generated image by comparing the difference between the generated image and the real image. G is GANloss, which measures the adversarial process between the generator and the discriminator, L1 is L1 loss, which directly calculates the average of the absolute difference between the pixel values ​​of the generated image and the real image, and L percep The expression is:

[0089] L percep =∥φ(x i )-φ(y i )∥1(2);

[0090] Among them, φ is the VGG19 model, x i and i It represents the output of the generative model and the real data ground truth. The VGG19 model is used to obtain their feature representation in the network, minimizing the distance between the input image and the target image in the feature space. G The expression is:

[0091]

[0092] in, is the real label, which is a matrix of all 1s. D is the discriminant model. The generated result and the real label are used to calculate the binary cross entropy loss BCELoss. The expression of L1 is:

[0093] L1=mean∥G(x i )-y i ∥1(4);

[0094] Among them, G(x i ) represents the output of the generative model, y irepresents the real data ground truth, mean represents the average value, and L1loss is calculated to measure the average absolute error between the model-generated results and the real results.

[0095] Through the above steps, Real-ESRGAN can achieve super-resolution reconstruction of small target images. In small target detection, the reconstruction results can be combined with subsequent classification results for detection;

[0096] S4: Use the Long-CLIP zero-shot classification method to identify the output super-resolution reconstructed detection frame image, and update the detection result after classification;

[0097] like Figure 5 As shown, Figure 5 Aphoto of in the text is a prompt word, and the text input is composed of the category label [Transparent bag, paper, cardboard box...]; the specific steps of using the Long-CLIP zero-sample classification method to identify and classify the output super-resolution reconstructed detection frame image in step S4 are:

[0098] S41 data preprocessing: for each road small target category, the category label information is combined with the prompt word to be converted into text information, and then the cropped detection image is preprocessed to meet the input requirements of Long-CLIP; the preprocessing in step S41 includes image scaling and normalization operations;

[0099] S42 classifies the image in the detection frame: the multimodal classification model Long-CLIP is used to classify and identify the road small target image after cropping super resolution. Long-CLIP will identify the category of the road small target based on the similarity between the image features and the text features of each category, and make category predictions; the detection results are as follows Figure 3 shown; Figure 3 Aphoto of is the prompt word, and the category label [Transparent bag, paper, cardboard box...] constitutes the text input;

[0100] The multimodal classification model Long-CLIP of step S42 is evaluated using an unlabeled test data set to verify its generalization ability in different road scenarios; the evaluation indicators of step S42 include accuracy, recall, F1 score and detection speed; precision refers to the ratio of the number of samples correctly predicted by the model to the number of samples determined to be positive samples, recall refers to the ratio of the number of correctly classified positive samples to the number of true positive samples, and F1 score is the harmonic mean of precision and recall, which takes into account both the accuracy and recall of the classification model; the specific formula is as follows:

[0101]

[0102] Among them, TP represents the number of samples correctly predicted as positive samples, FP represents the number of negative samples in the sample that are incorrectly predicted as positive samples, and FN represents the number of positive samples in the sample that are incorrectly predicted as negative samples. When calculating this feature, the similarity calculation is not performed one by one, but can be performed in batches. Therefore, Long-CLIP has fast inference speed and high efficiency.

[0103] S43 result update: According to the results output by the classifier, that is, the classification results predicted by the multimodal classification model Long-CLIP, if the category confidence predicted by a certain image is the highest among the candidate categories, but the absolute value is still at a low level, then this result can be regarded as an effective filtering mechanism for potential false detections (especially false detections of empty frames generated in the background area). It can eliminate many falsely detected empty frames generated in the background and update the detection results. After super-resolution reconstruction, a high-resolution target image is obtained, and then another corresponding stage is entered. At this time, the model will make a deeper judgment and analysis of the target based on the reconstructed image, and use the Long-CLIP technology to recognize the small target image combined with the category text; the feature tensors of the target image and the category text are input into the model, and the target detection results obtained by YOLO v5 are matched according to the maximum similarity to obtain the recognition result. According to different situations, it can be divided into 3 types of processing. If the category of small target recognition is consistent, it remains unchanged; if the image mistakenly detects the background as the detection category, after recognition, each category may not be high. At this time, the detection box can be deleted to reduce false detection; if the image is mistakenly detected as other categories, the category recognition of the detection box is updated to eliminate false detection.

[0104] The above technical solution introduces an image segmentation training strategy. By dividing the image into small blocks for training and then detecting it on a large image, the problem of missed detection of small road targets due to their small size and blur is effectively solved, and the detection accuracy of small targets is significantly improved and the missed detection rate is reduced. The original network neck structure of YOLOv5 is replaced with SPPFCSCP, and the GD aggregation distribution mechanism is introduced for small target detection. The network speed is improved by replacing the traditional convolution module in GD with GSConv, and the model is lightweight, the number of parameters and the computational complexity are reduced. The NWD loss function is introduced to further improve the detection accuracy by optimizing for small targets. The Decoupled Head is introduced to process the position and category information separately, which reduces the resource requirements and improves the generalization ability of the model. After detection, the image output by the model is saved and super-resolution reconstruction is performed using Real-ESRGAN. Subsequently, zero-sample classification is performed using the two-stage classification model Long-CLIP, and the prediction result is updated according to the classification result. For the falsely detected blurry and targetless images, the detection result is decided whether to be retained according to the category probability. For the misclassified images, the category of the detection box is updated, thereby significantly reducing the false detection rate.

[0105] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A road small target detection algorithm based on Real-ESRGAN and improved YOLOv5, characterized in that: The specific steps include: S1: Prepare the data set, collect data, and divide it into training set, validation set, and test set; S2: Build the YOLOV5 algorithm model, input data and train it to obtain the YOLOV5 algorithm model weights; use the trained YOLOV5 algorithm model weights to detect whether there are small road targets, and if so, capture the detected image frame and save it; S3: Perform super-resolution reconstruction on the detected image frame and output the super-resolution reconstructed detection frame image; S4: Identify the output super-resolution reconstructed detection frame image, classify it, and then update the detection result.

2. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 1, characterized in that: In the step S1, image data is obtained by inputting an image sequence or a video, and the image data is cropped. The image data is then enhanced using an image segmentation strategy, and then divided into a training set, a validation set, and a test set.

3. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 1, characterized in that: In step S3, the deep learning Real-ESRGAN super-resolution algorithm is used to perform super-resolution reconstruction, and the specific steps are as follows: S31: Simulate the actual degradation process, including smearing, downsampling, adding noise and JPEG compression, to generate low-resolution blurred images; S32: Upscaling low-resolution images to high-definition through the generator; S33: The discriminator determines whether the generated image is similar to the real high-definition image until the best match is achieved.

4. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 3, characterized in that: The loss function of the deep learning Real-ESRGAN super-resolution algorithm in S3 adopts a combination of L1 loss, perceptual loss and GAN loss.

5. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 4, characterized in that: The generative model loss function expression in step S3 is: L G =L percep +λL G +ηL1(1); Among them, λ and η are hyperparameters, λ defaults to 0.1, and η defaults to 1; they are used to balance the contribution between different loss functions, where L percep is the perceptual loss, which measures the quality of the generated image by comparing the difference between the generated image and the real image. G is GANloss, which measures the adversarial process between the generator and the discriminator, L1 is L1 loss, which directly calculates the average of the absolute difference between the pixel values ​​of the generated image and the real image, and L percep The expression is: L percep =∥φ(x i )-φ(y i )∥1(2); Among them, φ is the VGG19 model, x i and i Represents the input and ground truth data. The VGG19 model is used to obtain their feature representation in the network, minimizing the distance between the input image and the target image in the feature space. G The expression is: in, is the real label, which is a matrix of all 1s. D is the discriminant model. The generated result and the real label are used to calculate the binary cross entropy loss BCELoss. The expression of L1 is: L1=mean∥G(x i )-and i ∥1(4); Among them, G(x i ) represents the output of the generative model, y i represents the real data ground truth, mean represents the average value, and L1loss is calculated to measure the average absolute error between the model-generated results and the real results.

6. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 4, characterized in that: The data set in step S1 includes small road objects to be detected in different scenes.

7. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 6, characterized in that: The specific steps of using the Long-CLIP zero-sample classification method in step S4 to identify and classify the output super-resolution reconstructed detection frame image are as follows: S41 Data preprocessing: For each road small target category, the category label information is combined with the prompt word to be converted into text information, and then the cropped detection image is preprocessed to meet the input requirements of Long-CLIP; S42 classifies the image in the detection frame: the multimodal classification model Long-CLIP is used to extract image features of the small road target image after cropping super-resolution. The classifier will judge the input data and identify the category of the small road target based on the similarity between the image features and the text features of each category; S43 result update: Update the detection result according to the output of the classifier, that is, the classification result predicted by the multimodal classification model Long-CLIP.

8. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 7, characterized in that: The preprocessing in step S41 includes image scaling and normalization operations.

9. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 7, characterized in that: The multimodal classification model Long-CLIP in step S42 is evaluated using an unlabeled test data set to verify its generalization ability in different road scenarios.

10. The road small target detection algorithm based on Real-ESRGAN and improved YOLOv5 according to claim 9, characterized in that: The evaluation indicators of step S42 include precision, recall, F1 score and detection speed. Precision refers to the ratio of the number of samples correctly predicted by the model to the number of samples determined as positive samples. Recall refers to the ratio of the number of correctly classified positive samples to the number of true positive samples. F1 score is the harmonic mean of precision and recall, which takes into account both the accuracy and recall of the classification model. The specific formula is as follows: Among them, TP represents the number of samples correctly predicted as positive samples, FP represents the number of negative samples in the sample that are incorrectly predicted as positive samples, and FN represents the number of positive samples in the sample that are incorrectly predicted as negative samples.

Images