Vehicle and pedestrian detection method on airport runway based on improved YOLOv7-tiny

Through the improved YOLOv7-tiny model, combined with attention mechanism and data enhancement technology, the accuracy and real-time problems of vehicle and pedestrian detection in the airport environment are solved, the detection accuracy and speed are improved, and the safety and efficiency of the airport are ensured.

CN119091395APending Publication Date: 2024-12-06THE 28TH RES INST OF CHINA ELECTRONICS TECH GROUP CORP
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410978687.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing vehicle and pedestrian detection technologies are difficult to accurately identify targets in complex airport environments, and there is real-time detection lag, affecting safety and efficiency.

Method used

Based on the improved YOLOv7-tiny model, by adding 1×1 convolutional layer and residual connection, the coordinate attention mechanism and global attention mechanism are introduced, and the Focal CIOU loss function is used, combined with Mosaic and Mixup data enhancement methods are constructed to build an efficient Vehicle pedestrian detection model.

Benefits of technology

It improves the accuracy and speed of vehicle pedestrian detection, ensures the safety management of airport runways and the safety and efficiency of aircraft take-off and landing processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091395A_ABST
    Figure CN119091395A_ABST
Patent Text Reader

Abstract

The invention discloses a vehicle and pedestrian detection method on an airport runway based on improved YOLOv7-tiny. The method comprises the following steps: constructing a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny; designing a loss function for training the model; establishing a vehicle and pedestrian detection data set and dividing a training set and a test set; training a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny by using the training set; inputting the test set into a trained vehicle and pedestrian detection model, testing the performance of the model, and evaluating the detection precision and speed of the model; and performing vehicle and pedestrian detection in the airport runway by using the trained vehicle and pedestrian detection model. According to the invention, a vehicle and pedestrian detection model on the airport runway is realized, the accuracy of vehicle and pedestrian detection is improved, and the method can be applied to environments such as automatic driving, traffic safety, airports and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a vehicle and pedestrian detection method, in particular to a vehicle and pedestrian detection method on an airport runway based on an improved YOLOv7-tiny. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] Current object detection technology relies on computer vision, deep learning, and sensor technology. Traditional methods include algorithms based on feature extraction and machine learning, as well as methods based on sensor data fusion. In recent years, the rise of deep learning technology has brought major breakthroughs in the field of object detection, such as convolutional neural network structures, which greatly improve the accuracy and robustness of detection. Object detection is divided into one-stage detection and two-stage detection. The two-stage detection first generates candidate regions, and then classifies and regresses these candidate regions; the one-stage object detection directly predicts the category and location of the object from the input image. Object detection aims to accurately identify and locate the target objects of interest, such as people, cars, animals, etc., from images or videos, providing key visual information processing capabilities for video surveillance, aviation and other fields.

[0004] In today's fast-paced air transport industry, the volume of airport operations continues to increase, the density of aircraft takeoffs and landings increases, and the safety dispatch of airport flight areas has always attracted much attention. As an important part of the air transport network, the safety and efficiency of airport runways are crucial to ensure the punctual takeoff and landing of flights and the safety of passengers. However, there are many safety hazards on airport runways, one of which is the improper entry of vehicles and pedestrians. In the past, vehicles or pedestrians mistakenly entered the airport runway, which caused extremely serious safety consequences. This has prompted people to realize and pay attention to the necessity and urgency of vehicle and pedestrian detection on airport runways. With the continuous advancement of sensors, cameras and deep learning, more reliable and efficient means are provided for vehicle and pedestrian detection on airport runways. It not only improves the safety of airport operations, but also improves the work efficiency of airport staff, and escorts aviation transportation safety.

[0005] The current vehicle and pedestrian detection technology is difficult to accurately locate and identify targets in the complex and changing airport environment, such as background interference, obstructions, etc., which seriously affects the accuracy and robustness of detection. Existing technologies also have a certain lag in real-time detection and cannot achieve real-time monitoring of vehicles and pedestrians. In a highly dense and fast-paced environment like an airport, real-time performance is very important, and any delay may lead to safety hazards.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0007] Purpose of the invention: The technical problem to be solved by the present invention is to provide a vehicle and pedestrian detection method on an airport runway based on an improved YOLOv7-tiny in view of the shortcomings of the prior art.

[0008] In order to solve the above technical problems, the present invention discloses a vehicle and pedestrian detection method on an airport runway based on an improved YOLOv7-tiny, comprising the following steps:

[0009] Step 1: Build a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny;

[0010] Step 2, designing a loss function for training the model;

[0011] Step 3: Establish a vehicle and pedestrian detection dataset and divide it into a training set and a test set;

[0012] Step 4, using the training set, training a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny;

[0013] Step 5, inputting the test set into the trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway, testing the performance of the model, and evaluating the detection accuracy and speed of the model;

[0014] Step 6: Use the trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway to perform vehicle and pedestrian detection on the airport runway.

[0015] Furthermore, the construction described in step 1 is based on the improved YOLOv7-tiny vehicle and pedestrian detection model on the airport runway, that is, the existing YOLOv7-tiny model is modified, specifically including:

[0016] Step 1-1, add a 1×1 convolution layer based on the 3×3 convolution layer in the YOLOv7-tiny model and perform residual connection;

[0017] Step 1-2: In the inference phase of the YOLOv7-tiny model, the 1×1 convolutional layer and residual connection added in step 1-1 are fused using the reparameterization method to fuse the multi-branch structure into a single-branch structure.

[0018] Step 1-3, introduce the coordinate attention mechanism CA into the connection part of the backbone network module Backbone and the neck module Neck of the YOLOv7-tiny model;

[0019] In steps 1-4, a global attention mechanism GAM is added before the last three detection heads in the YOLOv7-tiny model.

[0020] Furthermore, the design loss function described in step 2 uses the Focal CIOU loss function as the bounding box regression loss function.

[0021] Furthermore, the establishment of a vehicle and pedestrian detection dataset described in step 3 specifically includes:

[0022] Step 3-1, select the public data set PASCAL VOC as the vehicle and pedestrian detection data set, among which VOC2007train+trainval and VOC2012train+trainval are selected as training sets, and VOC2007test is selected as the test set;

[0023] In step 3-2, the training set and test set are preprocessed, and the Mosaic and Mixup methods are used for data enhancement.

[0024] Furthermore, the training described in step 4 is based on the improved YOLOv7-tiny vehicle and pedestrian detection model on the airport runway, specifically including:

[0025] The training set preprocessed in step 3-2 is input into the vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway constructed in step 1 for training until all samples in the training set are trained for a preset number of times, and the training is terminated to obtain a trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway.

[0026] Furthermore, the fusion is performed using the reparameterization method described in step 1-2, specifically including:

[0027] Step 1-2-1, perform the fusion of convolution and batch normalization BN, as follows:

[0028] The weights and bias terms of the convolutional layer are fused with the stretching parameter γ and bias parameter β of BN. The fused convolution kernel parameters are expressed as:

[0029] γ*(convolutional layer weight / standard deviation)

[0030] The fused bias term is expressed as:

[0031] β+(γ*(convolutional layer bias term - mean) / standard deviation);

[0032] Step 1-2-2, perform the fusion of convolution branches, as follows:

[0033] In the convolution branch obtained by the fusion of convolution and BN, the 1×1 convolution is padded to expand it to a larger receptive field, turning it into a 3×3 convolution. For the residual connection, it is converted to a 1×1 convolution, and then padded to expand the convolution kernel to a size of 3×3. The converted 1×1 convolution branch, the residual connection, and the original 3×3 convolution branch are connected together and fused into a single 3×3 convolution branch.

[0034] Furthermore, the CA attention mechanism is introduced into the connection part of the Backbone module and the Neck module of the YOLOv7-tiny model described in steps 1-3, specifically including:

[0035] Step 1-3-1, extract features from the input image using global average pooling kernels (H, 1) and (1, W) to obtain feature maps Z in two different horizontal directions of height and width h and Z w , where H represents height and W represents width;

[0036] Step 1-3-2, the feature map Z h and Z w Perform concatenate connection, use convolution transformation and nonlinear activation function to transform features, and generate feature map f∈R 1×(W×H)×C / r ; Where R represents space, C represents channel, and r represents the downsampling ratio;

[0037] Step 1-3-3, split the feature map f along the spatial dimension to obtain the feature map f h and f w , use convolution and sigmoid activation function to calculate the attention weight between each position and other positions, and get the attention weight g h and g w ;

[0038] Step 1-3-4, according to the calculated attention weight g h and g w , dynamic weighted fusion is performed to obtain the weighted feature map in the height and width directions.

[0039] Furthermore, the GAM attention mechanism described in steps 1-4 specifically includes:

[0040] The channel attention submodule and the spatial attention submodule use three-dimensional transformation to save information across three dimensions and use multi-layer perceptrons to amplify spatial correlation across channels. The spatial attention submodule uses two convolutional layers to fuse spatial information, reduce losses and amplify global interactions, as follows:

[0041]

[0042]

[0043] Among them, the input feature map is F1, the intermediate state feature map is F2, and the output feature map is F3: M c and M s They are channel attention submodule and spatial attention submodule respectively; Represents element-wise multiplication.

[0044] Furthermore, the Focal CIOU loss function described in step 2 is used as the bounding box regression loss function, specifically including:

[0045] Step 2-1, calculate the bounding box regression loss function L CIOU , as follows:

[0046]

[0047]

[0048] Among them, IOU represents the ratio of the intersection area of ​​the predicted box and the real box to the union area. A value of 1 indicates complete overlap, and a value of 0 indicates no overlap. An IOU value greater than the threshold indicates that the target has been detected; b represents the predicted box, b gt represents the real box; c 2 Represents the square of the diagonal length of the minimum enclosing rectangle of the two bounding boxes, the predicted box and the real box; ρ 2 (b,b gt ) represents the distance between the coordinates of the center points of the two bounding boxes; α is the balance factor, which takes a value between [-1, 1] and is calculated as follows:

[0049]

[0050] The parameter v is used to determine whether the aspect ratio of the bounding box is consistent. The calculation method is as follows:

[0051]

[0052] Among them, w gt Indicates the width of the real box, h gt represents the height of the real box, w represents the width of the predicted box, and h represents the height of the predicted box;

[0053] Step 2-2, use the IOU value to L CIOU The weighting is as follows:

[0054] L Focal CIOU =IOU γ L CIOU

[0055] Among them, γ is a coefficient that controls the degree of outlier suppression.

[0056] Furthermore, the Mosaic and Mixup methods described in step 3-2 are used to enhance the data, specifically including:

[0057] Step 3-2-1, perform Mosaic data enhancement, as follows:

[0058] Select four different images from the training set and randomly crop each image to retain the complete object or only a part of the object;

[0059] The four cropped images are stitched together according to preset rules to form a larger composite image;

[0060] Step 3-2-2, perform Mixup data enhancement, as follows:

[0061] Randomly select two different images from the training set, perform weighted averaging on their positions, sizes, and categories, and generate new training samples.

[0062] Beneficial effects:

[0063] The present invention realizes a vehicle and pedestrian detection model on an airport runway, improves the accuracy of vehicle and pedestrian detection, and can be applied in environments such as autonomous driving, traffic safety, and airports. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0065] Figure 1 The figure is a flow chart of the method of the present invention.

[0066] Figure 2 It is a schematic diagram of the network structure of the present invention.

[0067] Figure 3 Schematic diagram of the network structure of the CA attention mechanism.

[0068] Figure 4 Schematic diagram of the network structure of the GAM attention mechanism.

[0069] Figure 5 It is a schematic diagram of specific detection results in an embodiment. DETAILED DESCRIPTION

[0070] The present invention provides a vehicle and pedestrian detection method on an airport runway based on an improved YOLOv7-tiny based on the existing YOLOv7-tiny image recognition model (reference: C.-Y.Wang, A.Bochkovskiy, and H.-YM Liao, "Yolov7: Trainable bag-of-freebies sets new state-of-the-artfor real-time object detectors", in 2023 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.7464-7475, doi:10.1109 / CVPR52729.2023.00721, 2023), so as to improve the accuracy and speed of vehicle and pedestrian detection, enhance the safety management of airport runways, and ensure the safety and efficiency of aircraft take-off and landing.

[0071] In order to achieve the above object, the technical solutions adopted by the present invention are as follows:

[0072] The vehicle and pedestrian detection method on the airport runway based on the improved YOLOv7-tiny includes the following steps:

[0073] Step 1: Build a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny. Figure 2 As shown, the red part is the improvement of the existing model in the present invention, specifically:

[0074] Step 1.1: Add 1×1 convolution and residual connection on the basis of 3×3 convolution to improve the feature extraction capability of vehicles and pedestrians and enhance the safety of airport runways.

[0075] Step 1.2: In the inference stage, the convolutional layers and residual connections added in step 1.1 are re-parameterized to merge multiple branches into a single branch structure, thereby improving the inference speed of the vehicle and pedestrian detection model and performing real-time detection.

[0076] Step 1.3, introduce the CA (Coordinate attention) mechanism in the connection part between Backbone and Neck to help the network model find the area of ​​interest more quickly and accurately;

[0077] Step 1.4: Add the GAM (Global Attention Mechanism) attention mechanism before sending to the last three detection heads to improve the accuracy of vehicle and pedestrian detection under complex background conditions and reduce information loss;

[0078] Step 2: Use the proposed Focal CIOU loss function as the bounding box regression loss function to focus the regression process on high-quality anchor boxes, better locate vehicles and pedestrians, and ensure safe scheduling of the runway.

[0079] Step 3: Create a vehicle and pedestrian detection dataset. Specifically:

[0080] Step 3.1, select the public data set PASCAL VOC to establish the vehicle and pedestrian detection training set and test set, select VOC2007train+trainval and VOC2012train+trainval as the training set, and VOC2007test as the test set;

[0081] In step 3.2, the established vehicle and pedestrian detection training set and test set are preprocessed, and the Mosaic and Mixup algorithms are used to enhance the vehicle and pedestrian data.

[0082] Step 4: Use the dataset to train a vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway. Specifically:

[0083] Step 4.1, set the training parameters, set the training batch size to 64, and the training iteration number to 300;

[0084] In step 4.2, the vehicle detection training set preprocessed in step 3.2 is input into the vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway constructed in step 1 for training until all training samples in the training set have been trained for the number of times required by the hyperparameter epoch, and the training is terminated to obtain a well-converged vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway.

[0085] Step 5: Input the vehicle and pedestrian detection test set into the trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway to test the performance of the model.

[0086] Step 6: Evaluate the model.

[0087] According to the test results obtained in step 5, the vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway trained in step 4 is evaluated from the average detection accuracy and detection speed.

[0088] Example:

[0089] An embodiment of the present invention is further described in detail below in conjunction with the accompanying drawings to clarify the technical solution of the present invention.

[0090] Figure 1 The vehicle and pedestrian detection method based on the improved YOLOv7-tiny on the airport runway according to the present invention is shown, comprising the following steps:

[0091] Step 1: Build a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny. Figure 2 The red part is the improvement of the existing model in this embodiment, specifically:

[0092] Step 1.1: Add 1×1 convolution and residual connection on the basis of 3×3 convolution to improve the feature extraction capability of vehicles and pedestrians and enhance the safety of airport runways.

[0093] Step 1.2: In the inference stage, the convolutional layers and residual connections added in step 1.1 are re-parameterized to merge multiple branches into a single branch structure, thereby improving the inference speed of the vehicle and pedestrian detection model and performing real-time detection. The specific steps are as follows:

[0094] Reparameterization can reduce the computational complexity and number of parameters of the model, improve the inference speed of the model, and improve the generalization ability of the model to a certain extent. There are two aspects. First, the convolution and BatchNormalization (BN) are integrated to integrate the weights and bias items of the convolution layer with the stretching parameter γ and bias parameter β of BN. The fused convolution kernel parameters can be expressed as γ*(convolution layer weight / standard deviation), and the bias item can be expressed as β+(γ*(convolution layer bias item-average value) / standard deviation).

[0095] The second is the fusion of convolution branches. In the convolution branch obtained by fusion of convolution and BN, the 1×1 convolution is first padded to expand it to a larger receptive field, turning it into a 3×3 convolution; secondly, for the residual connection, it is first converted to a 1×1 convolution, and then padded to obtain a convolution kernel size of 3×3; finally, the converted 1×1 convolution branch, the residual connection, and the original 3×3 convolution branch are connected together, and the convolution operations of multiple branches are fused into a single 3×3 convolution branch. This conversion and fusion method helps to reduce the complexity and number of parameters of the network while maintaining the ability to effectively model the input features.

[0096] Step 1.3, introduce the CA (Coordinate Attention) mechanism in the connection between Backbone and Neck, such as Figure 3 As shown, it helps the network model find the area of ​​interest more quickly and accurately. The specific steps are as follows:

[0097] First, the input image is feature extracted by global average pooling kernels (H, 1) and (1, W) to obtain feature maps Z in two different horizontal directions of height and width. h and Z w , (H, 1) and (1, W) are the sizes of the average pooling convolution kernels, and the average pooling is performed on the height and width of the input feature map at different levels; secondly, the feature maps Z in the two directions are h and Z w Perform concatenate connection, use convolution transformation and nonlinear activation function to transform features, and generate feature map f∈R 1 ×(W×H)×C / r ; Then, along the spatial dimension, the feature map f is split to obtain f h and f w , use convolution and sigmoid activation function to calculate the attention weight between each position and other positions, and get g h and g w ; Finally, according to the calculated attention weight g h and g w , dynamic weighted fusion is performed to obtain the weighted feature map in the height and width directions, thereby strengthening the feature representation of important positions.

[0098] CA attention helps the network better understand and utilize location information by modeling the relationship between features at different locations in the image, thereby improving the performance of image understanding and processing.

[0099] Step 1.4, before sending to the last three detection heads, add GAM (Global Attention Mechanism) attention mechanism to improve the accuracy of vehicle and pedestrian detection under complex background conditions and reduce information loss, such as Figure 4 As shown, the specific steps are as follows:

[0100] GAM considers important information of channel and spatial dimensions. The channel attention submodule uses 3D transformation to preserve information across three dimensions and uses multi-layer perceptron to amplify spatial correlation across channels. The spatial attention submodule uses two convolutional layers to fuse spatial information, improving the performance of the network by reducing loss and amplifying global interactions.

[0101] Given the input feature map F1, the intermediate state feature map F2, and the output feature map F3:

[0102]

[0103]

[0104] Among them, M c and M s They are channel attention submodule and spatial attention submodule respectively; Represents element-wise multiplication.

[0105] Step 2: Use the proposed Focal CIOU loss function as the bounding box regression loss function to focus the regression process on high-quality anchor boxes, better locate vehicles and pedestrians, and ensure safe scheduling of the runway.

[0106] The CIOU loss function takes into account the intersection-over-union ratio between bounding boxes, the center point distance, and the aspect ratio of the bounding box, which can more comprehensively reflect the matching degree of the target detection box. The calculation formula is as follows:

[0107]

[0108]

[0109]

[0110] Among them, IOU (Intersection over Union) represents the ratio of the intersection area of ​​the predicted box and the true box to the union area. 1 represents complete overlap and 0 represents no overlap. It can evaluate the degree of overlap between the predicted box and the true box. When the IOU value is greater than the specified threshold, it means that the target has been detected, while when it is less than the specified threshold, it means that the target has not been correctly detected; b represents the predicted box, b gt represents the real box; c 2 Respectively represent the square of the diagonal length of the minimum enclosing rectangle of the two bounding boxes; ρ 2 (b,b gt ) represents the distance between the coordinates of the center points of the two bounding boxes; α is a balance factor with a value between [-1, 1]; v is used to measure whether the aspect ratio of the bounding box is consistent.

[0111] In order to solve the problem of imbalance between difficult and easy samples and improve the contribution of high-quality anchor boxes in the network model, the IOU value is used to weight L CIOU , reduce the weight of easy-to-classify samples and increase the weight of difficult-to-classify samples. The specific formula is as follows:

[0112] L Focal CIOU =IOU γ L CIOU

[0113] Among them, γ is used to control the degree of outlier suppression and is set to γ ​​= 0.5 as the default value.

[0114] Step 3: Create a vehicle and pedestrian detection dataset. Specifically:

[0115] Step 3.1, select the public data set PASCAL VOC to establish the vehicle and pedestrian detection training set and test set, select VOC2007train+trainval and VOC2012train+trainval as the training set, and VOC2007test as the test set;

[0116] In step 3.2, the established vehicle and pedestrian detection training set and test set are preprocessed, and the Mosaic and Mixup algorithms are used to enhance the vehicle and pedestrian data.

[0117] Mosaic data augmentation selects four different images from the training set, and then randomly crops each image, retaining the complete object or only a part of the object; the four cropped images are stitched together according to certain rules to form a larger composite image. Through Mosaic data augmentation, the model can see objects in different scenes in one image at the same time, which helps improve the model's generalization ability to different scenes, scales, and complexities.

[0118] Mixup data enhancement randomly selects two different images from the training set, performs weighted averaging on their positions, sizes, and categories, and generates new training samples. Mixup data enhancement can help the model learn smoother decision boundaries, thereby improving the generalization ability of the model. It can also reduce the model's sensitivity to noise and abnormal samples, and help prevent overfitting.

[0119] Step 4: Use the dataset to train a vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway. Specifically:

[0120] Step 4.1, set the training parameters, set the training batch size to 64, and the training iteration number to 300;

[0121] In step 4.2, the vehicle detection training set preprocessed in step 2.2 is input into the vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway constructed in step 1 for training until all training samples in the training set have been trained for the number of times required by the hyperparameter epoch, and the training is terminated to obtain a well-converged vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway.

[0122] Step 5: Input the vehicle and pedestrian detection test set into the trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway to test the performance of the model.

[0123] Step 6: Evaluate the model.

[0124] According to the test results obtained in step 5, the vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway trained in step 4 is evaluated from the average detection accuracy and detection speed.

[0125] In order to better demonstrate the effectiveness of the present invention, the detection effects of the original YOLOv7-tiny and the improved YOLOv7-tiny on the vehicle and pedestrian detection dataset are compared, and the results are as follows:

[0126] Table 1 Detection results of YOLOv7-tiny and improved YOLOv7-tiny

[0127] Model mAP@0.5 mAP@0.5:0.95 FPS YOLOv7-tiny 79.4% 54.4% 275 Improved YOLOv7-tiny 82.2% 59.6% 252

[0128] mAP is the average accuracy of each category, which is used to measure the accuracy of the model at different IOU thresholds. mAP@0.5 is the mAP value when the IOU threshold is 0.5, and mAP@0.5:0.95 is the mAP value when the IOU threshold range is 0.5 to 0.95. FPS represents the number of images processed per second, which is used to measure the inference speed of the model.

[0129] like Figure 5 FIG. 2 is a schematic diagram of the detection results using the method of the present application in another embodiment.

[0130] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and the computer program can run the invention content of a vehicle and pedestrian detection method based on an improved YOLOv7-tiny on an airport runway provided by the present invention and some or all of the steps in each embodiment when executed by the data processing unit. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0131] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on such an understanding, the technical solutions in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of computer programs, i.e., software products, which can be stored in a storage medium and include several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0132] The present invention provides a method and idea for detecting vehicles and pedestrians on airport runways based on an improved YOLOv7-tiny. There are many methods and approaches to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A vehicle and pedestrian detection method on an airport runway based on an improved YOLOv7-tiny, characterized in that: The steps include: Step 1: Build a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny; Step 2, designing a loss function for training the model; Step 3: Establish a vehicle and pedestrian detection dataset and divide it into a training set and a test set; Step 4, using the training set, training a vehicle and pedestrian detection model on the airport runway based on the improved YOLOv7-tiny; Step 5, inputting the test set into the trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway, testing the performance of the model, and evaluating the detection accuracy and speed of the model; Step 6: Use the trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway to perform vehicle and pedestrian detection on the airport runway.

2. According to claim 1, a vehicle and pedestrian detection method based on improved YOLOv7-tiny on an airport runway is characterized in that: The construction described in step 1 is based on the improved YOLOv7-tiny vehicle and pedestrian detection model on the airport runway, that is, the existing YOLOv7-tiny model is modified, specifically including: Step 1-1, add a 1×1 convolution layer based on the 3×3 convolution layer in the YOLOv7-tiny model and perform residual connection; Step 1-2: In the inference phase of the YOLOv7-tiny model, the 1×1 convolutional layer and residual connection added in step 1-1 are fused using the reparameterization method to fuse the multi-branch structure into a single-branch structure. Step 1-3, introduce the coordinate attention mechanism CA into the connection part of the backbone network module Backbone and the neck module Neck of the YOLOv7-tiny model; In steps 1-4, a global attention mechanism GAM is added before the last three detection heads in the YOLOv7-tiny model.

3. According to claim 2, a vehicle and pedestrian detection method based on improved YOLOv7-tiny on an airport runway is characterized in that: The design loss function described in step 2 uses the Focal CIOU loss function as the bounding box regression loss function.

4. According to claim 3, a vehicle and pedestrian detection method based on improved YOLOv7-tiny on an airport runway is characterized in that: The establishment of a vehicle and pedestrian detection dataset described in step 3 specifically includes: Step 3-1, select the public data set PASCAL VOC as the vehicle and pedestrian detection data set, among which VOC2007train+trainval and VOC2012train+trainval are selected as training sets, and VOC2007test is selected as the test set; In step 3-2, the training set and test set are preprocessed, and the Mosaic and Mixup methods are used for data enhancement.

5. According to claim 4, a vehicle and pedestrian detection method based on improved YOLOv7-tiny on an airport runway is characterized in that: The training described in step 4 is based on the improved YOLOv7-tiny vehicle and pedestrian detection model on the airport runway, which specifically includes: The training set preprocessed in step 3-2 is input into the vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway constructed in step 1 for training until all samples in the training set are trained for a preset number of times, and the training is terminated to obtain a trained vehicle and pedestrian detection model based on the improved YOLOv7-tiny on the airport runway.

6. According to claim 5, a vehicle and pedestrian detection method based on improved YOLOv7-tiny on an airport runway is characterized in that: The fusion using the reparameterization method described in step 1-2 includes: Step 1-2-1, perform the fusion of convolution and batch normalization BN, as follows: The weights and bias terms of the convolutional layer are fused with the stretching parameter γ and bias parameter β of BN. The fused convolution kernel parameters are expressed as: γ*(convolutional layer weight / standard deviation) The fused bias term is expressed as: β+(γ*(convolutional layer bias term - mean) / standard deviation); Step 1-2-2, perform the fusion of convolution branches, as follows: In the convolution branch obtained by the fusion of convolution and BN, the 1×1 convolution is padded to expand it to a larger receptive field, turning it into a 3×3 convolution. For the residual connection, it is converted to a 1×1 convolution, and then padded to expand the convolution kernel to a size of 3×3. The converted 1×1 convolution branch, the residual connection, and the original 3×3 convolution branch are connected together and fused into a single 3×3 convolution branch.

7. The vehicle and pedestrian detection method based on the improved YOLOv7-tiny on the airport runway according to claim 6 is characterized in that: The CA attention mechanism is introduced in the connection part of the Backbone module and the Neck module of the YOLOv7-tiny model described in steps 1-3, specifically including: Step 1-3-1, extract features from the input image using global average pooling kernels (H, 1) and (1, W) to obtain feature maps Z in two different horizontal directions: height and width. h and Z w , where H represents height and W represents width; Step 1-3-2, feature map Z h and Z w Perform concatenate connection, use convolution transformation and nonlinear activation function to transform features, and generate feature map f∈R 1×(W×H)×C / r ; Where R represents space, C represents channel, and r represents the downsampling ratio; Step 1-3-3, split the feature map f along the spatial dimension to obtain the feature map f h and f w , use convolution and sigmoid activation function to calculate the attention weight between each position and other positions, and get the attention weight g h and g w ; Step 1-3-4, according to the calculated attention weight g h and g w , dynamic weighted fusion is performed to obtain the weighted feature map in the height and width directions.

8. The vehicle and pedestrian detection method based on improved YOLOv7-tiny on an airport runway according to claim 7, characterized in that: The GAM attention mechanism described in steps 1-4 specifically includes: The channel attention submodule and the spatial attention submodule use three-dimensional transformation to save information across three dimensions and use multi-layer perceptrons to amplify spatial correlation across channels. The spatial attention submodule uses two convolutional layers to fuse spatial information, reduce losses and amplify global interactions, as follows: Among them, the input feature map is F1, the intermediate state feature map is F2, and the output feature map is F3: M c and M s They are channel attention submodule and spatial attention submodule respectively; Represents element-wise multiplication.

9. The method for detecting vehicles and pedestrians on airport runways based on improved YOLOv7-tiny according to claim 8, characterized in that: The Focal CIOU loss function described in step 2 is used as the bounding box regression loss function, specifically including: Step 2-1, calculate the bounding box regression loss function L CIOU , as follows: Among them, IOU represents the ratio of the intersection area of ​​the predicted box and the real box to the union area. A value of 1 indicates complete overlap, and a value of 0 indicates no overlap. An IOU value greater than the threshold indicates that the target has been detected; b represents the predicted box, b gt represents the real box; c 2 Represents the square of the diagonal length of the minimum enclosing rectangle of the two bounding boxes, the predicted box and the real box; ρ 2 (b,b gt ) represents the distance between the coordinates of the center points of the two bounding boxes; α is the balance factor, which takes a value between [-1,1] and is calculated as follows: The parameter v is used to determine whether the aspect ratio of the bounding box is consistent. The calculation method is as follows: Among them, w gt Indicates the width of the real box, h gt represents the height of the real box, w represents the width of the predicted box, and h represents the height of the predicted box; Step 2-2, use the IOU value to L CIOU The weighting is as follows: L FocalCIOU =IOU γ L CIOU Among them, γ is a coefficient that controls the degree of outlier suppression.

10. The vehicle and pedestrian detection method based on improved YOLOv7-tiny on an airport runway according to claim 9, characterized in that: The data enhancement using Mosaic and Mixup methods described in step 3-2 specifically includes: Step 3-2-1, perform Mosaic data enhancement, as follows: Select four different images from the training set and randomly crop each image to retain the complete object or only a part of the object; The four cropped images are stitched together according to preset rules to form a larger composite image; Step 3-2-2, perform Mixup data enhancement, as follows: Randomly select two different images from the training set, perform weighted averaging on their positions, sizes, and categories, and generate new training samples.

Citation Information

Cited By

  • Foreign matter identification and detection method for unmanned motor sweeper in airport

    CN122073045A