Vehicle detection algorithm and device based on FasterNet lightweight framework and feature fusion

Abstract

This application provides a vehicle detection algorithm and device based on the FasterNet lightweight framework and feature fusion, involving the fields of deep learning and image processing. The algorithm includes: inputting an image and preprocessing the image, which mainly includes LetterBox and normalization operations; inputting the preprocessed image into a backbone network based on the FasterNet network and multi-head attention mechanism for feature extraction to obtain a preliminary extracted feature map; inputting the preliminary extracted feature map into a neck network based on the F-C2f module for multi-scale feature fusion to obtain a hybrid fusion feature map; inputting the hybrid fusion feature map into the head network of the vehicle detection model, and training using the Focal-EIoU loss function. When the training cycle reaches 100, training is stopped to obtain a trained vehicle detection model; and inputting a test image into the trained vehicle detection model to obtain a vehicle detection result for the test image. The vehicle detection model provided by this algorithm can take into account both the accuracy and real-time performance of vehicle detection.

Description

Technical Field

[0001] The present application relates to the fields of deep learning and image processing, and in particular to a vehicle detection algorithm and device based on the FasterNet lightweight framework and feature fusion. Background Art

[0002] With the rapid development of intelligent transportation systems and autonomous driving technologies, vehicle detection, as an important branch of target detection, is of great significance in improving traffic safety and optimizing traffic management.

[0003] Vehicle detection involves locating vehicles of interest in an image, assigning a category based on their features, and enclosing them with bounding boxes. In recent years, with the rapid development of IoT and computer vision technologies, deep learning-based vehicle detection algorithms have gained widespread attention. Common deep learning-based vehicle detection algorithms include two-stage detection algorithms, such as the R-CNN (Regions with Convolutional Neural Network) series, and single-stage detection algorithms, such as the SSD (Single Shot Multibox Detector) and YOLO (You Only Look Once) series. Two-stage detection algorithms offer higher accuracy but are slower, while single-stage detection algorithms, while faster, sacrifice a certain degree of accuracy. Therefore, traditional vehicle detection algorithms fail to meet the demands for both real-time and accurate vehicle detection in traffic scenarios. Summary of the Invention

[0004] This application provides a vehicle detection algorithm and device based on the FasterNet lightweight framework and feature fusion. The technical solution is as follows:

[0005] According to one aspect of the present application, a vehicle detection algorithm based on the FasterNet lightweight framework and feature fusion is provided, the algorithm comprising:

[0006] Input an image and perform preprocessing on the image, wherein the preprocessing mainly includes LetterBox and normalization operations;

[0007] The preprocessed image is input into the backbone network of the vehicle detection model for feature extraction to obtain a preliminary extracted feature map. The feature extraction network in the backbone network is a feature extraction network based on a lightweight FasterNet network and a multi-head attention mechanism;

[0008] Inputting the initially extracted feature map into the neck network for multi-scale feature fusion to obtain a hybrid fusion feature map, wherein the neck network is a neck network based on the F-C2f module;

[0009] Input the hybrid fusion feature map into the head network of the vehicle detection model, use the Focal-EIoU loss function for training, and stop training when the training cycle reaches 100 to obtain a trained vehicle detection model;

[0010] Input the test image into the trained vehicle detection model to obtain the vehicle detection result of the test image.

[0011] According to another aspect of the present application, a vehicle detection device based on the FasterNet lightweight framework and feature fusion is provided, the device comprising:

[0012] A preprocessing module is used to input an image and perform preprocessing on the image, wherein the preprocessing mainly includes LetterBox and normalization operations;

[0013] A feature extraction module is used to input the preprocessed image into the backbone network of the vehicle detection model for feature extraction to obtain a preliminary extracted feature map. The feature extraction network in the backbone network is a feature extraction network based on a lightweight FasterNet network and a multi-head attention mechanism;

[0014] A feature fusion module is used to input the preliminary extracted feature map into the neck network based on the F-C2f module for multi-scale feature fusion to obtain a hybrid fusion feature map, wherein the neck network is a neck network based on the F-C2f module;

[0015] A model training module is used to input the hybrid fusion feature map into the head network of the vehicle detection model and use the Focal-EIoU loss function for training. When the training cycle reaches a set value, the model training stops. In the process of training the vehicle detection model in this application, a total of 100 training cycles (epochs) are set to ensure that the vehicle detection model is fully converged;

[0016] The model prediction module is used to input the test image into the trained vehicle detection model to obtain the vehicle detection result of the test image.

[0017] According to one aspect of the present application, an electronic device is provided, comprising: a processor and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the vehicle detection algorithm and device based on the FasterNet lightweight framework and feature fusion as described above.

[0018] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the vehicle detection algorithm and device based on the FasterNet lightweight framework and feature fusion as described above.

[0019] According to another aspect of the present application, a computer program product is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to execute the aforementioned vehicle detection algorithm and apparatus based on the FasterNet lightweight framework and feature fusion.

[0020] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0021] In response to the problems of high computational complexity, poor real-time performance, and high resource consumption of the current deep learning-based vehicle detection algorithm in traffic road scenes, as well as low detection accuracy of multiple targets and small targets in complex scenes, which makes it difficult to meet the requirements of vehicle detection in practical application scenarios such as intelligent transportation and autonomous driving, this application proposes a vehicle detection algorithm with FasterNet lightweight framework and feature fusion. The model uses FasterNet lightweight network to replace the CSPDarkNet53 backbone network of the original YOLOv8, combines the idea of partial convolution, and reconstructs the C2f module in the neck of YOLOv8, realizing efficient fusion of multi-scale features, reducing the complexity and computational cost of the model, and proposing Focal-EIoU loss function to replace the CIoU loss function in YOLOv8, accurately defining the aspect ratio of the prediction box, alleviating the imbalance problem of positive and negative samples, and improving the detection accuracy of multiple targets and small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Further details, features and advantages of the present application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A flowchart of a vehicle detection algorithm based on the FasterNet lightweight framework and feature fusion according to an exemplary embodiment of the present application is shown;

[0024] Figure 2 This is the structural diagram of the FasterNet network;

[0025] Figure 3 It is the convolution structure diagram of the PConv module;

[0026] Figure 4 This is the structural diagram of the F-C2f module;

[0027] Figure 5 This is the structural diagram of the multi-head self-attention mechanism;

[0028] Figure 6 It is a parameter diagram of EIoU loss;

[0029] Figure 7 This is a network structure diagram of an improved YOLOv8 model provided by an exemplary embodiment of the present application;

[0030] Figure 8 The following are the results of vehicle detection experiments using four methods;

[0031] Figure 9 Schematic diagram of a vehicle detection device based on the FasterNet lightweight framework and feature fusion provided in an embodiment of the present application;

[0032] Figure 10 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present application is shown. DETAILED DESCRIPTION

[0033] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0034] It should be understood that the various steps described in the algorithm implementation of the present application can be performed in different orders and / or in parallel. In addition, the algorithm implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0035] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; and the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. It should be noted that the modifiers of "one" and "a plurality of" mentioned in this application are illustrative and not restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, they should be understood as "one or more". The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not used to limit the scope of these messages or information.

[0036] The following describes the solution of the present invention with reference to the accompanying drawings, and illustrates the technical solution provided by the embodiments of the present application in detail through specific embodiments and their application scenarios.

[0037] Currently, deep learning-based vehicle detection algorithms include two-stage algorithms and single-stage algorithms. The two-stage vehicle detection algorithm inherits the sliding window concept of traditional detection algorithms. The entire algorithm is divided into two parts: candidate region selection and classification. The advantage of this algorithm is that the selection of multiple candidate boxes can fully extract the feature information of vehicle targets, resulting in high detection accuracy and precise positioning. However, since it is performed in two steps, it runs slowly and has low real-time performance, making it suitable for scenarios with high accuracy requirements. The single-stage vehicle detection algorithm omits the candidate region selection step, simplifies the network structure, and greatly improves the algorithm's detection speed. However, it also leads to insufficient feature extraction, sacrifices a certain degree of accuracy, and reduces the accuracy of target positioning. It is suitable for scenarios with high real-time requirements.

[0038] Therefore, in order to address the problems of high computational complexity, poor real-time performance, and high resource consumption of the current deep learning-based vehicle detection algorithm in traffic road scenarios, as well as the low accuracy of multi-target and small target detection in complex scenarios, which makes it difficult to meet the requirements of vehicle detection in practical application scenarios such as intelligent transportation and autonomous driving, this application proposes a vehicle detection algorithm and device with a FasterNet lightweight framework and feature fusion, which improves the real-time performance and accuracy of vehicle detection in complex scenarios such as urban roads, highways, and severe weather, and reduces resource consumption.

[0039] Please refer to Figure 1, which shows a flow chart of a vehicle detection algorithm based on the FasterNet lightweight framework and feature fusion according to an exemplary embodiment of the present application. The algorithm is applied to electronic equipment as an example. Figure 1 As shown, the algorithm includes:

[0040] Step 101: input an image and perform preprocessing on the image. The preprocessing mainly includes LetterBox and normalization operations.

[0041] The vehicle detection algorithm based on deep learning essentially requires pre-training of the vehicle detection model. After the training completion conditions are met, the trained vehicle detection model can be used to detect vehicles on the input image. Before training the vehicle detection model, it is first necessary to select a dataset as model training data. In the vehicle detection application scenario, this application selects the UA-DETRAC dataset as the model training data. The images in the dataset are vehicle images taken at 24 different locations in the city using a Cannon EOS 550D camera on various road types (such as highways, urban main roads, secondary roads) and in various weather conditions (such as normal weather and bad weather); moreover, the vehicle images in the dataset have three different degrees of occlusion, namely, completely visible, partially obscured by other vehicles, and partially obscured by the scene; the vehicle types contained in the images in the dataset include four categories commonly seen in road traffic: "Car", "Bus", "Van", and "Other (other types of vehicles)". Each vehicle type includes a variety of vehicles from small to large, from private to public, and the dataset is diverse. Since the number of images in this dataset is very large and the differences between adjacent images are very small, this application randomly extracts the UA-DETRAC dataset, selecting one out of every ten images as the final dataset, resulting in a training set of 8232 images and a test set of 5625 images.

[0042] Optionally, in order to increase the diversity of the dataset, the images in the training set can also be processed using Mosaic data augmentation technology. Specifically, four images are randomly selected from the training set, and different random scaling ratios between 0.5 and 2.0 are applied to each image to enhance the model's detection ability for vehicles of different sizes. According to the 640*640 image size required for the vehicle detection model input, a random cropping operation is performed on the scaled image to ensure that the cropped area is close to or equal to the required size and contains key target information (e.g., vehicle). If the cropped image block size is not large enough to fill the preset canvas (such as a 2*2 grid layout), adjustments are made by re-cropping, adjusting the scaling ratio, or changing the cropping strategy to ensure that the image blocks can be seamlessly combined. The four cropped and adjusted image blocks are arranged in a random order on a new canvas. If the cropped image block size is not large enough to fill the entire canvas, black (grayscale value 0) or white (grayscale value 255) is used as the fill color to ensure the consistency of the Mosaic image size. These two colors will not visually conflict strongly with most natural scenes, nor will they introduce additional color information to interfere with model training.

[0043] Mosaic technology increases the diversity of the training dataset by stitching together four different images. This not only improves the model's generalization capabilities but also enhances the model's detection of small objects, potentially including those that would otherwise be difficult to detect, through random scaling and cropping. Furthermore, by increasing the amount of training data, Mosaic data augmentation effectively reduces the risk of the model overfitting to the training data.

[0044] Moreover, considering that the vehicle detection model has a fixed-size image input requirement, for example, the target size of the vehicle detection model input of this application is 640*640. The size of the original image selected from the UA-DETRAC dataset may not meet this requirement, so the original image needs to be preprocessed. The preprocessing method mainly includes LetterBox and normalization operations, and the training efficiency and model performance of the model are improved through image preprocessing. Among them, LetterBox is an image preprocessing method that is mainly used to adjust the size of the image while maintaining the original aspect ratio of the image. This method usually involves scaling the image to the longest side of the target size, while padding the short sides to ensure that the entire image meets the size requirements required by the model.

[0045] Specifically, the process of performing LetterBox processing on the original image may include the following steps:

[0046] (1) Calculate the scaling ratio based on the size of the original image and the target size (the target size is the image size required for the vehicle detection model input). For example, the scaling ratio is calculated as shown in formula (1):

[0047]

[0048] It is assumed that the size of the original image is (H, W), the target size is (new_H, new_W), and r represents the scaling ratio.

[0049] (2) Calculate the height and width of the scaled image based on the scaling ratio and the size of the original image. For example, the calculation formulas for the height and width of the scaled image are shown in formula (2) and formula (3):

[0050] new_unpad_H=round(H*r) (2)

[0051] new_unpad_W=round(W*r) (3)

[0052] Among them, new_unpad_H represents the height of the scaled image, H represents the height of the original image, r represents the scaling ratio, new_unpad_W represents the width of the scaled image, and W represents the width of the original image.

[0053] (3) Based on the scaled image height, the scaled image width, and the target size, the difference between the target size and the scaled image size is calculated to obtain the padding height and padding width that still need to be filled. Exemplary calculation formulas for the padding height and padding width are shown in Formula (4) and Formula (5).

[0054] dh=new_H-new_unpad_H (4)

[0055] dw=new_W-new_unpad_W (5)

[0056] Among them, dh represents the padding height that needs to be filled on the image after scaling the original image, and dw represents the padding width that needs to be filled on the image after scaling the original image.

[0057] (4) Based on the padding height and padding width, determine the top padding size, bottom padding size, left padding size, and right padding size. Exemplarily, the padding size calculation formulas are shown in Formula (6), Formula (7), Formula (8), and Formula (9), respectively.

[0058]

[0059] bottom=dh-top (7)

[0060]

[0061] right=dw-left (9)

[0062] Among them, top indicates the top padding size, bottom indicates the bottom padding size, left indicates the left padding size, and right indicates the right padding size.

[0063] Based on formulas (1) to (9), the process of scaling the original image to the target size is as follows: first, the scaling ratio is calculated, and the original image is scaled to the size (new_unpad_H, new_unpad_W) using an image scaling algorithm combined with the scaling ratio; then, padding is performed around the scaled image, with the padding size being (top, bottom, left, right), thereby achieving the goal of adjusting the original image to the target size while maintaining the original aspect ratio. Using LetterBox for image preprocessing reduces information loss and image distortion caused by resizing, which is crucial for maintaining the authenticity of image content. Moreover, by minimizing the padding area, LetterBox mitigates the negative effects of excessive padding, such as reduced model training efficiency.

[0064] Based on the above description, in one possible implementation, after obtaining the original image from the training dataset, the original image is subjected to image preprocessing, namely, LetterBox and normalization processing, to obtain the preprocessed image required for the final training of the vehicle detection model.

[0065] In step 102, the preprocessed image is input into the backbone network of the vehicle detection model for feature extraction to obtain a preliminary extracted feature map. The feature extraction network in the backbone network is a feature extraction network based on a lightweight FasterNet network and a multi-head attention mechanism.

[0066] Among existing vehicle detection algorithms, the YOLOv8 model achieves excellent detection results. Its network architecture primarily consists of a backbone network (Backbone), a neck network (Neck), and a head network (Head). The backbone network begins with the input image and progressively extracts high-level features through a series of convolutional layers and C2f modules. The YOLOv8 backbone network is modeled after the CSPDarkNet-53 architecture, but with several improvements. The most notable is the introduction of the C2f (Cross-convolution with 2 filters) architecture, replacing the C3 architecture in YOLOv5. The C2f architecture enhances model performance by optimizing gradient flow. The neck network utilizes a PAN-FPN architecture similar to YOLOv5, namely the Path Aggregation Network (PANet), which enables cross-scale information transfer and improves the model's multi-scale detection capabilities. The PANet design enables the YOLOv8 model to excel in object detection at various scales, particularly for small objects. The output feature maps of the backbone network undergo multi-scale feature extraction using the SPP (Spatial Pyramid Pooling) architecture, followed by feature fusion using PANet. This architecture enables the YOLOv8 model to more effectively detect objects of varying scales. The head network of the YOLOv8 model employs a decoupled head architecture similar to that of YOLOX, responsible for converting the fused feature maps into the final detection results. The head network separates the regression and classification branches, helping to improve the model's convergence speed and detection performance.

[0067] To address the high computational complexity, poor real-time performance, and high resource consumption of current deep learning-based vehicle detection algorithms in traffic scenarios, this application has partially improved the original YOLOv8 model, ensuring detection accuracy while improving detection speed and timeliness. First, the lightweight FasterNet network replaces the YOLOv8 backbone network CSPDarkNet53, and adds an MHSA layer at the end of the backbone network to fully extract vehicle features. The FasterNet network successfully achieves a balance between speed and accuracy by introducing the innovative technology of partial convolution (PConv). Figure 2The FasterNet network architecture is shown in the figure. The FasterNet network consists of four main layers, with the embedding layer and fusion layer consisting of 4x4 and 2x2 convolutions, respectively, which perform spatial downsampling and expand the number of channels. FasterNetBlock is the primary feature extraction module. Each FasterNet Block consists of a 3x3 PConv layer followed by two 1x1 point-wise convolution (PWConv) layers. Batch normalization and ReLU activation functions are used between these two 1x1 convolutional layers to accelerate training and enhance the model's generalization capabilities. Finally, a residual connection is performed.

[0068] Among them, the PConv module uses the Conv operation to extract features from some channels of the feature map with input dimensions of H*W*C, while other channels remain unchanged. For the continuity of memory access, the front or last continuous C p Channels. When C p When the number of floating-point operations per second (FLOPS) is 1 / 4 that of conventional convolution, PConv achieves only 1 / 16 of the floating-point operations per second (FLOPS). The processed channels are concatenated with the unprocessed channels, and the output is mapped to the same dimension as the original feature map, thus having the same number of channels. By reducing redundant computation and memory access, PConv effectively addresses model overfitting and excessive computational overhead caused by feature redundancy. Figure 3 The convolution structure of the PConv module is given. For a feature map with an input dimension of H*W*C, only the first C p The convolution operation is performed on continuous channels, and the convolution kernel used in the convolution operation is k*k*C p , get the feature map after convolution, and keep the same CC p The channels are spliced together to obtain the feature map after convolution processing by the PConv module.

[0069] The FasterNet feature extraction network, built on PConv, not only effectively captures and refines key spatial feature information, but also fully leverages the computing power of hardware devices, significantly improving detection speed. This optimization strategy not only improves the model's ability to handle large data sets but also reduces processing time, providing strong technical support for vehicle detection in real-time application scenarios. Therefore, replacing the CSPDarkNet53 in the YOLOv8 backbone network with the lightweight FasterNet network based on the original YOLOv8 model allows the improved vehicle detection model to achieve real-time detection performance.

[0070] Compared with the traditional self-attention mechanism, MHSA extends the standard self-attention process by introducing the concept of multiple "heads". MHSA focuses on different positions in the input data and can better capture long-range dependencies and structures in the data. Figure 5 The structural diagram of the multi-head self-attention mechanism is presented. The multi-head self-attention mechanism performs a linear transformation on the input sequence, dividing it into multiple independent attention heads, each of which calculates attention weights. These attention heads independently learn different relationships and feature representations. Finally, their outputs are integrated through concatenation (splicing) or weighted summation to form the final multi-head self-attention output, resulting in a richer representation. Single dot product attention, additive attention, and self-attention mechanisms cannot capture all information from multiple perspectives and granularity. Compared to the three single-head attention mechanisms, the multi-head self-attention mechanism allows different attention heads to learn different relationships and patterns, thereby better capturing diverse information. Multiple attention heads share parameters, reducing the overall number of parameters and the risk of overfitting.

[0071] The number of heads of the multi-head self-attention mechanism used in the original Transformer model is 8, while in some tasks where large models require fine-grained features, such as complex semantic understanding in natural language processing or fine feature capture in image recognition, the number of heads of the multi-head self-attention mechanism is generally 12. In resource-constrained environments, or in application scenarios where the model is lightweight and has low latency, in order to improve model performance without significantly increasing the computational burden, the number of attention heads is usually reduced to 4. This strategy not only helps to reduce the number of model parameters and memory usage, but also significantly improves the operating efficiency of the model to meet real-time requirements. Therefore, the multi-head self-attention mechanism layer in this application has 4 attention heads to ensure higher resource utilization and faster response speed while maintaining model performance.

[0072] This application adds a multi-head self-attention mechanism layer to the last layer of the backbone network to perform global information integration on the features extracted by the backbone network. This application uses four attention heads, each of which is an independent self-attention mechanism that requires learning a set of weights. After multiple self-attention heads are connected, an output is formed as the calculation result of the multi-head self-attention mechanism, thereby improving the expressive power of the model. The calculation formulas are shown in Formulas (10), (11), (12), (13), and (14).

[0073] Q=[q1,q2...q h ]=W Q ·[x1,x2...xh ] (10)

[0074] K=[k1,k2...k h ]=W K ·[x1,x2...x h ] (11)

[0075] V=[v1,v2...v h ]=W V ·[x1,x2...x h ] (12)

[0076]

[0077] M h (Q,K,V)=C(h1,h2...h n )W O (14)

[0078] Among them, Q, K, V, are the input query vector, key vector and value vector respectively, h1,h2…,h n is the output of each self-attention layer, is the attention weight, S is the Softmax function, is the dot product scaling factor, h is the number of heads, and W O 、W K 、W V is a linear transformation, and C represents the concat function connecting multiple attention heads.

[0079] In step 103, the initially extracted feature map is input into the neck network based on the F-C2f module for multi-scale feature fusion to obtain a hybrid fusion feature map. The neck network is a neck network based on the F-C2f module.

[0080] In the neck network of the original YOLOv8 model, the C2f module is a complex structure consisting of a standard convolution (Conv) and multiple Bottleneck blocks. These Bottleneck blocks perform numerous skip-layer connections and split operations, resulting in a complex network structure and high computational complexity, making it unable to meet the real-time requirements of vehicle detection. Therefore, in addition to improving the original YOLOv8 model's backbone network, the C2f module in the neck network was also improved. The split operations and redundant skip-layer connections in the original C2f module were removed, the large number of 3*3 ordinary convolutions in the C2f module was reduced, and all Bottleneck blocks in the original network's C2f module were replaced with Fasterblocks, forming the new F-C2f module. While reducing the number of model parameters, the F-C2f module enables the model to fully and effectively utilize information from all channels and maintain the diversity of features extracted by the model. Figure 4 The structural diagram of the F-C2f module is given. The F-C2f module consists of CBS, Split operation, connection layer and F-Bottleneck, which is mainly composed of convolutional layer and Fasterblock layer in FasterNet network.

[0081] The embodiment of the present application uses the idea of partial convolution to improve the C2f module. Specifically, the Split operation and redundant skip-layer connections in the original C2f structure are deleted, a large number of 3*3 ordinary convolutions in C2f are reduced, and the Bottleneck layer in the original network C2f module is completely replaced with Fasterblock to form a new F-C2f module, thereby obtaining the vehicle detection model provided by the present application.

[0082] In step 104, the hybrid fusion feature map is input into the head network of the vehicle detection model, and the Focal-EIoU loss function is used for training. When the training cycle reaches 100, the training is stopped to obtain a trained vehicle detection model.

[0083] In order to further improve the target detection effect of the model, this application also improves the loss function of the original YOLOv8 model and adopts the Focal-EIoU loss function as the loss function of the vehicle detection model. This improves the position accuracy of the vehicle detection model in the bounding box regression stage and accelerates the convergence speed of the model. When detecting complex traffic scenes, it can not only effectively solve the category imbalance problem, but also ensure the accuracy of the bounding box, and improve the performance and robustness of the vehicle detection algorithm. For example, the calculation formula of the Focal-EIoU loss function is shown in formula (15):

[0084] L (Focal-EIoU) =IoU γ L EIoU (15)

[0085] Among them, IoU is the intersection over union ratio; γ is a hyperparameter that controls the degree of suppression of negative samples. When the background is extremely unbalanced, increasing the γ coefficient can improve the performance of the loss function. EIoU Represents EIoU loss. EIoU loss effectively solves the shortcomings of CIoU (Complete-IoU) loss function when considering the simultaneous increase or decrease of width and height by introducing a loss penalty for aspect ratio, ensuring that the width and height difference between the real box and the predicted box is minimized, and achieving rapid convergence of the loss function. EIoU loss function not only improves the accuracy and training efficiency of vehicle detection, but also enhances the robustness of the model to complex scenes and various vehicle detections. EIoU loss includes IoU loss L IoU , distance loss L dis and aspect ratio loss Lasp , the calculation formula of EIoU loss is shown in formula (16):

[0086] L EIoU =L IoU +L dis +L asp (16)

[0087] Among them, L IoU Represents the IoU loss, which is usually used to measure the similarity between the predicted bounding box and the true bounding box. The calculation formula of the IoU loss can be shown as formula (17):

[0088]

[0089] In formula (17), IOU is defined as the ratio of the intersection area of the predicted box and the labeled box (or the true box) to the union area, which ranges from [0, 1].

[0090] L dis represents the distance loss, which uses the square of the Euclidean distance between the center point of the predicted box and the labeled box, and normalizes it by the square of the diagonal length of the minimum circumscribed rectangle of the two to enhance the accuracy of the model in locating the predicted box position; L asp Represents the aspect ratio loss, which focuses on minimizing the deviation between the predicted box and the annotated box in the width and height dimensions, so that the model can learn a more accurate bounding box shape. The calculation formula of the distance loss can be shown in formula (18), and the calculation formula of the aspect ratio loss can be shown in formula (19).

[0091]

[0092] Among them, ω c and h c The width and height of the minimum bounding box covering the prediction box and the annotation box are b and b respectively. gt are the center points of the prediction box and the annotation box, ω, h, ω gt With h gt are the width and height of the prediction box and the annotation box respectively, ρ 2 Represents the distance squared function. Please refer to Figure 6 , Figure 6 A parameter diagram of EIoU loss is given. By decomposing the loss function into IoU loss, distance loss, and aspect ratio loss, L EIoU It can more comprehensively evaluate the geometric differences between the predicted box and the annotated box, and directly optimize the size of the predicted box to make it closer to the annotated box, thereby achieving faster convergence speed and more accurate positioning effect.

[0093] In summary, as shown in formulas (15) to (19), the parameters required for the calculation of the loss function of the vehicle detection model provided by this application include: the first center point, first width, and first height of the prediction box (the prediction box is obtained by inputting the hybrid fusion feature network into the head network of the vehicle detection model and performing prediction); the second center point, second width, and second height of the annotation box (the annotation box is the annotation data of the preprocessed image or the real box); the third width and third height of the minimum external frame corresponding to the prediction box and the annotation box; the intersection area and union area of the prediction box and the annotation box, etc. After obtaining the above parameters, the first center point, second center point, first width, second width, first height, second height, third width, third height, intersection area, and union area can be brought into the above loss function calculation formula to determine the prediction regression loss of the vehicle detection model.

[0094] In combination with formula (15) to formula (19), the specific process of determining the predicted regression loss based on the above parameters can also include the following steps, that is, step 104 can also include steps 104A to 104E.

[0095] In step 104A, the ratio of the intersection area to the union area is determined as the intersection-to-union ratio loss.

[0096] First, the intersection-over-union loss, i.e., the IOU in formula (15), is determined based on the ratio of the intersection area and the union area of the predicted box and the labeled box.

[0097] Step 104B: Determine the similarity loss between the predicted box and the labeled box based on the intersection area, the union area, and a preset value.

[0098] The preset value is 1. Combined with formula (17), the similarity loss between the predicted box and the labeled box can be calculated based on the intersection area, union area and the preset value.

[0099] Step 104C: Determine the distance loss between the predicted box and the labeled box based on the first center point, the second center point, the third width, and the third height.

[0100] Combined with formula (18), the distance loss between the predicted box and the annotated box can be calculated based on the first center point of the predicted box, the second center point of the annotated box, the third width and the third height of the minimum bounding box corresponding to the predicted box and the annotated box.

[0101] Step 104D: Determine the aspect ratio loss between the predicted box and the labeled box based on the first width, the second width, the first height, the second height, the third width, and the third height.

[0102] Combined with formula (19), the aspect ratio loss between the predicted box and the annotated box is calculated based on the first width of the predicted box, the second width of the annotated box, the first height of the predicted box, the second height of the annotated box, the third width and the third height of the minimum bounding box corresponding to the predicted box and the annotated box.

[0103] Step 104E: Determine the prediction regression loss of the vehicle detection model based on the intersection-over-union loss, the similarity loss, the distance loss, and the aspect ratio loss.

[0104] Then, the prediction regression loss of the vehicle detection model is determined based on the intersection-over-union loss, similarity loss, distance loss, and aspect ratio loss.

[0105] As shown in formula (15) and formula (16), there is also a certain operational relationship between the intersection-over-union loss, similarity loss, distance loss, aspect ratio loss and the final prediction regression loss. Optionally, step 104E may also include steps 104E1 to 104E3.

[0106] Step 104E1: Determine the first part of the loss based on the intersection-over-union loss and preset hyperparameters.

[0107] In step 104E2, the sum of the similarity loss, the distance loss, and the aspect ratio loss is determined as the second part of the loss.

[0108] Step 104E3: The product of the first part of the loss and the second part of the loss is determined as the prediction regression loss of the vehicle detection model.

[0109] First, the first part of the loss is determined based on the intersection-over-union loss and the preset hyperparameters. Then, the sum of the similarity loss, distance loss, and aspect ratio loss is determined as the second part of the loss (i.e., EIoU loss). Finally, the product of the first and second part of the loss is determined as the prediction regression loss of the vehicle detection model.

[0110] The Focal-EIOU loss function introduced in the embodiment of the present application separates high-quality anchor boxes from low-quality anchor boxes from a gradient perspective, optimizes the sample imbalance problem in the bounding box regression task, focuses the regression process on high-quality anchor boxes, and improves the regression accuracy.

[0111] When calculating the loss function, it involves calculating the position of the prediction box. For example, the process of calculating the position of the prediction box can be as follows:

[0112] (1) Assuming that the coordinate offset of the center point of the bounding box output by the model is (dx, dy) and the coordinates of the reference point are (cx, cy), the calculation formula of the coordinates of the center point of the bounding box can be shown as shown in formula (20) and formula (21):

[0113]

[0114] Among them, anchor_width and anchor_heigth represent the width and height of the prediction box respectively, and scale_factor is the scaling ratio, which is used to convert the offset output by the model into absolute coordinates relative to the input image.

[0115] (2) Assume that the width and height scaling factors of the model output are (dw, dh), and the width and height of the preset box are (anchor width , anchor_height), the actual height and width of the prediction box can be calculated as shown in formula (22) and formula (23):

[0116] bw=anchor_width·exp(dw) (22)

[0117] bh=anchor_height·exp(dh) (23)

[0118] (3) Based on the known coordinates of the center point (bx, by), width (bw), and height (bh) of the prediction box, the coordinates of the four vertices of the prediction box can be calculated. The coordinates of the upper left corner of the prediction box are (bx-bw / 2, by-bh / 2), and the coordinates of the lower right corner are (bx+bw / 2, by+bh / 2).

[0119] In addition to outputting the prediction box, the vehicle detection model also outputs the category to which the vehicle target belongs. In one possible embodiment, the Cls branch of the vehicle detection model is used to predict the category of the detection target (i.e., the vehicle). After the image data passes through the network, the Cls branch outputs a vector with a dimension of (1, nc, 8400), where nc represents the number of categories and 8400 represents the total number of grid cells predicted on the feature maps of three different scales. Each element of the vector is processed by the Sigmoid function, which maps each element to the interval (0, 1) to represent the confidence of the corresponding category. The Sigmoid function calculation formula can be shown as formula (24).

[0120]

[0121] This processing method can limit the output to a probability range, maintain the independence between category predictions, and introduce nonlinear relationships. The output of the Sigmoid function is not only easy to interpret as a probability value, but also facilitates gradient calculation and nonlinear activation, improving the generalization ability of the model.

[0122] After obtaining the preliminary prediction results of the model, it is necessary to use the non-maximum suppression (NMS) module to process the output prediction box. The main purpose of NMS processing is to filter out highly repeated detection boxes that may point to the same target, and only retain the detection boxes with the highest confidence. The NMS module filters out low-confidence prediction results based on the preset confidence threshold, and converts the remaining prediction results from XYWH format to XYXY format to more intuitively represent the coordinates of the bounding box. The converted prediction results are processed using the NMS algorithm. By calculating the intersection-over-union ratio between the bounding boxes, overlapping prediction boxes are removed, and the optimal prediction results for each vehicle target are retained.

[0123] After obtaining the NMS results, they need to be processed using the Scale_boxes module. The Scale_boxes module scales the NMS predictions back to the size of the original input image. During this process, the predicted boxes are offset-corrected to ensure the accuracy of the bounding boxes, taking into account the gray borders of the letterbox padding. The corrected prediction boxes are then cropped to prevent them from exceeding the actual image size. The steps for cropping the prediction boxes include: determining the actual size of the original input image, namely the image width W and height H; calculating the cropping range to ensure that the top-left corner coordinates (x1, y1) of the prediction box are no less than (0, 0) and the bottom-right corner coordinates (x2, y2) are no greater than (W, H), to ensure they are within the actual image size; and checking each corrected prediction box. If its coordinates exceed the image boundaries, they are adjusted based on the image size. For example, if x1 is less than 0, it is set to 0, and if x2 is greater than W, it is set to W. Similarly, the prediction boxes for y1 and y2 are processed similarly. After these adjustments, the resulting prediction box is the cropped result.

[0124] Step 105: Input the test image into the trained vehicle detection model to obtain a vehicle detection result of the test image.

[0125] Exemplarily, the model training environment of the present application is: running under Windows 10 operating system, CPU 1GHz, memory 16G, hard disk 1T, graphics card NVIDIA GTX1070Ti, programming language is Python 3.8, and model training is performed in the deep learning framework Pytorch 1.2.1 version. In the process of training the vehicle detection model in the present application, a total of 100 training cycles (epochs) are set to ensure that the vehicle detection model is fully converged. The number of images processed in a single time (batch size) is set to 8 to balance computational efficiency and memory usage; the number of threads (works) used by the CPU during data loading is 8; the learning rate is 0.01; the loss function selects Focal-EIoU Loss; in terms of optimization algorithm, considering the demand for large-scale data processing and the gradient update characteristics during training, the present application adopts stochastic gradient descent (SGD) to train and optimize the neural network (i.e., vehicle detection model). SGD updates the model parameters by sample by sample or small batch samples instead of using the entire data set. This method significantly improves computational efficiency. The training parameters of the vehicle detection model are shown in Table 1.

[0126] Table 1 Training parameters of vehicle detection model

[0127]

[0128]

[0129] In summary, the embodiments of the present application provide a vehicle detection algorithm based on the FasterNet lightweight framework and multi-head self-attention mechanism. The lightweight FasterNet network replaces the YOLOv8 backbone network CSPDarkNet53, reconstructs the C2f module, reduces the complexity of the model, realizes the lightweight model, and improves the speed of vehicle detection. By introducing the multi-head self-attention mechanism, the model's ability to understand complex spaces is enhanced. The Focal-EIoU Loss loss function is used to enhance the detection accuracy of multiple vehicle targets and small vehicle targets in complex traffic road scenes, thereby improving the accuracy of vehicle detection.

[0130] To verify the vehicle detection performance of the vehicle detection model (vehicle detection algorithm) proposed in this application in traffic scenarios, this application uses six commonly used evaluation indicators in target detection to comprehensively evaluate the vehicle detection model. These indicators are precision, recall, mean average precision (mAP), gigaflops (GFLOPs), frames per second (FPS), and model parameter count (Params).

[0131] Precision and recall directly reflect a model's vehicle detection capabilities and are fundamental metrics for evaluating model performance. mAP, a comprehensive measure of precision and recall, more comprehensively reflects a model's overall performance and is a widely used evaluation metric in the field of object detection. GFLOPs, FPS, and Params quantitatively evaluate models based on computational complexity, processing speed, and model size, providing important insights for practical application. The six objective evaluation metrics used in this application comprehensively and effectively evaluate the performance of vehicle detection algorithms from multiple dimensions, providing a reference for further optimization and practical application.

[0132] Precision refers to the proportion of samples that are actually positive among the samples predicted by the model. Recall refers to the model's ability to find all positive samples. A larger precision value indicates a lower false positive rate. A larger recall value indicates a lower missed detection rate. The calculation formulas for precision and recall are shown in Formula (25) and Formula (26), respectively. TP represents the number of correctly detected samples, FP represents the number of incorrectly detected samples, and FN represents the number of missed detection samples.

[0133]

[0134] The curve drawn with recall as the horizontal axis and precision as the vertical axis is the PR curve. The integrated average value below the curve is the AP value of the category, and its calculation formula is shown in formula (27).

[0135]

[0136] mAP is the sum of the average precision of all labels divided by the total number of all categories. The larger the value of average precision, the better the overall performance of the model. Its calculation formula is shown in formula (28).

[0137]

[0138] mAP@0.5 (Mean Average Precision at Intersection over Union = 0.50) measures the performance of the object detection model by calculating the average precision across all detection categories under the condition that the intersection over union ratio is set to a threshold of 0.5. The intersection over union ratio threshold of 0.5 means that a prediction is considered a correct detection only when the intersection area of the predicted bounding box and the ground-truth bounding box accounts for 50% or more of their union area. Higher mAP@0.5 values indicate that the model is able to more accurately detect vehicles during the detection process, while maintaining low false detections (incorrectly detecting non-vehicle objects as vehicles) and missed detections (failing to detect actual vehicles).

[0139] GFLOPs refers to the theoretical number of billion floating-point operations per second (GFLOPs) required by a model during forward propagation or backward propagation. It is often used to evaluate the theoretical computational effort required by a model during forward reasoning or backward optimization. A lower GFLOPs value means that the model requires fewer floating-point operations per unit time during execution, demonstrating its computational efficiency advantage. This advantage is reflected not only in processing speed but also in its resource-friendliness. Given the same hardware conditions, low GFLOPs models can be more easily deployed on various platforms, such as embedded systems or edge computing devices.

[0140] FPS refers to the number of still image frames per second that the model can efficiently process and continuously output detection results. This indicator is directly related to the model's real-time response capability, that is, the speed at which the model processes data and immediately feedbacks the results. The higher the frame rate, the more image frames the model can process per second, the stronger the real-time response capability, and the model can be seamlessly compatible and efficiently integrated into various real-time systems, ensuring the overall smoothness and immediacy of the system. The FPS calculation formula is shown in Formula (29), where N(p) represents the total number of images processed and T(p) represents the time it takes to process the image.

[0141]

[0142] The number of model parameters, a key indicator of model complexity, reflects the total number of adjustable weight parameters in the model. The number of model parameters is directly related to the model's structural complexity and its demand for computing resources. Models with fewer parameters are more compact, reducing the computational burden during training and the hardware resource consumption required for prediction, thereby improving vehicle detection efficiency.

[0143] In order to verify the advancement and effectiveness of the algorithm proposed in this application, the algorithm proposed in this patent (vehicle detection model) was compared with the EfficientDet model, D-FINE model and YOLOv11 model. The performance comparison results are shown in Table 2.

[0144] Table 2 Performance comparison results of different methods

[0145]

[0146]

[0147] In terms of accuracy, the D-FINE model ranked first with 84.61%, achieving the highest accuracy in vehicle detection. YOLOv11 and EfficientDet followed closely behind, achieving 83.62% and 83.54%, respectively, both showing high accuracy. The algorithm proposed in this application, while slightly lower than the top three, also demonstrated high accuracy with an accuracy of 81.57%.

[0148] In terms of recall, D-FINE maintained its lead, reaching 85.28%, demonstrating its comprehensiveness and low missed detection rate in vehicle detection. In comparison, YOLOv11 achieved the lowest recall rate, at 80.59%, indicating that it may miss detections in certain complex scenarios. EfficientDet and the proposed algorithm maintained stable recall rates of 82.31% and 81.37%, respectively.

[0149] mAP@0.5 is an important indicator for measuring the overall performance of object detection algorithms. D-FINE achieved 84.57%, maintaining its leading position. The algorithm proposed in this application achieved 83.81% mAP@0.5, second only to D-FINE.

[0150] The EfficientDet model maintains a low level in both GFLOPs and parameter count, indicating that it has a good balance between computational complexity and model size, and is suitable for scenarios with certain requirements on computing resources. The D-FINE model has the highest computational complexity and parameter count, indicating that while D-FINE pursues higher performance, it also increases computational complexity and model size. The YOLOv11 model has a similar number of parameters to EfficientDet, but has higher GFLOPs, indicating YOLOv11's advantages in speed and accuracy, and is suitable for scenarios with high requirements on computing resources. The algorithm proposed in this application performs well in GFLOPs, slightly higher than EfficientDet, but far lower than D-FINE and YOLOv11, and has the lowest number of parameters. This shows that while achieving similar performance, the algorithm significantly reduces the model's computational complexity and model size, making it more practical on resource-constrained devices.

[0151] Overall, the algorithm proposed in this application is slightly lower than the D-FINE model in terms of precision, recall and average precision. This is mainly because the D-FINE model introduces FDR to refine the distribution representation of the bounding box, and transfers positioning knowledge from the deep layer to the shallow layer through GO-LSD to achieve accurate positioning of the target. Compared with the YOLOv11 model, although the accuracy and average accuracy indicators of the algorithm proposed in this application are slightly reduced, the recall rate indicator is improved, and the GFLOPs and parameter quantity indicators are significantly reduced. This shows that although the algorithm proposed in this application has some reductions in individual accuracy indicators, it has advantages in comprehensive performance, real-time processing capabilities and resource efficiency. After the vehicle detection model training is completed, in actual traffic scenarios, the real-time images captured can be input into the vehicle detection model to output real-time detection results of the vehicle.

[0152] The EfficientDet model outperforms the algorithm proposed in this application in terms of accuracy, recall, and GFLOPs. However, the EfficientDet model has a large number of parameters and takes a relatively long time to train, which does not meet the real-time requirements of practical applications. Overall, the GFLOPs value of the algorithm proposed in this application is slightly higher than that of the EfficientDet model, but much lower than the D-FINE model and YOLOv11 model. In addition, the number of model parameters is significantly reduced to 4.78, which is much lower than the other three models. This shows that the algorithm proposed in this application achieves model lightweighting while maintaining high performance.

[0153] This application conducted experiments on the proposed algorithm, selecting multiple images from the UA-DETRAC dataset for vehicle detection testing. The following uses an image from the UA-DETRAC dataset as an example for illustration. Figure 8 The experimental results of vehicle detection for this image using the algorithm proposed in this application, the EfficientDet model, the D-FINE model, and the YOLOv11 model are given. Figure 8 It can be seen that all four methods can detect vehicles on urban main roads and mark the vehicle locations and detection probabilities.

[0154] Depend on Figure 8As can be seen, the algorithm proposed in this application uses a red border to mark the location of the detected vehicle and gives the detection probability. The detected vehicle bounding box has a high accuracy rate, with few false detections, and can accurately detect vehicles of different colors and types, improving the detection accuracy of multiple vehicle targets and small vehicle targets in complex traffic scenes. This is mainly due to the algorithm's optimization in using the FasterNet backbone network to perform preliminary feature extraction on preprocessed images, using the neck network based on the F-C2f module for multi-scale feature fusion, and using the Focal-EIoU loss function to calculate the bounding box prediction results.

[0155] The EfficientDet model uses green borders to mark the locations of detected vehicles and provides a detection probability. While it performs well in vehicle detection, it does have some issues with repeated detections of the same vehicle, which may be related to factors such as the model structure and the feature fusion network.

[0156] The D-FINE model uses purple borders to mark the locations of detected vehicles and provides a detection probability. The vehicle detection results show a relatively high number of missed detections and false detections. This may be due to the model's limited generalization ability, which prevents it from accurately detecting vehicles in certain scenarios.

[0157] The YOLOv11 model uses a blue border to mark the location of detected vehicles and provides a detection probability. It can detect larger vehicles well, but performs poorly when detecting smaller, more distant vehicles. This may be due to the limited feature information the model can extract when processing small vehicles, making it more difficult for the model to accurately detect them.

[0158] Please refer to Figure 9 , which is a structural diagram of a vehicle detection device based on the FasterNet lightweight framework and feature fusion provided in an embodiment of the present application. For example, Figure 9 As shown, the apparatus 900 includes:

[0159] The preprocessing module 901 is used to input an image and perform preprocessing on the image, wherein the preprocessing mainly includes LetterBox and normalization operations;

[0160] Feature extraction module 902, which inputs the preprocessed image into the backbone network of the vehicle detection model for feature extraction to obtain a preliminary extracted feature map. The feature extraction network in the backbone network is a feature extraction network based on a lightweight FasterNet network and a multi-head attention mechanism;

[0161] The feature fusion module 903 inputs the initially extracted feature map into the neck network based on the F-C2f module for multi-scale feature fusion to obtain a hybrid fusion feature map. The neck network is a neck network based on the F-C2f module.

[0162] The model training module 904 inputs the hybrid fusion feature map into the head network of the vehicle detection model and uses the Focal-EIoU loss function for training. When the training cycle reaches the set value, the model training stops. In the process of training the vehicle detection model in this application, a total of 100 training cycles (epochs) are set to ensure that the vehicle detection model is fully converged;

[0163] The model testing module 905 is used to input the test image into the trained vehicle detection model to obtain the vehicle detection result of the test image.

[0164] Optionally, the backbone network includes the FasterNet network, and the multi-head self-attention mechanism layer has 4 attention heads.

[0165] Optionally, the model training module 904 is further configured to:

[0166] Inputting the hybrid fusion feature map into the head network of the vehicle detection model to obtain a prediction box;

[0167] Obtaining a first center point, a first width, and a first height of the prediction box, and a second center point, a second width, and a second height of the annotation box of the preprocessed image;

[0168] Determining a third width and a third height of the minimum bounding boxes corresponding to the prediction box and the annotation box;

[0169] Determine the intersection area and the union area of the prediction box and the annotation box;

[0170] Determining a prediction regression loss of the vehicle detection model based on the first center point, the second center point, the first width, the second width, the first height, the second height, the third width, the third height, the intersection area, and the union area;

[0171] The vehicle detection model is trained based on the prediction regression loss.

[0172] Optionally, the model training module 904 is further configured to:

[0173] Determine the ratio of the intersection area to the union area as the intersection-to-union ratio loss;

[0174] Determining a similarity loss between the predicted box and the labeled box based on the intersection area, the union area, and a preset value;

[0175] Determining a distance loss between the predicted box and the labeled box based on the first center point, the second center point, the third width, and the third height;

[0176] determining an aspect ratio loss between the predicted box and the labeled box based on the first width, the second width, the first height, the second height, the third width, and the third height;

[0177] The prediction regression loss of the vehicle detection model is determined based on the intersection-over-union loss, the similarity loss, the distance loss, and the aspect ratio loss.

[0178] Optionally, the model training module 904 is further configured to:

[0179] Determine the first part of the loss based on the intersection-over-union loss and preset hyperparameters;

[0180] Determine the sum of the similarity loss, the distance loss, and the aspect ratio loss as a second part of the loss;

[0181] The product of the first part of the loss and the second part of the loss is determined as the predicted regression loss of the vehicle detection model.

[0182] Optionally, the device further comprises:

[0183] A second acquisition module is used to acquire a preprocessed target input image, wherein the target input image includes a target vehicle to be detected;

[0184] a target detection module, configured to input the target input image into a trained vehicle detection model to obtain at least one initial detection frame output by the vehicle detection model;

[0185] The target processing module is used to perform non-maximum suppression processing on the initial detection frame to obtain a target detection frame corresponding to the target input image.

[0186] The exemplary embodiments of the present application also provide an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when executed by the at least one processor, the computer program is used to cause the electronic device to execute the vehicle detection algorithm and apparatus based on the FasterNet lightweight framework and feature fusion according to the embodiments of the present application. It should be noted that this electronic device is the device end shown in the above embodiments.

[0187] An exemplary embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to enable the computer to execute a vehicle detection algorithm and device based on the FasterNet lightweight framework and feature fusion according to an embodiment of the present application.

[0188] An exemplary embodiment of the present application also provides a computer program product, including a computer program, wherein, when the computer program is executed by a processor of a computer, it is used to enable the computer to execute the vehicle detection algorithm and device based on the FasterNet lightweight framework and feature fusion according to the embodiment of the present application.

[0189] refer to Figure 10 The structural block diagram of the electronic device 1000 that can be used as the server or client of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0190] like Figure 10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0191] Multiple components within electronic device 1000 are connected to I / O interface 1005, including an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. Input unit 1006 can be any type of device capable of inputting information into electronic device 1000. Input unit 1006 can receive input numeric or character information and generate key input signals related to user settings and / or function control of the electronic device. Output unit 1007 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 1008 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0192] Optionally, the electronic device 1000 is further provided with a single-channel EEG signal acquisition module (not shown in the figure) for acquiring EEG signals and transmitting the EEG signals to a signal processor of the electronic device 1000 for EEG signal processing.

[0193] The computing unit 1001 may be a variety of general and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above. For example, in an embodiment, Figure 1 The illustrated method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via the ROM 1002 and / or the communication unit 1009. In some embodiments, the computing unit 1001 may be configured to execute the computer program in any other suitable manner (e.g., by means of firmware). Figure 1 The method shown.

[0194] The program code for implementing the algorithms of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0195] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0196] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0197] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0198] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0199] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

Claims

1. A vehicle detection algorithm based on the FasterNet lightweight framework and feature fusion, characterized by: The algorithm includes: Input an image and perform preprocessing on the image, wherein the preprocessing mainly includes LetterBox and normalization operations; The preprocessed image is input into the backbone network of the vehicle detection model for feature extraction to obtain a preliminary extracted feature map. The feature extraction network in the backbone network is a feature extraction network based on a lightweight FasterNet network and a multi-head attention mechanism; Input the initially extracted feature map into the neck network for multi-scale feature fusion to obtain a hybrid fusion feature map. The neck network is a neck network based on the F-C2f module. The F-C2f module consists of CBS, Split operation, connection layer and F-Bottleneck. By deleting the Split operation and redundant skip layer connections in the original C2f module, reducing a large number of 3*3 ordinary convolutions in C2f, and replacing all Bottleneck in the original C2f module with Fasterblock, a new F-C2f module is obtained; Input the hybrid fusion feature map into the head network of the vehicle detection model, use the Focal-EIoU loss function for training, and stop training when the training cycle reaches 100 to obtain a trained vehicle detection model; Input the test image into the trained vehicle detection model to obtain the vehicle detection result of the test image.

2. The algorithm according to claim 1, characterized in that The backbone network includes the FasterNet network, and the multi-head self-attention mechanism layer has 4 attention heads.

3. The algorithm according to claim 1, characterized in that Inputting the hybrid fusion feature map into the head network of the vehicle detection model and training using the Focal-EIoU loss function includes: Inputting the hybrid fusion feature map into the head network of the vehicle detection model to obtain a prediction box; Obtaining a first center point, a first width, and a first height of the prediction box, and a second center point, a second width, and a second height of the annotation box of the preprocessed image; Determining a third width and a third height of the minimum bounding boxes corresponding to the prediction box and the annotation box; Determine the intersection area and the union area of the prediction box and the annotation box; Determining a prediction regression loss of the vehicle detection model based on the first center point, the second center point, the first width, the second width, the first height, the second height, the third width, the third height, the intersection area, and the union area; The vehicle detection model is trained based on the prediction regression loss.

4. The algorithm according to claim 3, characterized in that The determining the prediction regression loss of the vehicle detection model based on the first center point, the second center point, the first width, the second width, the first height, the second height, the third width, the third height, the intersection area, and the union area includes: Determine the ratio of the intersection area to the union area as the intersection-to-union ratio loss; Determining a similarity loss between the predicted box and the labeled box based on the intersection area, the union area, and a preset value; Determining a distance loss between the predicted box and the labeled box based on the first center point, the second center point, the third width, and the third height; determining an aspect ratio loss between the predicted box and the labeled box based on the first width, the second width, the first height, the second height, the third width, and the third height; The prediction regression loss of the vehicle detection model is determined based on the intersection-over-union loss, the similarity loss, the distance loss, and the aspect ratio loss.

5. The algorithm according to claim 4, characterized in that The determining the prediction regression loss of the vehicle detection model based on the intersection-over-union loss, the similarity loss, the distance loss, and the aspect ratio loss includes: Determine the first part of the loss based on the intersection-over-union loss and preset hyperparameters; Determine the sum of the similarity loss, the distance loss, and the aspect ratio loss as a second part of the loss; The product of the first part of the loss and the second part of the loss is determined as the predicted regression loss of the vehicle detection model.

6. The algorithm according to claim 1, characterized in that The algorithm also includes: Acquire a preprocessed target input image, wherein the target input image includes a target vehicle to be detected; Inputting the target input image into a trained vehicle detection model to obtain at least one initial detection frame output by the vehicle detection model; Non-maximum suppression processing is performed on the initial detection frame to obtain a target detection frame corresponding to the target input image.

7. A vehicle detection device based on the FasterNet lightweight framework and feature fusion, characterized in that: The device comprises: A preprocessing module is used to input an image and perform preprocessing on the image, wherein the preprocessing mainly includes LetterBox and normalization operations; A feature extraction module is used to input the preprocessed image into the backbone network of the vehicle detection model for feature extraction to obtain a preliminary extracted feature map. The feature extraction network in the backbone network is a feature extraction network based on a lightweight FasterNet network and a multi-head attention mechanism; A feature fusion module is used to input the initially extracted feature map into the neck network based on the F-C2f module for multi-scale feature fusion to obtain a hybrid fusion feature map. The neck network is a neck network based on the F-C2f module. The F-C2f module consists of CBS, Split operation, connection layer and F-Bottleneck. By deleting the Split operation and redundant skip layer connections in the original C2f module, reducing a large number of 3*3 ordinary convolutions in C2f, and replacing all Bottlenecks in the original C2f module with Fasterblock, a new F-C2f module is obtained; A model training module is used to input the hybrid fusion feature map into the head network of the vehicle detection model and use the Focal-EIoU loss function for training. When the training cycle reaches a set value, the model stops training. In the process of training the vehicle detection model in this application, a total of 100 training cycles (epochs) are set to ensure that the vehicle detection model is fully converged; The model prediction module is used to input the test image into the trained vehicle detection model to obtain the vehicle detection result of the test image.

8. An electronic device comprising: processor; as well as Memory for storing programs, The program includes instructions that, when executed by the processor, cause the processor to execute a vehicle detection algorithm based on the FasterNet lightweight framework and feature fusion according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the vehicle detection algorithm based on the FasterNet lightweight framework and feature fusion according to any one of claims 1 to 6.

Images