Global sensing method and system for vehicle detection

Abstract

The invention discloses a global perception method and system for vehicle detection. The method comprises the following steps: acquiring a vehicle picture and dividing the vehicle picture into a plurality of feature blocks; inputting each feature block into an attention benchmark network based on a cascade mode to obtain multi-scale feature maps of multiple stages; performing cross fusion and noise suppression on the multi-scale feature maps of the multiple stages to obtain fusion feature information; and outputting a vehicle detection frame, a vehicle category and a vehicle mask through floating point calculation and bilinear interpolation based on the fused feature information and the multi-scale feature map. Rich vehicle feature information is provided by obtaining a multi-scale feature map containing a vehicle image and fusion feature information, and then vehicle detection frame positioning and vehicle classification are performed based on floating point calculation and bilinear interpolation, so that high-precision vehicle detection and classification can be performed by greatly utilizing global vehicle feature information. And the detection and classification reliability and stability are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence target detection and target classification, and in particular to a global perception method and system for vehicle detection. Background Art

[0002] Early vehicle detection methods relied on background subtraction and image template matching, but these methods had low accuracy and speed due to background variations and noise. Later, researchers began to use artificial features such as shape context, Haar-like features, HOG, and SIFT for vehicle detection. These methods perform well when dealing with illumination changes and vehicle occlusion, but are complex and time-consuming due to the need to manually extract features. Based on the shortcomings of early methods, a domain-adaptive Faster R-CNN method is proposed for vehicle detection in various weather conditions such as sunny, cloudy, and snowy days. This method improves the detection accuracy of Faster R-CNN in complex environments, but it is more suitable for orderly traffic scenes rather than chaotic and disordered traffic. In contrast, in order to adapt to disordered traffic environments, a small sample multi-detection box detector (SSMD) is designed to achieve a balance between accuracy and speed. However, its accuracy is significantly lower when dealing with vehicles of different sizes. The vehicle detection method of the YOLOv3 algorithm uses anti-residual blocks and non-maximum suppression (NMS) to solve the problem of complex and variable vehicle features in vehicle detection. Although its running speed is commendable, it is limited to vehicle detection. Its detection performance is poor in complex traffic environments, such as vehicle occlusion and changeable weather conditions, making it unsuitable for vehicle detection under irregular traffic conditions. To this end, we improve YOLOv3 and propose a real-time vehicle detection technology for congested metropolises. The YOLOv3 algorithm detects vehicles from the front view to provide valuable data for autonomous driving. This method can identify vehicles from the front, but it is difficult to detect occluded vehicles. Despite its amazing speed, the accuracy is greatly reduced. Scientists have developed a non-hierarchical method backbone network (ViTDeT) to perform detection tasks. The backbone network can be used as a simple feature pyramid network with a window attention mechanism. SwinNet is a cross-modal fusion model based on Swin-Transformer for object detection. The model consists of a Swin-Transformer as a baseline network to extract discriminative hierarchical features, supplemented by an attention mechanism to bridge the gap between the two modalities, and finally uses edge information to highlight the detected targets. Although the above model can obtain more accurate detection results through its own algorithm optimization, there are still few small target pixels in low-resolution images, making it difficult to accurately capture their features, resulting in low accuracy in the final vehicle detection, positioning and classification results, reducing its practicality. Summary of the invention

[0003] In response to the above-mentioned problems, the present invention provides a global perception method and system for vehicle detection to solve the problem that the small target pixels in the resolution image collected by the model detection method based on the attention mechanism mentioned in the background technology are still relatively small, making it difficult to accurately capture the global features of the vehicle, resulting in the final vehicle detection positioning and classification results being of low accuracy, thereby reducing practicality.

[0004] A global perception method for vehicle detection comprises the following steps:

[0005] Obtain a vehicle image and divide it into multiple feature blocks;

[0006] Each feature block is input into the cascade-based attention baseline network to obtain multi-scale feature maps at multiple stages;

[0007] Cross-fuse and suppress noise on multi-scale feature maps from multiple stages to obtain fused feature information;

[0008] Based on the fusion feature information and multi-scale feature maps, the vehicle detection frame, vehicle category and vehicle mask are output through floating-point calculation and bilinear interpolation.

[0009] Preferably, the obtaining of the vehicle image and dividing it into a plurality of feature blocks comprises:

[0010] Obtain a vehicle image and convert it into an RGB three-channel image, divide the RGB three-channel image into multiple grids, and obtain feature blocks in each grid;

[0011] The step of inputting each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages includes:

[0012] The feature blocks are converted into a one-dimensional vector sequence and fed into the cascade-based attention baseline network;

[0013] The window processing mechanism of the attention baseline network is used to fuse the features of multiple one-dimensional vector sequences to obtain multi-scale feature maps at multiple stages.

[0014] Preferably, the step of performing feature fusion on multiple one-dimensional vector sequences through a window processing mechanism of an attention baseline network to obtain multi-scale feature maps of multiple stages includes:

[0015] Define each one-dimensional vector sequence, perform feature fusion on the defined sequence and the result of calculating the defined sequence using LN and non-overlapping window self-attention mechanism, and obtain the first feature map;

[0016] The first feature map is subjected to feature fusion with the result of calculating the first feature map by using LN and multi-layer perception mechanism to obtain a second feature map;

[0017] The second feature map is subjected to feature fusion with the result of calculating the second feature map using the LN and displacement window multi-head self-attention mechanism to obtain a third feature map;

[0018] The third feature map is subjected to feature fusion with the result of calculating the third feature map by using LN and multi-layer perception mechanism to obtain a fourth feature map;

[0019] The fourth feature map is used as the multi-scale feature map of the first stage, and the multi-scale feature map of the first stage is repeatedly feature fused to obtain the multi-scale feature map of the second stage. The above steps are repeated to obtain the multi-scale feature maps of the four stages.

[0020] Preferably, the cross-fusion and noise suppression of multi-scale feature maps of multiple stages to obtain fused feature information includes:

[0021] Determine the current resolution of the multi-scale feature map at each stage, and determine the feature learning adjustment weight of the multi-scale feature map at each stage according to the current resolution;

[0022] The top-down and bottom-up feature fusion methods are used and the feature learning and weight adjustment of the multi-scale feature map in each stage are used to cross-fuse the multi-scale feature map to obtain the output feature map of each stage;

[0023] The output feature map of each stage is denoised by a 1*1 convolution to obtain the denoised feature map of each stage;

[0024] The fused feature information is obtained based on the denoised feature map at each stage.

[0025] Preferably, the output of the vehicle detection frame, vehicle category and vehicle mask based on the fusion feature information and the multi-scale feature map through floating point calculation and bilinear interpolation includes:

[0026] Based on the fusion feature information and multi-scale feature map, a region proposal network is used to generate a candidate region containing the approximate location information of the vehicle target;

[0027] Calculate the confidence of each candidate region, select the target candidate region with the highest confidence and map it to the vehicle image;

[0028] Determine the region boundary and detail information of the target candidate region on the vehicle image through floating point calculation and bilinear interpolation;

[0029] Output vehicle detection box, vehicle category and vehicle mask based on region boundary and detail information.

[0030] A global perception system for vehicle detection, the system comprising:

[0031] A segmentation module, used to obtain a vehicle image and divide it into multiple feature blocks;

[0032] The first acquisition module is used to input each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages;

[0033] The second acquisition module is used to cross-fuse and suppress noise on multi-scale feature maps of multiple stages to obtain fused feature information;

[0034] The output module is used to output the vehicle detection frame, vehicle category and vehicle mask based on the fused feature information and multi-scale feature map through floating-point calculation and bilinear interpolation.

[0035] Preferably, the division module includes:

[0036] A division submodule is used to obtain a vehicle image and convert it into an RGB three-channel image, divide the RGB three-channel image into multiple grids, and obtain a feature block in each grid;

[0037] The first acquisition module includes:

[0038] A conversion submodule, which converts the feature blocks into a one-dimensional vector sequence and feeds it into the cascade-based attention baseline network;

[0039] The first acquisition submodule is used to perform feature fusion on multiple one-dimensional vector sequences through the window processing mechanism of the attention baseline network to obtain multi-scale feature maps of multiple stages.

[0040] Preferably, the first acquisition submodule includes:

[0041] A first feature fusion unit is used to define each one-dimensional vector sequence, perform feature fusion on the defined sequence and the result of calculating the defined sequence using LN and non-overlapping window self-attention mechanism, and obtain a first feature map;

[0042] A second feature fusion unit is used to perform feature fusion on the first feature map and a result of calculating the first feature map using the LN and multi-layer perception mechanism to obtain a second feature map;

[0043] A third feature fusion unit is used to perform feature fusion on the second feature map and a result of calculating the second feature map using the LN and displacement window multi-head self-attention mechanism to obtain a third feature map;

[0044] A fourth feature fusion unit is used to perform feature fusion on the third feature map and a result of calculating the third feature map using the LN and multi-layer perception mechanism to obtain a fourth feature map;

[0045] The acquisition unit is used to use the fourth feature map as the multi-scale feature map of the first stage, and repeatedly perform feature fusion on the multi-scale feature map of the first stage to obtain the multi-scale feature map of the second stage, and repeat the above steps to obtain the multi-scale feature maps of four stages.

[0046] Preferably, the second acquisition module includes:

[0047] A first determination submodule is used to determine the current resolution of the multi-scale feature map at each stage, and determine the feature learning adjustment weight of the multi-scale feature map at each stage according to the current resolution;

[0048] The second acquisition submodule is used to cross-fuse the multi-scale feature maps by adopting top-down and bottom-up feature fusion methods and learning and adjusting weights of the multi-scale feature maps at each stage to obtain the output feature map of each stage;

[0049] The denoising submodule is used to denoise the output feature map of each stage through a 1*1 convolution to obtain the denoised feature map of each stage;

[0050] The third acquisition submodule is used to obtain fusion feature information according to the denoising feature map of each stage.

[0051] Preferably, the output module comprises:

[0052] A generation submodule, for generating a candidate region containing approximate location information of a vehicle target using a region candidate network based on fused feature information and a multi-scale feature map;

[0053] The calculation submodule is used to calculate the confidence of each candidate area, select the target candidate area with the highest confidence and map it to the vehicle image;

[0054] The second determination submodule is used to determine the region boundary and detail information of the target candidate region on the vehicle image through floating point calculation and bilinear difference;

[0055] The output submodule is used to output the vehicle detection frame, vehicle category and vehicle mask according to the region boundary and detail information.

[0056] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0057] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0059] Figure 1 A flowchart of a global perception method for vehicle detection provided by the present invention;

[0060] Figure 2 Another working flow chart of a global perception method for vehicle detection provided by the present invention;

[0061] Figure 3 A schematic diagram of the structure of a global perception system for vehicle detection provided by the present invention;

[0062] Figure 4 A schematic diagram of the structure of a second acquisition module in a global perception system for vehicle detection provided by the present invention;

[0063] Figure 5 A model structure diagram of a vehicle detection model constructed according to a global perception method for vehicle detection provided by the present invention;

[0064] Figure 6 A working principle diagram of a CT module in a vehicle detection model constructed according to a global perception method for vehicle detection provided by the present invention;

[0065] Figure 7 A working principle diagram of a CSAF module in a vehicle detection model constructed according to a global perception method for vehicle detection provided by the present invention;

[0066] Figure 8 A diagram showing the working principle of an adaptive module in a vehicle detection model constructed according to a global perception method for vehicle detection provided by the present invention. DETAILED DESCRIPTION

[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0068] A global perception method for vehicle detection, such as Figure 1 As shown, the following steps are included:

[0069] Step S101, obtaining a vehicle image and dividing it into a plurality of feature blocks;

[0070] Step S102: input each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages;

[0071] Step S103, cross-fusing and noise-suppressing the multi-scale feature maps of multiple stages to obtain fused feature information;

[0072] Step S104: Based on the fused feature information and the multi-scale feature map, a vehicle detection frame, a vehicle category, and a vehicle mask are output through floating-point calculation and bilinear interpolation.

[0073] The working principle of the above technical solution is: obtain a vehicle picture and divide it into multiple feature blocks; input each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages; cross-fuse and noise suppress the multi-scale feature maps of multiple stages to obtain fused feature information; based on the fused feature information and multi-scale feature maps, output the vehicle detection frame, vehicle category and vehicle mask through floating-point calculation and bilinear interpolation.

[0074] The beneficial effects of the above technical solution are as follows: by obtaining a multi-scale feature map containing a vehicle image and fusing feature information to provide rich vehicle feature information, and then performing vehicle detection frame positioning and vehicle classification based on floating-point calculation and bilinear interpolation, the global vehicle feature information can be greatly utilized to perform high-precision vehicle detection and classification, thereby ensuring the reliability and stability of detection and classification, and solving the problem mentioned in the prior art that the small target pixels in the resolution image collected by the model detection method based on the attention mechanism are still small, making it difficult to accurately capture the global features of the vehicle, resulting in the final vehicle detection positioning and classification results being of low accuracy, thereby reducing the practicality.

[0075] In one embodiment, the step of acquiring a vehicle image and dividing the image into a plurality of feature blocks includes:

[0076] Obtain a vehicle image and convert it into an RGB three-channel image, divide the RGB three-channel image into multiple grids, and obtain feature blocks in each grid;

[0077] The step of inputting each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages includes:

[0078] The feature blocks are converted into a one-dimensional vector sequence and fed into the cascade-based attention baseline network;

[0079] The window processing mechanism of the attention baseline network is used to fuse the features of multiple one-dimensional vector sequences to obtain multi-scale feature maps at multiple stages.

[0080] The beneficial effect of the above technical solution is that by processing feature maps of different scales, the network is ensured to capture vehicle information of various scales. This is crucial for vehicle detection in complex traffic environments, because vehicles may appear at different positions and distances in the image and have different sizes. The acquisition of multi-scale information enables GCAFormer to detect and classify various types of vehicles more accurately, thereby improving the robustness and accuracy of detection.

[0081] In one embodiment, the feature fusion of multiple one-dimensional vector sequences is performed through the window processing mechanism of the attention baseline network to obtain multi-scale feature maps of multiple stages, including:

[0082] Define each one-dimensional vector sequence, perform feature fusion on the defined sequence and the result of calculating the defined sequence using LN and non-overlapping window self-attention mechanism, and obtain the first feature map;

[0083] The first feature map is subjected to feature fusion with the result of calculating the first feature map by using LN and multi-layer perception mechanism to obtain a second feature map;

[0084] The second feature map is subjected to feature fusion with the result of calculating the second feature map using the LN and displacement window multi-head self-attention mechanism to obtain a third feature map;

[0085] The third feature map is subjected to feature fusion with the result of calculating the third feature map by using LN and multi-layer perception mechanism to obtain a fourth feature map;

[0086] The fourth feature map is used as the multi-scale feature map of the first stage, and the multi-scale feature map of the first stage is repeatedly feature fused to obtain the multi-scale feature map of the second stage. The above steps are repeated to obtain the multi-scale feature maps of the four stages.

[0087] The beneficial effects of the above technical solution are as follows: by adopting bidirectional feature fusion technology, combining low-dimensional and high-dimensional semantic information, the feature information expression capability can be enhanced. From stage 1 to stage 4, low-dimensional feature maps and high-dimensional low-resolution feature maps with rich spatial position information are received, which can better identify vehicles of different sizes and shapes, thereby improving recognition accuracy and efficiency.

[0088] In one embodiment, Figure 2 As shown, the multi-scale feature maps of multiple stages are cross-fused and noise suppressed to obtain fused feature information, including:

[0089] Step S201, determining the current resolution of the multi-scale feature map at each stage, and determining the feature learning adjustment weight of the multi-scale feature map at each stage according to the current resolution;

[0090] Step S202, adopting top-down and bottom-up feature fusion methods and adjusting weights of multi-scale feature map feature learning at each stage to cross-fuse the multi-scale feature map to obtain an output feature map at each stage;

[0091] Step S203, denoising the output feature map of each stage through a 1*1 convolution to obtain a denoised feature map of each stage;

[0092] Step S204: Obtain fusion feature information according to the denoising feature map of each stage.

[0093] The beneficial effects of the above technical solution are: fully utilizing multi-scale feature information to realize the stage vehicle output feature map, indirectly improving the recognition accuracy, and effectively suppressing noise. In complex traffic scenes, more pure and useful feature information can be extracted.

[0094] In one embodiment, the output of the vehicle detection frame, vehicle category and vehicle mask based on the fusion feature information and the multi-scale feature map through floating point calculation and bilinear interpolation includes:

[0095] Based on the fusion feature information and multi-scale feature map, a region proposal network is used to generate a candidate region containing the approximate location information of the vehicle target;

[0096] Calculate the confidence of each candidate region, select the target candidate region with the highest confidence and map it to the vehicle image;

[0097] Determine the region boundary and detail information of the target candidate region on the vehicle image through floating point calculation and bilinear interpolation;

[0098] Output vehicle detection box, vehicle category and vehicle mask based on region boundary and detail information.

[0099] The beneficial effects of the above technical solution are: by calculating the confidence of the candidate area of ​​the vehicle's approximate position information and mapping it to the vehicle picture, the area boundary and detail information are determined through floating-point calculation and bilinear difference, and multiple relevant information of the vehicle can be quickly and accurately obtained based on rich feature information, ensuring the globality of vehicle position detection and classification.

[0100] In one embodiment, Figure 5The vehicle detection model proposed in this patent is demonstrated, which includes a cascaded self-attention backbone network (CT), a cross-scale aggregated feature module (CSAF) and a detection head (the main task of the detection head is to accept multi-scale feature maps from the CSAF module, and then perform detection, classification and masking). GCAFormer converts input images of different sizes to 512×512 for better training. The input RGB three-channel image is divided into blocks, and these feature blocks (tokens) are converted into a one-dimensional vector sequence and then fed to the cascade transformer block (CT) encoder. First, assume that the input image is I∈R H*W*3 , where H, W and 3 represent the spatial height, width and number of channels of the image respectively, and R represents a natural number. Then, I is divided into (H×W) / K 2 There are grids, each of size K*K*3, and each feature block is regarded as a marker, whose attribute is defined as a segment of the original image RGB value. All feature blocks in the grid are flattened to form a sequence x∈R N*P , where N = HW / K 2 represents the number of segmented feature blocks, and P = K × K × 3 represents the size of each feature block. In the sequence x, we apply a learnable mapping l: To get the sequence e i ∈R N*P (i∈1,

[0101] ..., N), and the processed sequence is input into CT for encoding. As the backbone of GCAFormer hierarchical feature extraction, CT not only adopts window self-attention design, but also introduces Transformer-based CNN position encoding. CT limits the self-attention calculation to each window, ensuring that the information within the window can fully interact with each other, greatly reducing the amount of calculation. CT also introduces a shift window so that information between different windows can be exchanged. CT also introduces a shift window so that information between different windows can be exchanged. Therefore, CT can effectively introduce multi-scale feature information and solve the problem of multi-task vehicle detection in complex traffic environments. The framework of CT is as follows Figure 2As shown, W-MSA is to divide the image into non-overlapping windows, each window performs self-attention calculation, and SW-MSA is a multi-head self-attention module dedicated to the shift window. The hierarchical normalization (LN) layer is used before the multi-head self-attention (MSA) and multi-layer perceptron (MLP) calculations, and there is a residual connection after the MSA and MLP calculations. Multiple CT blocks receive input from the previous step. The backbone of GCAFormer consists of 4 stages. Stage 1 (Stage1) consists of linear embedding and CT blocks to ensure that the input dimension is H / 4×W / 4×C. In stage 2 (Stage2), the feature blocks are merged to merge the attributes of each group of adjacent 2×2 patches. A linear layer is then applied to the merged feature blocks to change their size from 4c to 2c for output. The CT block is then used for feature extraction and output, while the resolution is changed to H / 8×W / 8. The above process of stage 2 is repeated twice to obtain stage 3 (Stage3) and stage 4 (Stage4). The output resolutions are H / 16 × W / 16 and H / 32 × W / 32. The above feature maps are generated in a layered manner similar to traditional convolutional neural networks with feature maps of the same resolution.

[0102] like Figure 6 As shown in the figure, the main contribution of CT is to create a cascaded attention baseline network that can deeply exchange information in feature blocks (feature maps are divided into small blocks, representing feature maps of smaller scales). As can be seen from the figure, Stage1-Stage4 represent four different stages, respectively representing the process from shallow semantic information to deep semantic information, so CT can make full use of feature information of more scales.

[0103] The CSAF module represents receiving low-dimensional feature maps with rich spatial location information and high-dimensional low-resolution feature maps from stage 1 to stage 4, respectively. Since deep feature maps and shallow feature maps involve different semantic information, the combination of these two types of information can be applicable to vehicles of different sizes and shapes in traffic environments. Therefore, in this patent, we designed a CSAF module to receive multi-scale feature maps from the CT backbone network and then process them to promote the fusion of multi-scale information.

[0104] The CSAF module adopts a bidirectional path fusion strategy (top-down and bottom-up) and an adaptive module (SA). Figure 7 This process is demonstrated. Figure 8The SA* process is demonstrated. CSAF cross-fuses the multi-scale feature maps from stage 1 to stage 4, and the SA module suppresses background noise on the fused feature maps. The traditional method of adjusting the resolution and then performing pixel-level addition operations is not sufficient for vehicle detection in complex traffic environments. Previous studies have pointed out that feature maps of different resolutions have different weight distributions, so in the CSAF module, we add additional weights to each input branch, allowing the network to learn important feature information on each input. For the additional weights, we use a fast normalization fusion strategy To adjust the weight of the CSAF module. i is the learnable weight, σ is the correlation coefficient, I i is the input feature map from stage 1 to stage 4, and then θ i Processing, using the ReLU activation function to make the weight greater than 0. The output feature map of the CSAF module can be described as follows:

[0105]

[0106]

[0107]

[0108]

[0109] In the above formula, the input feature maps of stage 1, stage 2, stage 3 and stage 4 are respectively and denoted by . And the output features are denoted by Y1, Y2, Y3, and Y4. θ1′ and θ2′ represent the weights of each level, respectively, and re(·) represents the adjustment of the size of the feature map to match the input. Since the CSAF module as a whole is a cross-fusion of different stages in the feature map, it is easy to introduce background noise to affect the detection effect. Therefore, we propose an adaptive block (SA), which aims to suppress the semantic information of the shallow stage interfered by background noise, such as Figure 4 The detailed description of SA is as follows.

[0110] G=w1(S,λ)(5)

[0112]

[0113] in, represents a 1×1 convolution operation; λ represents the correlation coefficient of the input S feature map; represents pixel-level multiplication operation; δ(·) represents the S-shaped activation function that maps the input to a 0 to 1 distribution; δ(·) represents the use of the Sigmoid activation function to map the input to a 0 to 1 distribution; M(·) is the information interaction between channels; GAP(·) is the global average pooling operation that generates channel-dimensional statistics, and γ is the correlation coefficient with G, and The intermediate feature layer is represented as follows:

[0114]

[0115] Where Conv(·) represents the convolution operation, and θ1 and θ2 represent the corresponding weights. The detection head consists of two stages, designed to predict object classes and output detection boxes. Through the experiments we conducted, we found that the performance of Mask-RCNN meets our requirements for the detection head component. Therefore, we choose to use Mask RCNN as the detection head in this study. Figure 1 As shown in the head of , it includes a fully connected layer, a region proposal network (RPN), bounding box prediction, category prediction, a region of interest (ROI), and a loss function. It aligns between pixels through ROI alignment and outputs a vehicle detection box (bbox), vehicle category (class), and vehicle mask (mask).

[0116] The detection head obtains multi-scale features from the CSAF module and uses the region proposal network (RPN) to generate candidate regions containing the approximate location information of the target. The RPN uses 3×3 and 1×1 convolution operations to obtain the class and detection anchor boxes. These RPNs are input as suggestions into the region of interest (ROI) alignment module, where the ROI alignment module can accurately map the high-scoring candidate regions generated by the RPN to fixed-size feature patches, and retain more boundary and detail information through floating-point calculations and bilinear interpolation.

[0117] First, GCAFormer utilizes the Cascaded Transformer (CT) module to provide multi-scale information. The CT module ensures that the network can capture vehicle information of various scales by processing feature maps of different scales. This is crucial for vehicle detection in complex traffic environments, as vehicles may appear at different locations and distances in the image and have different sizes. The acquisition of multi-scale information enables GCAFormer to more accurately detect and classify various types of vehicles, thereby improving the robustness and accuracy of detection. Secondly, the Cross-Scale Aggregate Feature (CSAF) module is another important component of GCAFormer. The CSAF module uses a bidirectional feature fusion technique to combine low-dimensional and high-dimensional semantic information. This fusion method not only enhances the expressiveness of feature information, but also effectively suppresses noise through the adaptive (SA) module. In complex traffic scenes, images often contain a lot of background interference and noise, and the CSAF module can extract purer and more useful feature information through this bidirectional fusion and noise suppression. This technical approach significantly improves the detection performance of the model in complex environments, allowing the detection head to receive more detailed and accurate feature information. As the last key component of GCAFormer, the detection head performs the final vehicle detection and classification by receiving multi-scale and fused feature information from the CT and CSAF modules. Since the first two modules provide rich feature information, the detection head is able to locate and classify vehicles more accurately. This process significantly improves the overall detection accuracy of the system. As shown in Table 1, GCAFormer achieves 93.33% mAP on the BDD-100k dataset, which is 2.88% higher than its strongest competitor YOLO-SE. As shown in Table 2, on the KITTI dataset (E, M, and H represent easy, medium, and difficult, respectively), GCAFormer achieves 90.33% in mAP, surpassing the current SOTA models SatDetX-YOLO and YOLOSR-IST. This result fully demonstrates the effectiveness and superiority of GCAFormer in practical applications.

[0118] In general, GCAFormer achieves efficient detection of vehicles in complex traffic environments through the collaborative work of CT and CSAF, combined with the precise classification and positioning capabilities of the detection head. These technical means not only improve the detection robustness and accuracy of the system, but also significantly enhance its adaptability and practicality in a variety of practical application scenarios.

[0119] In one embodiment, this embodiment also discloses a global perception system for vehicle detection, such as Figure 3 As shown, the system includes:

[0120] A division module 301 is used to obtain a vehicle image and divide it into a plurality of feature blocks;

[0121] A first acquisition module 302 is used to input each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages;

[0122] The second acquisition module 303 is used to cross-fuse and suppress noise on multi-scale feature maps of multiple stages to obtain fused feature information;

[0123] The output module 304 is used to output the vehicle detection frame, vehicle category and vehicle mask based on the fused feature information and the multi-scale feature map through floating-point calculation and bilinear interpolation.

[0124] The working principle and beneficial effects of the above technical solution have been explained in the method embodiment and will not be repeated here.

[0125] In one embodiment, the partitioning module includes:

[0126] A division submodule is used to obtain a vehicle image and convert it into an RGB three-channel image, divide the RGB three-channel image into multiple grids, and obtain a feature block in each grid;

[0127] The first acquisition module includes:

[0128] A conversion submodule, which converts the feature blocks into a one-dimensional vector sequence and feeds it into the cascade-based attention baseline network;

[0129] The first acquisition submodule is used to perform feature fusion on multiple one-dimensional vector sequences through the window processing mechanism of the attention baseline network to obtain multi-scale feature maps of multiple stages.

[0130] In one embodiment, the first acquisition submodule includes:

[0131] A first feature fusion unit is used to define each one-dimensional vector sequence, perform feature fusion on the defined sequence and the result of calculating the defined sequence using LN and non-overlapping window self-attention mechanism, and obtain a first feature map;

[0132] A second feature fusion unit is used to perform feature fusion on the first feature map and a result of calculating the first feature map using the LN and multi-layer perception mechanism to obtain a second feature map;

[0133] A third feature fusion unit is used to perform feature fusion on the second feature map and a result of calculating the second feature map using the LN and displacement window multi-head self-attention mechanism to obtain a third feature map;

[0134] A fourth feature fusion unit is used to perform feature fusion on the third feature map and a result of calculating the third feature map using the LN and multi-layer perception mechanism to obtain a fourth feature map;

[0135] The acquisition unit is used to use the fourth feature map as the multi-scale feature map of the first stage, and repeatedly perform feature fusion on the multi-scale feature map of the first stage to obtain the multi-scale feature map of the second stage, and repeat the above steps to obtain the multi-scale feature maps of four stages.

[0136] In one embodiment, Figure 4 As shown, the second acquisition module 303 includes:

[0137] A first determination submodule 3031 is used to determine the current resolution of the multi-scale feature map at each stage, and determine the feature learning adjustment weight of the multi-scale feature map at each stage according to the current resolution;

[0138] The second acquisition submodule 3032 is used to cross-fuse the multi-scale feature map by adopting the top-down and bottom-up feature fusion methods and the feature learning adjustment weights of the multi-scale feature map at each stage to obtain the output feature map of each stage;

[0139] The denoising submodule 3033 is used to denoise the output feature map of each stage through a 1*1 convolution to obtain a denoised feature map of each stage;

[0140] The third acquisition submodule 3034 is used to acquire fusion feature information according to the denoising feature map of each stage.

[0141] In one embodiment, the output module comprises:

[0142] A generation submodule, for generating a candidate region containing approximate location information of a vehicle target using a region candidate network based on fused feature information and a multi-scale feature map;

[0143] The calculation submodule is used to calculate the confidence of each candidate area, select the target candidate area with the highest confidence and map it to the vehicle image;

[0144] The second determination submodule is used to determine the region boundary and detail information of the target candidate region on the vehicle image through floating point calculation and bilinear difference;

[0145] The output submodule is used to output the vehicle detection frame, vehicle category and vehicle mask according to the region boundary and detail information.

[0146] Those skilled in the art should understand that the first and second in the present invention merely refer to different application stages.

[0147] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0148] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A global perception method for vehicle detection, characterized in that: The following steps are involved: Obtain a vehicle image and divide it into multiple feature blocks; Each feature block is input into the cascade-based attention baseline network to obtain multi-scale feature maps at multiple stages; Cross-fuse and suppress noise on multi-scale feature maps from multiple stages to obtain fused feature information; Based on the fusion feature information and multi-scale feature maps, the vehicle detection frame, vehicle category and vehicle mask are output through floating-point calculation and bilinear interpolation.

2. The global perception method for vehicle detection according to claim 1, characterized in that: The method of obtaining a vehicle image and dividing it into a plurality of feature blocks includes: Obtain a vehicle image and convert it into an RGB three-channel image, divide the RGB three-channel image into multiple grids, and obtain feature blocks in each grid; The step of inputting each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages includes: The feature blocks are converted into a one-dimensional vector sequence and fed into the cascade-based attention baseline network; The window processing mechanism of the attention baseline network is used to fuse the features of multiple one-dimensional vector sequences to obtain multi-scale feature maps at multiple stages.

3. The global perception method for vehicle detection according to claim 2, characterized in that: The window processing mechanism of the attention baseline network is used to perform feature fusion on multiple one-dimensional vector sequences to obtain multi-scale feature maps of multiple stages, including: Define each one-dimensional vector sequence, perform feature fusion on the defined sequence and the result of calculating the defined sequence using LN and non-overlapping window self-attention mechanism, and obtain the first feature map; The first feature map is subjected to feature fusion with the result of calculating the first feature map by using LN and multi-layer perception mechanism to obtain a second feature map; The second feature map is subjected to feature fusion with the result of calculating the second feature map using the LN and displacement window multi-head self-attention mechanism to obtain a third feature map; The third feature map is subjected to feature fusion with the result of calculating the third feature map by using LN and multi-layer perception mechanism to obtain a fourth feature map; The fourth feature map is used as the multi-scale feature map of the first stage, and the multi-scale feature map of the first stage is repeatedly feature fused to obtain the multi-scale feature map of the second stage. The above steps are repeated to obtain the multi-scale feature maps of the four stages.

4. The global perception method for vehicle detection according to claim 1, characterized in that: The method of cross-fusing and noise-suppressing the multi-scale feature maps of multiple stages to obtain fused feature information includes: Determine the current resolution of the multi-scale feature map at each stage, and determine the feature learning adjustment weight of the multi-scale feature map at each stage according to the current resolution; The top-down and bottom-up feature fusion methods are used and the feature learning and weight adjustment of the multi-scale feature map in each stage are used to cross-fuse the multi-scale feature map to obtain the output feature map of each stage; The output feature map of each stage is denoised by a 1*1 convolution to obtain the denoised feature map of each stage; The fused feature information is obtained based on the denoised feature map at each stage.

5. The global perception method for vehicle detection according to claim 1, characterized in that: The method outputs a vehicle detection frame, a vehicle category, and a vehicle mask based on the fusion feature information and the multi-scale feature map through floating point calculation and bilinear interpolation, including: Based on the fusion feature information and multi-scale feature map, a region proposal network is used to generate a candidate region containing the approximate location information of the vehicle target; Calculate the confidence of each candidate region, select the target candidate region with the highest confidence and map it to the vehicle image; Determine the region boundary and detail information of the target candidate region on the vehicle image through floating point calculation and bilinear interpolation; Output vehicle detection box, vehicle category and vehicle mask based on region boundary and detail information.

6. A global perception system for vehicle detection, characterized in that: The system includes: A segmentation module, used to obtain a vehicle image and divide it into multiple feature blocks; The first acquisition module is used to input each feature block into an attention baseline network based on a cascade mode to obtain multi-scale feature maps of multiple stages; The second acquisition module is used to cross-fuse and suppress noise on multi-scale feature maps of multiple stages to obtain fused feature information; The output module is used to output the vehicle detection frame, vehicle category and vehicle mask based on the fused feature information and multi-scale feature map through floating-point calculation and bilinear interpolation.

7. The global perception system for vehicle detection according to claim 6, characterized in that: The division module comprises: A division submodule is used to obtain a vehicle image and convert it into an RGB three-channel image, divide the RGB three-channel image into multiple grids, and obtain a feature block in each grid; The first acquisition module includes: A conversion submodule, which converts the feature blocks into a one-dimensional vector sequence and feeds it into the cascade-based attention baseline network; The first acquisition submodule is used to perform feature fusion on multiple one-dimensional vector sequences through the window processing mechanism of the attention baseline network to obtain multi-scale feature maps of multiple stages.

8. The global perception system for vehicle detection according to claim 7, characterized in that: The first acquisition submodule includes: A first feature fusion unit is used to define each one-dimensional vector sequence, perform feature fusion on the defined sequence and the result of calculating the defined sequence using LN and non-overlapping window self-attention mechanism, and obtain a first feature map; A second feature fusion unit is used to perform feature fusion on the first feature map and a result of calculating the first feature map using the LN and multi-layer perception mechanism to obtain a second feature map; A third feature fusion unit is used to perform feature fusion on the second feature map and a result of calculating the second feature map using the LN and displacement window multi-head self-attention mechanism to obtain a third feature map; A fourth feature fusion unit is used to perform feature fusion on the third feature map and a result of calculating the third feature map using the LN and multi-layer perception mechanism to obtain a fourth feature map; The acquisition unit is used to use the fourth feature map as the multi-scale feature map of the first stage, and repeatedly perform feature fusion on the multi-scale feature map of the first stage to obtain the multi-scale feature map of the second stage, and repeat the above steps to obtain the multi-scale feature maps of four stages.

9. The global perception system for vehicle detection according to claim 6, characterized in that: The second acquisition module includes: A first determination submodule is used to determine the current resolution of the multi-scale feature map at each stage, and determine the feature learning adjustment weight of the multi-scale feature map at each stage according to the current resolution; The second acquisition submodule is used to cross-fuse the multi-scale feature maps by adopting top-down and bottom-up feature fusion methods and learning and adjusting weights of the multi-scale feature maps at each stage to obtain the output feature map of each stage; The denoising submodule is used to denoise the output feature map of each stage through a 1*1 convolution to obtain the denoised feature map of each stage; The third acquisition submodule is used to obtain fusion feature information according to the denoising feature map of each stage.

10. The global perception system for vehicle detection according to claim 6, characterized in that: The output module comprises: A generation submodule, for generating a candidate region containing approximate location information of a vehicle target using a region candidate network based on fused feature information and a multi-scale feature map; The calculation submodule is used to calculate the confidence of each candidate area, select the target candidate area with the highest confidence and map it to the vehicle image; The second determination submodule is used to determine the region boundary and detail information of the target candidate region on the vehicle image through floating point calculation and bilinear difference; The output submodule is used to output the vehicle detection frame, vehicle category and vehicle mask according to the region boundary and detail information.

Images