A vehicle detection method, device, equipment and medium based on deformable convolution

Abstract

The application provides a vehicle detection method based on deformable convolution, comprising: inputting a vehicle image into a feature extraction module of a pre-trained image detection model to obtain vehicle feature maps of different scales; inputting the vehicle feature maps of each scale into a feature alignment network of the image detection model respectively to obtain aligned feature maps of each scale under different preset anchor frames; inputting all aligned feature maps under all anchor frames of each scale into a target detection module of the image detection model to generate a candidate region corresponding to each anchor frame, screening and deduplicating the candidate region, and performing category determination and position adjustment on the retained candidate region to obtain a region and a category corresponding to a vehicle in the vehicle image. Through the vehicle detection method, device, equipment and medium based on deformable convolution provided by the application, the accuracy of vehicle detection can be improved.

Description

Technical Field

[0001] This invention relates to the field of transportation, and in particular to a vehicle detection method, apparatus, equipment and medium based on deformable convolution. Background Technology

[0002] With the development of intelligent transportation systems, vehicle detection technology plays a crucial role in traffic monitoring, autonomous driving, and other fields. Traditional methods rely on manually designed features and classifiers, which are easily affected by changes in lighting, occlusion, and background interference in complex scenes, resulting in poor detection performance. Deep learning-based detection algorithms typically employ an anchor box mechanism, using a region proposal network to pre-define multiple anchor boxes with different aspect ratios at each spatial location in the feature map to cover the vehicle, and then classifying and regressing the positions of these anchor boxes.

[0003] However, the sampling regions of the convolutional kernels within the region proposal network are fixed, while the image regions covered by anchor boxes with different aspect ratios are different. This results in a spatial misalignment between the features extracted through the fixed sampling regions and the actual coverage areas of each anchor box, affecting the accuracy of anchor box classification and location regression, and consequently affecting the accuracy of vehicle detection. Summary of the Invention

[0004] This invention provides a vehicle detection method, apparatus, equipment, and medium based on deformable convolution, which can improve the accuracy of vehicle detection.

[0005] This invention provides a vehicle detection method based on deformable convolution, comprising: Vehicle images are input into the feature extraction module of a pre-trained image detection model to obtain vehicle feature maps at different scales; The vehicle feature map at each scale is input into the feature alignment network of the image detection model to obtain the aligned feature map at each scale under different preset anchor boxes; wherein, for the vehicle feature map at each scale: The current scale vehicle feature map is fused with all vehicle feature maps of smaller scales to obtain a fused feature map. The fused feature map is input into deformable convolutions that match different anchor frames for sampling, so as to obtain the sampling features corresponding to different anchor frames at each spatial location of the fused feature map; wherein, the convolution kernel of the deformable convolution adjusts the sampling offset according to different anchor frames; For each anchor frame: combine the sampled features of all spatial locations under the anchor frame to obtain the alignment feature map of that scale under the anchor frame; All aligned feature maps under all anchor frames at each scale are input into the target detection module of the image detection model to generate candidate regions corresponding to each anchor frame. The candidate regions are filtered and deduplicated, and the retained candidate regions are classified and their positions are adjusted to obtain the region and category corresponding to the vehicle in the vehicle image.

[0006] In one embodiment of the present invention, the feature extraction module includes a backbone network, a feature pyramid network, and a feature enhancement network; the step of inputting the vehicle image into the feature extraction module of a pre-trained image detection model to obtain vehicle feature maps at different scales includes: Vehicle images are input into the backbone network to obtain initial vehicle feature maps at different scales; Initial vehicle feature maps at different scales are input into the feature pyramid network, and feature fusion is performed step by step from small scale to large scale to obtain intermediate vehicle feature maps at different scales. The intermediate vehicle feature maps at each scale are input into the feature enhancement network. The feature importance of each channel and the salience of vehicles at each spatial location in the intermediate vehicle feature map at that scale are analyzed and fused together to obtain the vehicle feature map corresponding to that scale.

[0007] In one embodiment of the present invention, the step of inputting the intermediate vehicle feature map at each scale into the feature enhancement network, and performing collaborative analysis and fusion of the feature importance of each channel and the vehicle saliency at each spatial location in the intermediate vehicle feature map at that scale to obtain the vehicle feature map corresponding to that scale includes: Intermediate vehicle feature maps for each scale: Based on the feature dependencies between channels, attention analysis is performed on the intermediate vehicle feature map along the channel dimension to generate channel attention weights that characterize the importance of each channel to the vehicle features. Based on the feature correlation of spatial location, spatial attention analysis is performed on the intermediate vehicle feature map to generate spatial attention weights that characterize the salience of vehicles at each spatial location. The channel attention weight and the spatial attention weight are fused to obtain a comprehensive attention weight. The intermediate vehicle feature map is then weighted according to the comprehensive attention weight to obtain the vehicle feature map corresponding to that scale.

[0008] In one embodiment of the present invention, the step of fusing the vehicle feature map at the current scale with all vehicle feature maps at scales smaller than the current scale to obtain a fused feature map includes: Obtain feature maps of all vehicles with a scale smaller than the current scale; Starting with the smallest vehicle feature map, the features are gradually fused to the current scale to obtain a fused feature map.

[0009] In one embodiment of the present invention, the step of inputting the fused feature map into deformable convolutions matched with different anchor boxes for sampling, so as to obtain the sampling features corresponding to different anchor boxes at each spatial location of the fused feature map, includes: The fused feature map is input into the deformable convolution, and the sampling offset of the convolution kernel in the deformable convolution is adjusted for different anchor boxes. Based on the sampling offset corresponding to each anchor frame, the fused feature map is sampled separately to obtain the sampling features corresponding to different anchor frames at each spatial location of the fused feature map.

[0010] In one embodiment of the present invention, the step of inputting all aligned feature maps under all anchor frames at each scale into the target detection module of the image detection model to generate candidate regions corresponding to each anchor frame, filtering and deduplicating the candidate regions, and classifying and adjusting the positions of the retained candidate regions to obtain the region and category corresponding to the vehicle in the vehicle image includes: All aligned feature maps under all anchor boxes at each scale are input into the region proposal network of the target detection module of the image detection model to obtain the corresponding candidate regions. All candidate regions are filtered using a nonmaximum suppression method; The classification and regression branches of the target detection module are used to determine the category and adjust the position of the retained candidate regions, respectively, to obtain the region and category of the vehicle in the vehicle image.

[0011] In one embodiment of the present invention, the step of determining the category and adjusting the position of the retained candidate regions through the classification branch and regression branch of the target detection module to obtain the region and category of the vehicle in the vehicle image includes: The retained candidate regions are input into the classification branch, and category-related information is extracted to obtain the category of the vehicle in the vehicle image; The retained candidate regions are input into the regression branch to extract information related to the vehicle's position, obtain the position offset, and adjust the position of the candidate regions according to the position offset to obtain the region of the vehicle in the vehicle image.

[0012] The present invention also provides a vehicle detection device based on deformable convolution, comprising: The feature extraction unit is used to input vehicle images into the feature extraction module of a pre-trained image detection model to obtain vehicle feature maps at different scales. The feature alignment unit is used to input the vehicle feature map at each scale into the feature alignment network of the image detection model to obtain the aligned feature map at each scale under different preset anchor boxes; wherein, for the vehicle feature map at each scale: The current scale vehicle feature map is fused with all vehicle feature maps of smaller scales to obtain a fused feature map. The fused feature map is input into deformable convolutions that match different anchor frames for sampling, so as to obtain the sampling features corresponding to different anchor frames at each spatial location of the fused feature map; wherein, the convolution kernel of the deformable convolution adjusts the sampling offset according to different anchor frames; For each anchor frame: combine the sampled features of all spatial locations under the anchor frame to obtain the alignment feature map of that scale under the anchor frame; The target detection unit is used to input all aligned feature maps under all anchor boxes at various scales into the target detection module of the image detection model, generate candidate regions corresponding to each anchor box, filter and deduplicate the candidate regions, and perform category determination and position adjustment on the retained candidate regions to obtain the region and category corresponding to the vehicle in the vehicle image.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the vehicle detection method based on deformable convolution.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the vehicle detection method based on deformable convolution.

[0015] The beneficial effects of this invention are as follows: By using deformable convolution to adjust the sampling area of ​​the convolution kernel according to the shape of different anchor frames before the features enter the region proposal network, spatially aligned feature maps are generated for each anchor frame, thus eliminating the feature misalignment between the fixed sampling area of ​​the convolution kernel and the coverage area of ​​the multi-shaped anchor frames in the region proposal network. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] In the attached diagram: Figure 1A flowchart illustrating a vehicle detection method based on deformable convolution provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of an image detection model provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of a feature enhancement network provided in one embodiment of the present invention; Figure 4 This is a schematic diagram of a feature alignment network provided in one embodiment of the present invention; Figure 5 This is a schematic diagram of the classification branch and regression branch provided in one embodiment of the present invention; Figure 6 This is a schematic diagram of a vehicle detection device based on deformable convolution provided in one embodiment of the present invention; Figure 7 This is a schematic diagram of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0018] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0021] In applications such as intelligent transportation systems and autonomous driving, vehicle detection and vehicle type classification technologies play a crucial role. Currently, deep learning-based vehicle detection methods typically employ an anchor box mechanism, pre-setting multiple anchor boxes with different aspect ratios at each spatial location in the feature map to cover vehicle targets of varying shapes. These anchor boxes are then classified and their locations regressed using a region proposal network. However, the sampling regions of the convolutional kernels within the region proposal network are fixed, while the image regions covered by anchor boxes with different aspect ratios vary. This leads to a spatial misalignment between the features extracted from the fixed sampling regions and the actual coverage areas of each anchor box, resulting in insufficient confidence in anchor box classification and accuracy in location regression, thus affecting the accuracy of vehicle detection.

[0022] To address this, the present invention discloses a vehicle detection method based on deformable convolution. This method utilizes deformable convolution to adjust the sampling region of the convolution kernel according to the shape of different anchor boxes before the feature map enters the region proposal network, generating spatially aligned feature maps for each anchor box. This eliminates feature misalignment between the fixed sampling region and the anchor box coverage area. Based on this, the region proposal network performs anchor box classification and position regression based on the aligned feature maps, obtaining more accurate candidate regions and thus improving the accuracy and robustness of vehicle detection.

[0023] Please see Figure 1 and Figure 2 The vehicle detection method based on deformable convolution disclosed in this invention includes the following steps: S10, inputting the vehicle image into the feature extraction module of a pre-trained image detection model to obtain vehicle feature maps at different scales. The feature extraction module includes a backbone network, a feature pyramid network, and a feature enhancement network. In this embodiment, S10 includes the following steps: S11, inputting the vehicle image into the backbone network to obtain initial vehicle feature maps at different scales.

[0024] Vehicle images can be acquired through cameras, drones, or other image acquisition devices, such as vehicle images taken under actual traffic conditions including different lane environments, shooting angles, and lighting intensities. Before being input into the backbone network, vehicle images need to be resized to a uniform size, for example, scaling the vehicle image to 3×512×512, where 3 represents the three RGB color channels and 512×512 represents the resolution. The resized vehicle image is then input into the backbone network as a tensor.

[0025] The backbone network 100 consists of multiple stacked network layers of different scales, used to extract features layer by layer from the input vehicle image, resulting in initial vehicle feature maps at different scales. As the network layers deepen, the extracted initial vehicle feature maps gradually decrease in scale and gradually increase in the number of channels, forming feature representations from low to high level. Small-scale initial vehicle feature maps have lower resolution but contain more abstract high-level semantic information; large-scale initial vehicle feature maps have higher resolution and contain rich details, such as vehicle edges, headlight outlines, window corners, tire textures, and local color variations.

[0026] For example, the backbone network 100 may contain 4 network layers. After the vehicle image is processed through these 4 network layers in sequence, 4 initial vehicle feature maps of different scales are obtained. Their dimensions [C, W, H] can be represented as [64, 256, 256], [128, 128, 128], [256, 64, 64], and [512, 32, 32], respectively. C represents the number of channels, and W and H represent the resolution. These 4 initial vehicle feature maps, from shallow to deep and from large to small, capture different levels of information in the vehicle image, from local details to overall semantics.

[0027] The S10 disclosed in this embodiment includes the following steps: S12, inputting initial vehicle feature maps of different scales into the feature pyramid network, and performing feature fusion step by step from small scale to large scale to obtain intermediate vehicle feature maps of different scales.

[0028] The Feature Pyramid Network 200 consists of multiple processing layers used to progressively fuse initial vehicle feature maps of different scales output by the backbone network, from small to large scale. Since small-scale initial vehicle feature maps contain rich abstract semantic information, while large-scale initial vehicle feature maps contain rich detailed information, progressive fusion from small to large scale allows the semantic information in the small-scale initial vehicle feature maps to be gradually transferred and integrated into the large-scale initial vehicle feature maps, ensuring that intermediate vehicle feature maps at each scale simultaneously possess both detailed and semantic information.

[0029] Specifically, the feature pyramid network 200 contains one more processing layer than the backbone network 100, for example, five layers. The first processing layer of the feature pyramid network receives the initial vehicle feature map at the smallest scale output by the backbone network, i.e., the initial vehicle feature map at the first scale. The first processing layer performs convolution processing on the initial vehicle feature map at the smallest scale to obtain an intermediate vehicle feature map at the first scale. This intermediate feature map is further reduced in scale and has a further increased number of channels compared to the initial vehicle feature map at the smallest scale, for example, with a size of [1024, 16, 16].

[0030] The second processing layer of the feature pyramid network also receives the initial vehicle feature map at the minimum scale output by the backbone network and outputs the initial vehicle feature map at the minimum scale directly as the intermediate vehicle feature map at the second scale, for example, with a size of [512, 32, 32].

[0031] The third processing layer of the Feature Pyramid Network 200 receives two inputs: one is the second-scale intermediate vehicle feature map output from the second processing layer, and the other is the second-scale initial vehicle feature map output from the backbone network. The second scale is larger than the first scale. The third processing layer first upsamples the second-scale intermediate vehicle feature map to enlarge its spatial size to match that of the second-scale initial vehicle feature map. Then, it fuses the two to obtain the third-scale intermediate vehicle feature map. Through this fusion operation, semantic information from the smaller-scale intermediate vehicle feature map is injected into the larger-scale intermediate vehicle feature map, allowing the fused feature map to gain semantic supplementation while retaining the original detailed information.

[0032] The fourth processing layer of the Feature Pyramid Network 200 receives the intermediate vehicle feature map at the third scale output from the third processing layer and the initial vehicle feature map at the third scale output from the backbone network. It first performs upsampling to align the scales and then fuses them to obtain the intermediate vehicle feature map at the fourth scale.

[0033] The fifth processing layer of the Feature Pyramid Network 200 receives the fourth-scale intermediate vehicle feature map output from the fourth processing layer and the fourth-scale initial vehicle feature map output from the backbone network. It also performs upsampling alignment and fusion to obtain the fifth-scale intermediate vehicle feature map, which is the largest-scale intermediate vehicle feature map.

[0034] Through the aforementioned stepwise fusion process from small to large scales, the Feature Pyramid Network 200 outputs multiple intermediate vehicle feature maps at different scales, such as five scales. Among these, the small-scale intermediate vehicle feature maps contain rich semantic information and are suitable for detecting larger vehicle targets in the image; the large-scale intermediate vehicle feature maps, while preserving detailed information, fuse semantic information from the small scale and are suitable for detecting smaller vehicle targets in the image. These intermediate vehicle feature maps provide a multi-layered foundation for subsequent feature enhancement processing.

[0035] S10 of this embodiment includes the following steps: S13, inputting the intermediate vehicle feature map of each scale into the feature enhancement network, and performing collaborative analysis and fusion of the feature importance of each channel and the vehicle saliency of each spatial location in the intermediate vehicle feature map of that scale to obtain the vehicle feature map corresponding to that scale. Specifically, S13 of this embodiment includes the following steps: Intermediate vehicle feature maps for each scale: Based on the feature dependencies between channels, attention analysis is performed on the intermediate vehicle feature map along the channel dimension to generate channel attention weights that characterize the importance of each channel to the vehicle features. Based on the feature correlation of spatial location, spatial attention analysis is performed on the intermediate vehicle feature map to generate spatial attention weights that characterize the salience of vehicles at each spatial location. The channel attention weight and spatial attention weight are fused to obtain the comprehensive attention weight. The intermediate vehicle feature map is then weighted according to the comprehensive attention weight to obtain the vehicle feature map corresponding to this scale.

[0036] In vehicle detection scenarios, the importance of features extracted by different channels for vehicle recognition varies. Some channels may focus more on vehicle edge contours, others on body color or texture, and still others may respond to background noise. The purpose of channel-dimensional attention analysis is to model the dependencies between channels and generate a weight value for each channel to reflect its importance for feature extraction.

[0037] Please see Figure 3 Specifically, for the intermediate vehicle feature map input to the feature enhancement network 300, a global average pooling (GAP) operation is first performed on it. Global average pooling averages the feature values ​​of all spatial locations in each channel of the intermediate vehicle feature map to obtain a global feature vector. This global feature vector compresses the overall response level of each channel while preserving the global information of the channel dimension. For example, if the input feature map size is [256, 64, 64], after global average pooling, a global feature vector of size [256, 1, 1] is obtained.

[0038] Subsequently, the global feature vector is fed into a Multilayer Perceptron (MLP) for processing. The MLP consists of fully connected layers and can learn the non-linear dependencies between channels, automatically adjusting the importance score of each channel based on the interaction information between them. The MLP ultimately outputs channel attention weights with the same number of input channels, for example, a size of [256, 1, 1]. Channels with larger attention weights indicate that the information contained in that channel is more important for feature extraction, while channels with smaller weights indicate that the information contained in that channel is not important for feature extraction or is background noise.

[0039] In vehicle detection scenarios, the position of a vehicle in an image is not fixed and may appear in any region of the image, while the background may contain interfering information such as roads, buildings, and trees. The purpose of spatial dimension attention analysis is to identify which spatial locations in the intermediate vehicle feature map are more likely to contain a vehicle and assign higher weights to these spatial locations.

[0040] Specifically, max pooling and average pooling operations are performed on the input intermediate vehicle feature map, both along the channel dimension. Max pooling extracts the maximum value of the feature values ​​of all channels at each spatial location, preserving the most significant feature responses; average pooling extracts the average value of the feature values ​​of all channels at each spatial location, preserving global spatial distribution information. Each pooling operation yields an intermediate feature of size [1, H, W]. For example, if the size of the input intermediate vehicle feature map is [256, 64, 64], max pooling and average pooling each yield an intermediate feature of size [1, 64, 64].

[0041] Subsequently, the two intermediate features obtained from max pooling and average pooling operations are concatenated along the channel dimension to obtain a fused feature of size [2, 64, 64]. This fused feature contains both salient feature responses and global spatial distribution information, providing richer spatial feature basis for subsequent spatial attention weight generation.

[0042] Next, the concatenated and fused features are subjected to dynamic convolution. Dynamic convolution uses multiple 1×1 kernels, whose parameters are adaptively updated during model training, capturing feature correlations between different spatial locations. After dynamic convolution, the output is mapped to a range of 0 to 1 using a normalization function (such as the sigmoid function), resulting in spatial attention weights, for example, with a size of [1, 64, 64]. Values ​​close to 1 in these spatial attention weights indicate a high probability that the spatial location contains a vehicle target, with a high vehicle significance; values ​​close to 0 indicate that the spatial location may belong to the background region, with a low vehicle significance.

[0043] Channel attention weights focus on which channels are more important for feature extraction, while spatial attention weights focus on which spatial locations are more likely to contain vehicles. By combining the two, vehicle-related features can be enhanced and irrelevant features suppressed simultaneously from both channel and spatial dimensions.

[0044] Specifically, the channel attention weights and spatial attention weights are fused to obtain a comprehensive attention weight. The channel attention weights have a size of [256, 1, 1], and the spatial attention weights have a size of [1, 64, 64]. Multiplying them element-wise yields a comprehensive attention weight with a size of [256, 64, 64]. Since both the channel and spatial attention weights are normalized, the weights in the fused comprehensive attention weight are also between 0 and 1, enabling it to jointly represent the saliency of vehicles at various spatial locations in the feature map from both the channel and spatial dimensions.

[0045] Subsequently, the intermediate vehicle feature map of the original input is weighted element-wise using a comprehensive attention weighting method, such as element-wise multiplication. During the weighting process, the closer the weight value in the comprehensive attention weight is to 1, the more the corresponding feature value is preserved or amplified; the closer the weight value is to 0, the more the corresponding feature value is suppressed or attenuated. Through this collaborative weighting mechanism, the salient region information related to the vehicle in the feature map is effectively highlighted, while background regions and irrelevant information are effectively suppressed, ultimately yielding a vehicle feature map representing the salient region information of the vehicle at this scale. The size of the vehicle feature map remains consistent with the input feature map, without changing the resolution or number of channels.

[0046] Please see Figure 1 The vehicle detection method based on deformable convolution disclosed in this invention further includes the following steps: S20, inputting the vehicle feature map at each scale into the feature alignment network of the image detection model to obtain the aligned feature map at each scale under preset different anchor boxes. S20 disclosed in this embodiment includes the following steps: S21, for each scale of vehicle feature map: fusing the vehicle feature map at the current scale with all vehicle feature maps at scales smaller than the current scale to obtain a fused feature map. S21 disclosed in this embodiment includes the following steps: Obtain feature maps of all vehicles with a scale smaller than the current scale; Starting with the smallest vehicle feature map, the features are gradually fused to the current scale to obtain a fused feature map.

[0047] Please see Figure 2 and Figure 4 The feature alignment network 400 is used to perform cross-scale fusion and spatial alignment of vehicle feature maps at various scales. Multiple feature alignment networks 400 are cascaded in ascending order of scale, with each feature alignment network 400 corresponding to a vehicle feature map at one scale. In the cascaded structure, the previous level feature alignment network passes the fused feature map to the next level, which then uses it as the feature map to be fused with its own vehicle feature map at its corresponding scale.

[0048] The internal processing flow of each feature alignment network 400 can be divided into three steps: First, channel compression is performed on the received vehicle feature map; then, the compressed vehicle feature map is fused with the features to be fused from the previous stage to obtain a fused feature map; finally, convolution processing is performed on the fused feature map to eliminate the aliasing effect caused by fusion. For example, the number of scales is n, and the number of feature alignment networks is n, where n is an integer greater than 2, such as... Figure 4 As shown, i is an integer greater than 2 and less than n. The specific processing of each level of the feature alignment network 400 is described below in cascade order.

[0049] The first feature alignment network corresponds to the smallest-scale vehicle feature map. Since it's at the front of the cascaded structure and doesn't have a previous feature map to be fused, it directly uses its own vehicle feature map as the fusion feature map. Specifically, the first feature alignment network performs a 1×1 convolution (1×1 Conv) on the input first-scale vehicle feature map, where the first scale is the smallest. The 1×1 convolution here reduces the number of channels in the vehicle feature map, compressing the computational cost of subsequent processing while preserving detailed information. The compressed vehicle feature map serves as the fusion feature map for this feature alignment network, used for subsequent spatial alignment processing. Furthermore, the first feature alignment network also needs to pass the fusion feature map to the second feature alignment network. To do this, the first feature alignment network performs an upsampling operation on the compressed fusion feature map. Upsampling is achieved through linear interpolation, enlarging the spatial size of the fusion feature map to match the second-scale vehicle feature map. Subsequently, before performing the subsequent spatial alignment processing, the fused feature map needs to be processed by 3×3 convolution (3×3 Conv). 3×3 convolution can effectively smooth the transition regions between features from different sources by performing convolution operations on the local neighborhood of each spatial location in the fused feature map, eliminating the spatial discontinuity problem caused by feature splicing, and making the local regions of the fused feature map smoother and more consistent.

[0050] For the 2nd to n-1th feature alignment networks, each feature alignment network 400 receives the feature map to be fused from the previous level and its corresponding vehicle feature map. First, the vehicle feature map is processed by 1×1 convolution. Then, the compressed vehicle feature map is fused with the feature map to be fused to obtain a fused feature map. Then, the fused feature map is upsampled to obtain the corresponding feature to be fused to the next level. Finally, the fused feature map is processed by 3×3 convolution to eliminate the aliasing effect.

[0051] The nth feature alignment network corresponds to the largest-scale vehicle feature map. It is the last stage of the cascaded structure and receives the feature map to be fused from the (n-1)th feature alignment network. Since the nth feature alignment network no longer needs to pass the feature map to be fused downwards, its processing is as follows: first, a 1×1 convolution is performed on the nth-scale vehicle feature map; then, the compressed vehicle feature map is fused with the received feature map to be fused; finally, a 3×3 convolution is performed on the fused result to eliminate aliasing.

[0052] Through the above-mentioned stepwise transfer and fusion from small scale to large scale, the feature alignment network at each scale has obtained a fused feature map that integrates the detailed information and semantic information related to the vehicle at the corresponding scale.

[0053] S20 of this embodiment includes the following steps: S22, the fused feature map is input into deformable convolutions matched with different anchor frames for sampling, so as to obtain the sampling features corresponding to different anchor frames at each spatial location of the fused feature map; wherein, the convolution kernel of the deformable convolution adjusts the sampling offset according to different anchor frames. S22 of this embodiment includes the following steps: The fused feature map is input into the deformable convolution, and the sampling offset of the convolution kernel in the deformable convolution is adjusted for different anchor boxes. Based on the sampling offset corresponding to each anchor frame, the fused feature map is sampled separately to obtain the sampling features corresponding to different anchor frames at each spatial location of the fused feature map.

[0054] Please see Figure 4 In vehicle detection tasks, the object detection module needs to classify and regress the aligned feature map using different preset anchor boxes. However, the sampling area of ​​the convolutional kernel inside the object detection module is fixed, while the preset anchor boxes have different aspect ratios, and the areas covered by each anchor box on the aligned feature map are different. If the features extracted from the fixed sampling area are directly used to provide the prediction basis for all anchor boxes, there will be a spatial deviation between the image content represented by the features and the actual image content covered by each anchor box, i.e., feature misalignment. To eliminate this feature misalignment, the feature alignment network 400 uses deformable convolution (Align Conv) to adjust the sampling area of ​​the convolutional kernel according to the shape of different anchor boxes before feeding the aligned feature map into the object detection module, so that the extracted sampling features are spatially aligned with the coverage area of ​​the corresponding anchor box.

[0055] The difference between deformable convolution and regular convolution lies in the fact that the sampling positions of the convolution kernel are no longer fixed grid points, but can be adaptively adjusted according to the sampling offset field. In Feature Alignment Network 400, the determination of the sampling offset is directly related to the shape of the anchor boxes. For each spatial location of the fused feature map, multiple anchor boxes with different aspect ratios are pre-defined at that location, and each anchor box has known shape information, including parameters such as the width and height of the anchor box.

[0056] When deformable convolution sampling is required on the fused feature map, a set of sampling offsets is determined for each anchor frame shape based on its shape parameters. These sampling offsets adjust the spatial offset direction and magnitude of each preset sampling position of the convolution kernel, ensuring that the adjusted sampling area is spatially aligned with the coverage area of ​​the anchor frame. For example, for a 3×3 convolution kernel, there are originally 9 fixed preset sampling positions evenly distributed in a 3×3 grid. After introducing sampling offsets, these 9 sampling positions are offset in both the horizontal and vertical directions of the image. The magnitude and direction of the sampling offsets are determined by the shape of the corresponding anchor frame. For horizontally longer anchor frames, the horizontal offset is larger, stretching the sampling area horizontally to match the width of the anchor frame; for vertically longer anchor frames, the vertical offset is larger, stretching the sampling area vertically to match the height of the anchor frame. Different anchor frame shapes correspond to different sampling offsets; therefore, for the same fused feature map, deformable convolution requires determining the corresponding sampling offset for each type of anchor frame.

[0057] After determining the corresponding sampling offset for each anchor box, the deformable convolution samples the fused feature map based on each set of sampling offsets. Since different anchor boxes correspond to different sampling offsets, the sampling region of the convolution kernel is adaptively adjusted according to the shape of the corresponding anchor box during each sampling.

[0058] For each sampling, deformable convolution uses the sampling offset corresponding to a certain anchor box to move each sampling position of the convolution kernel according to the offset, forming a new sampling region spatially aligned with the anchor box. Then, convolution sampling is performed on the fused feature map in this new sampling region. Since multiple anchor boxes are preset for each spatial location, deformable convolution needs to perform a sampling operation for each type of anchor box. Each sampling covers all spatial locations of the fused feature map, obtaining the sampling result corresponding to the current anchor box.

[0059] Taking the fused feature map with dimensions [256, 64, 64] and three different aspect ratios preset for each spatial location as an example, the deformable convolution first determines the first set of sampling offsets based on the shape of the first type of anchor map, and samples the fused feature map based on this set of sampling offsets to obtain the sampling features corresponding to the first type of anchor map at each spatial location; then, it determines the second set of sampling offsets based on the shape of the second type of anchor map, and samples the fused feature map again to obtain the sampling features corresponding to the second type of anchor map at each spatial location; finally, it determines the third set of sampling offsets based on the shape of the third type of anchor map, and samples the fused feature map a third time to obtain the sampling features corresponding to the third type of anchor map at each spatial location.

[0060] Through the above processing, multiple sampled features corresponding to different anchor frames are obtained for each spatial location on the fused feature map. These sampled features are spatially aligned with the coverage area of ​​their respective anchor frames, eliminating feature misalignment between the fixed sampled area and the anchor frame coverage area, and providing a foundation for subsequent generation of aligned feature maps by anchor frame type.

[0061] The S20 disclosed in this embodiment includes the following steps: S23, for each anchor frame: combine the sampling features of all spatial positions under the anchor frame to obtain the alignment feature map of the scale under the anchor frame.

[0062] After deformable convolution sampling, each spatial location on the fused feature map obtains multiple sampled features corresponding to different preset anchor boxes. These sampled features are distributed across various spatial locations, and each spatial location stores sampled features corresponding to each type of anchor box.

[0063] For an anchor box of a specific shape, each spatial location in the fused feature map has a corresponding sampled feature. This sampled feature is obtained by deformable convolution sampling based on the sampling offset of the anchor box. Since the sampled features at each spatial location are already spatially aligned with the current anchor box, these sampled features are arranged according to their respective spatial locations, i.e., maintaining their original spatial coordinate relationships on the fused feature map. By combining the sampled features from all spatial locations, a complete aligned feature map is formed. This aligned feature map is the aligned feature map of that scale for that anchor box. The spatial dimensions of the aligned feature map are consistent with the input fused feature map.

[0064] Since the number of pre-defined anchor frames at each spatial location determines the number of alignment feature maps, for each scale of the fused feature maps, the final result will be multiple alignment feature maps equal to the number of pre-defined anchor frames at that scale. For example, if three different aspect ratios of anchor frames are pre-defined at each spatial location, then three alignment feature maps will be generated at each scale, each corresponding to an anchor frame of a certain shape. The first alignment feature map is composed of the sampled features corresponding to the first-shaped anchor frame at all spatial locations, the second alignment feature map is composed of the sampled features corresponding to the second-shaped anchor frame at all spatial locations, and the third alignment feature map is composed of the sampled features corresponding to the third-shaped anchor frame at all spatial locations.

[0065] Through the above combined processing, each aligned feature map corresponds to an anchor box of a certain shape, and its sampled features are spatially aligned with the coverage area of ​​the anchor box, eliminating feature misalignment between the fixed sampling area and the anchor box coverage area. When the subsequent object detection module processes each aligned feature map, it can perform more accurate foreground-background classification and position regression based on the spatially aligned features, thereby improving the quality of candidate region generation and ultimately enhancing the accuracy of vehicle detection.

[0066] Please see Figure 1 , Figure 2 and Figure 5 The vehicle detection method based on deformable convolution disclosed in this invention further includes the following steps: S30, inputting all aligned feature maps under all anchor boxes at each scale into the target detection module of the image detection model to generate candidate regions corresponding to each anchor box, filtering and deduplicating the candidate regions, and performing category determination and position adjustment on the retained candidate regions to obtain the region and category corresponding to the vehicle in the vehicle image; wherein, the target detection module includes a region proposal network 500, a classification branch 600 and a regression branch 700.

[0067] The S30 disclosed in this embodiment includes the following steps: S31, inputting all aligned feature maps under all anchor boxes at each scale into the region proposal network of the target detection module of the image detection model to obtain the corresponding candidate regions.

[0068] The Region Proposal Network 500 is a neural network based on the Faster R-CNN model. It processes each aligned feature map separately. Each aligned feature map corresponds to a specific shape of anchor box. The Region Proposal Network 500 performs a convolution operation on the corresponding aligned feature map using the anchor box of that specific shape, outputting the corresponding foreground confidence and a first position offset. Based on the first position offset, it adjusts the position of the corresponding anchor box to obtain candidate regions. Since multiple anchor boxes of different shapes are preset, multiple aligned feature maps are generated. After processing by the Region Proposal Network 500, each aligned feature map outputs a set of candidate regions.

[0069] The S30 disclosed in this embodiment also includes the following step: S32, screening all candidate regions by using a non-maximum suppression method.

[0070] The screening process begins by assessing the confidence level of each candidate region output by the Region Proposal Network 500 (RPN 500). The confidence level reflects the RPN 500's belief that the candidate region contains a vehicle. All candidate regions are sorted from highest to lowest confidence, retaining those with higher confidence. This is done to quickly eliminate low-confidence candidate regions, which are likely background or targetless areas. Retaining them does not help subsequent detection and only increases computational cost.

[0071] Even after confidence-based filtering, overlapping may still exist among the remaining candidate regions. The same vehicle target may be covered by multiple high-confidence candidate regions simultaneously, with a high degree of overlap between these regions. To further refine the candidate regions, a non-maximum suppression method is used to remove duplicates from the filtered candidate regions.

[0072] The specific processing procedure of the non-maximum suppression method is as follows: From the filtered candidate regions, the candidate region with the highest current confidence is selected as the benchmark, and the intersection-union ratio (IUR) between the remaining candidate regions and the benchmark candidate region is calculated. The IUR measures the degree of overlap between two candidate regions; a larger value indicates more severe overlap. If the IUR of a candidate region with the benchmark candidate region exceeds a preset threshold, it is considered that the candidate region and the benchmark candidate region cover the same vehicle target, belonging to redundant candidate regions, and is suppressed, i.e., removed from the candidate region set. Subsequently, the above process of calculating the IUR and suppression is repeated until all candidate regions have been processed. After non-maximum suppression, each retained candidate region corresponds to one vehicle target.

[0073] The S30 disclosed in this embodiment further includes the following step: S33 uses the classification branch and regression branch of the object detection module to determine the category and adjust the position of the retained candidate region, respectively, to obtain the region and category of the vehicle in the vehicle image. Specifically, S33 disclosed in this embodiment includes the following steps: The retained candidate regions are input into the classification branch to extract category-related information and obtain the vehicle category in the vehicle image; The retained candidate regions are input into the regression branch to extract information related to the vehicle's position, obtain the position offset, and adjust the position of the candidate regions based on the position offset to obtain the vehicle region in the vehicle image.

[0074] Classification branch 600 is used to determine the specific category of vehicles within the candidate region, such as cars, trucks, and buses. Before inputting the candidate region into classification branch 600, the area covered by the candidate region can be filtered out from the aligned feature map based on the coordinate position of the candidate region on the aligned feature map, and uniformly adjusted to a fixed size, such as 7×7 pixels, with the number of channels consistent with the aligned feature map.

[0075] The classification branch 600's internal network layers include convolutional layers, non-linear activation layers, and fully connected layers. First, 1×1 convolutional layers (1×1 Conv) are used to reduce the channel dimension and fuse features in candidate regions. 1×1 convolutions can efficiently fuse features from different channels while preserving key semantic information, reducing the number of parameters and computational cost in subsequent processing. Then, the output of the convolutional layers is processed by non-linear activation functions. These non-linear activation functions introduce non-linear expressive power to the features, enabling classification branch 600 to learn more complex class-discriminating features and enhancing its ability to distinguish between different classes. Commonly used non-linear activation functions include ReLU. Next, the output of the non-linear activation layers is fed into fully connected layers (FC), which map the features into a class confidence vector, where each element corresponds to a predefined class. Finally, the class confidence vector is normalized to a probability distribution using the Softmax function, where each element's value is between 0 and 1 and sums to 1, representing the probability that the candidate region belongs to each class. The class with the highest probability is selected as the vehicle class for that candidate region. Through the processing of classification branch 600, each retained candidate region is assigned a specific category, such as car, truck, or bus.

[0076] Regression branch 700 is used to fine-tune the position of the candidate region, so that the final output region is closer to the actual boundary of the vehicle.

[0077] The network layers within regression branch 700 include convolutional layers, normalization layers, and fully connected layers. First, candidate regions are processed using 3×3 convolutional layers (3×3 Conv). The 3×3 convolution has a large receptive field, enabling the extraction of local features related to the positional boundaries of the candidate regions, providing a basis for positional adjustment. Then, the output of the convolutional layers is normalized using a batch normalization layer (BN). This batch normalization technique accelerates convergence during training and stabilizes the numerical distribution, making the predictions of regression branch 700 more stable and reliable. Next, the output of the normalization layer is fed into a fully connected layer (FC), which maps the features output by the normalization layer to positional offsets. These positional offsets can include four parameters: adjustments to the x-coordinate of the candidate region's center point, the y-coordinate of the center point, the width of the candidate region, and the height of the candidate region. These positional offsets are adjustment parameters relative to the position of the candidate region.

[0078] Based on the position offset output by the regression branch, the current coordinates of the candidate region are adjusted. The adjusted position is the final vehicle region corresponding to the candidate region. This region is finely modified based on the candidate region, which can more accurately define the boundary of the vehicle.

[0079] Through parallel processing of classification branch 600 and regression branch 700, each retained candidate region acquires the vehicle category and the vehicle region. When the category confidence of a vehicle's region is higher than a threshold, the vehicle is considered to exist, and the corresponding vehicle region and vehicle category are retained; those below the threshold are discarded. Finally, the regions and categories corresponding to the vehicles in the vehicle image are obtained.

[0080] The image detection model described above is trained and updated based on a loss function. The formula for calculating the loss function is as follows:

[0081] in, For the total loss, , These are L1 loss and focus loss, respectively. , These are the weighting coefficients.

[0082] The formula for calculating L1 loss is as follows:

[0083] in, , For the first Location information of vehicle regions in the vehicle images used for training. For the first The coordinates of the center point of the vehicle region in the training vehicle images. For the first Width and height of the vehicle region in the vehicle images used for training; This provides the predicted location information for the vehicle's area.

[0084] The formula for calculating the loss is as follows:

[0085] in, This represents the total number of vehicle images used for training. This is a balancing factor (e.g., 0.25 or 0.75). For the first The difference between the true category and the predicted category of a vehicle in the training vehicle image. For example, for a candidate region, its true category is represented in one-hot encoding form, such as (sedan, truck, bus) = (1, 0, 0), which means that the true category of the vehicle in the candidate region is a sedan. The predicted category distribution output by the classification branch is (0.75, 0.2, 0.05), which means that the probability of predicting it as a sedan is 0.75, the probability of predicting it as a truck is 0.2, and the probability of predicting it as a bus is 0.05. Then the difference is 0.25. This is the focusing parameter (e.g., take 2).

[0086] As can be seen, in the above scheme, by using deformable convolution to adjust the sampling area of ​​the convolution kernel according to the shape of different anchor boxes before the features enter the region proposal network, spatially aligned feature maps are generated for each anchor box, eliminating the feature misalignment between the fixed sampling area of ​​the convolution kernel and the coverage area of ​​the multi-shaped anchor boxes in the region proposal network. Based on this, the region proposal network performs anchor box classification and position regression based on the aligned feature maps, resulting in more accurate candidate regions and thus improving the localization accuracy of vehicle detection. Simultaneously, by performing category determination and fine-tuning of the position of candidate regions through classification and regression branches respectively, the classification accuracy for different vehicle models in complex scenarios is further improved, effectively enhancing detection accuracy and robustness.

[0087] Please see Figure 6 The present invention also provides a vehicle detection device based on deformable convolution, which can apply the above-mentioned vehicle detection method based on deformable convolution. The vehicle detection device based on deformable convolution includes a feature extraction unit 810, a feature alignment unit 820, and a target detection unit 830.

[0088] The feature extraction unit 810 is used to input vehicle images into the feature extraction module of a pre-trained image detection model to obtain vehicle feature maps at different scales.

[0089] The feature alignment unit 820 is used to input the vehicle feature map at each scale into the feature alignment network of the image detection model to obtain the alignment feature map of each scale under different preset anchor boxes. Specifically, for each scale of vehicle feature map: the vehicle feature map at the current scale is fused with all vehicle feature maps at scales smaller than the current scale to obtain a fused feature map. The fused feature map is then input into deformable convolutions matched with different anchor boxes for sampling to obtain the sampling features corresponding to different anchor boxes at each spatial location of the fused feature map. The convolution kernel of the deformable convolution adjusts the sampling offset according to different anchor boxes. For each anchor box: the sampling features of all spatial locations under the anchor box are combined to obtain the alignment feature map of that scale under that anchor box.

[0090] The target detection unit 830 is used to input all aligned feature maps under all anchor boxes at all scales into the target detection module of the image detection model, generate candidate regions corresponding to each anchor box, filter and remove duplicates from the candidate regions, and perform category determination and position adjustment on the retained candidate regions to obtain the region and category corresponding to the vehicle in the vehicle image.

[0091] Specific limitations regarding the vehicle detection device based on deformable convolution can be found in the limitations of the vehicle detection method based on deformable convolution mentioned above, and will not be repeated here. Each module in the aforementioned vehicle detection device based on deformable convolution can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the memory in the electronic device, or stored in software form in the memory of the electronic device, so that the memory can call and execute the operations corresponding to each module.

[0092] Please see Figure 7 The electronic device 900 may include a memory 910, a processor 920, and a bus, and may also include a computer program stored in the memory 910 and executable on the processor 920, such as a program for a vehicle detection method based on deformable convolution.

[0093] The memory 910 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 910 can be an internal storage unit of the electronic device 900, such as the portable hard drive of the electronic device 900. In other embodiments, the memory 910 can also be an external storage unit of the electronic device 900, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 900. Furthermore, the memory 910 can include both internal and external storage units of the electronic device 900. The memory 910 can be used not only to store application software and various types of data installed on the electronic device 900, such as code for a vehicle detection method based on deformable convolution, but also to temporarily store data that has been output or will be output.

[0094] In some embodiments, the processor 920 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 920 is the control unit of the electronic device 900, connecting various components of the electronic device 900 via various interfaces and lines. It executes programs or modules stored in the memory 910 (e.g., programs for vehicle detection methods based on deformable convolution) and calls data stored in the memory 910 to perform various functions and process data of the electronic device 900.

[0095] The processor 920 executes the operating system of the electronic device 900 and various installed applications. The processor 920 executes the applications to implement the steps in the above-described vehicle detection method based on deformable convolution.

[0096] A computer program can be divided into one or more modules. One or more modules are stored in memory 910 and executed by processor 920 to complete this application. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in electronic device 900. For example, the computer program can be divided into a feature extraction unit 810, a feature alignment unit 820, a target detection unit 830, etc.

[0097] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A deformable convolution based vehicle detection method, characterized in that, include: Vehicle images are input into the backbone network of the feature extraction module of a pre-trained image detection model to obtain initial vehicle feature maps at different scales. Initial vehicle feature maps at different scales are input into the feature pyramid network of the feature extraction module, and feature fusion is performed step by step from small scale to large scale to obtain intermediate vehicle feature maps at different scales. The intermediate vehicle feature maps at each scale are input into the feature enhancement network of the feature extraction module. The feature importance of each channel and the salience of vehicles at each spatial location in the intermediate vehicle feature map at that scale are analyzed and fused together to obtain the vehicle feature map corresponding to that scale. The vehicle feature map at each scale is input into the feature alignment network of the image detection model to obtain the aligned feature map at each scale under different preset anchor boxes; wherein, for the vehicle feature map at each scale: The current scale vehicle feature map is fused with all vehicle feature maps of smaller scales to obtain a fused feature map. The fused feature map is input into a deformable convolution, and the sampling offset of the convolution kernel in the deformable convolution is adjusted for different anchor frames; wherein, the sampling offset of the convolution kernel of the deformable convolution is adjusted according to different anchor frames. Based on the sampling offset corresponding to each anchor frame, the fused feature map is sampled separately to obtain the sampling features corresponding to different anchor frames at each spatial location of the fused feature map; For each anchor frame: combine the sampled features of all spatial locations under the anchor frame to obtain the alignment feature map of that scale under the anchor frame; All aligned feature maps under all anchor boxes at each scale are input into the region proposal network of the target detection module of the image detection model to obtain the corresponding candidate regions. All candidate regions are filtered using a nonmaximum suppression method; The classification and regression branches of the target detection module are used to determine the category and adjust the position of the retained candidate regions, respectively, to obtain the region and category of the vehicle in the vehicle image.

2. The deformable convolution-based vehicle detection method of claim 1, wherein, The intermediate vehicle feature maps at each scale are input into the feature enhancement network of the feature extraction module. The feature enhancement network then performs a collaborative analysis and fusion of the feature importance of each channel and the salience of vehicles at each spatial location within the intermediate vehicle feature map at that scale, resulting in the vehicle feature map corresponding to that scale. This includes: Intermediate vehicle feature maps for each scale: Based on the feature dependencies between channels, attention analysis is performed on the intermediate vehicle feature map along the channel dimension to generate channel attention weights that characterize the importance of each channel to the vehicle features. Based on the feature correlation of spatial location, spatial attention analysis is performed on the intermediate vehicle feature map to generate spatial attention weights that characterize the salience of vehicles at each spatial location. The channel attention weight and the spatial attention weight are fused to obtain a comprehensive attention weight. The intermediate vehicle feature map is then weighted according to the comprehensive attention weight to obtain the vehicle feature map corresponding to that scale.

3. The deformable convolution-based vehicle detection method of claim 1, wherein, The step of fusing the vehicle feature map at the current scale with all vehicle feature maps at scales smaller than the current scale to obtain a fused feature map includes: Obtain feature maps of all vehicles with a scale smaller than the current scale; Starting with the smallest vehicle feature map, the features are gradually fused to the current scale to obtain a fused feature map.

4. The deformable convolution-based vehicle detection method of claim 1, wherein, The process involves using the classification and regression branches of the target detection module to determine the category and adjust the position of the retained candidate regions, respectively, to obtain the region and category of the vehicle in the vehicle image, including: The retained candidate regions are input into the classification branch, and category-related information is extracted to obtain the category of the vehicle in the vehicle image; The retained candidate regions are input into the regression branch to extract information related to the vehicle's position, obtain the position offset, and adjust the position of the candidate regions according to the position offset to obtain the region of the vehicle in the vehicle image.

5. A deformable convolution-based vehicle detection apparatus, characterized by, The vehicle detection device, using the vehicle detection method as described in any one of claims 1 to 4, comprises: The feature extraction unit is used to input vehicle images into the feature extraction module of a pre-trained image detection model to obtain vehicle feature maps at different scales. The feature alignment unit is used to input the vehicle feature map at each scale into the feature alignment network of the image detection model to obtain the aligned feature map at each scale under different preset anchor boxes; wherein, for the vehicle feature map at each scale: The current scale vehicle feature map is fused with all vehicle feature maps of smaller scales to obtain a fused feature map. The fused feature map is input into deformable convolutions that match different anchor frames for sampling, so as to obtain the sampling features corresponding to different anchor frames at each spatial location of the fused feature map; wherein, the convolution kernel of the deformable convolution adjusts the sampling offset according to different anchor frames; For each anchor frame: combine the sampled features of all spatial locations under the anchor frame to obtain the alignment feature map of that scale under the anchor frame; The target detection unit is used to input all aligned feature maps under all anchor boxes at various scales into the target detection module of the image detection model, generate candidate regions corresponding to each anchor box, filter and deduplicate the candidate regions, and perform category determination and position adjustment on the retained candidate regions to obtain the region and category corresponding to the vehicle in the vehicle image.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the vehicle detection method based on deformable convolution as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the vehicle detection method based on deformable convolution as described in any one of claims 1 to 4.

Images