A method and device for detecting small targets on a highway based on a convolutional neural network

Abstract

The application discloses a kind of based on convolutional neural network's highway small target detection method and device, obtains the highway image training data set with labeled, adopts local pasting strategy to carry out data enhancement to image training data set, input to backbone network and train, respectively through layer attention again fusion feature and double branch prediction result, using the network of trained detection to be detected image.This application adopts the target detector of multiple modules to detect small target, can effectively predict small target in image.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method and apparatus for detecting small targets on highways based on convolutional neural networks. Background Technology

[0002] Small target detection has long been a challenging task, with wide applications in fields including surveillance, pedestrian detection, garbage identification, and satellite remote sensing. With the rise of convolutional neural networks, small target detection has achieved unprecedented progress in both accuracy and speed; however, its current development level is still far from meeting our expectations.

[0003] The small target is defined on the COCO dataset as less than... Small objects, compared to medium or large objects, are challenging to detect primarily because they contain too few discriminative features. After downsampling, small objects often have only 4-5 pixels or less in the feature map. These features are usually mixed with background information, failing to provide enough features for effective network learning. Furthermore, in some anchored object detection models, small object instances are too small to match pre-set anchor boxes and are incorrectly classified as negative samples. This has no positive effect on network learning and instead inhibits network convergence. Finally, in many datasets, small objects are also at a disadvantage in terms of quantity. The imbalanced sample size leads to a situation where, in order to minimize the overall loss, a large number of samples dominate the network's gradient descent, causing the model to neglect smaller sample classes and exacerbating the difficulty of detecting small objects. Summary of the Invention

[0004] The purpose of this application is to provide a method and apparatus for detecting small targets on highways based on convolutional neural networks. A new framework for small target detection is proposed in the existing technical solutions. Combined with our proposed highway dataset, it overcomes the problem of the difficulty in detecting small targets.

[0005] To achieve the above objectives, the technical solution of this application is as follows:

[0006] A method for detecting small targets on highways based on convolutional neural networks, comprising:

[0007] Step 1: Obtain an image training dataset with labeled small target detection boxes for highways, and perform data augmentation on the image training dataset. A local pasting strategy is used to directly crop the small targets. After a series of operations including rotation, scaling, and affine transformation, the small targets are then displayed. center , When defining an area using coordinates as its center, The first one after random transformation of the master version One copy instance With small target instance set When the areas do not intersect, The defined region in the original image is pasted back using Poisson blending and simultaneously incorporated into the set. middle;

[0008] Step 2: Pad the image training dataset to a preset size and input it into the ResNet-50-FPN network to output feature maps of four different sizes. , , , ;

[0009] Step 3: Transfer the feature map , , , Feature maps are obtained by performing feature re-fusion through layer attention. , , , ,include:

[0010] First, the two layers to be merged and After aligning to a uniform size, they are stitched together. , here The first output from FPN layer, represent The next layer has feature maps that are half the size of the previous layer, so aligning them requires... Upsampling is performed here using bilinear interpolation. Upsampling is achieved by aligning the samples to the same size and then stitching them together. , here and These are the height and width of the feature map, respectively. The number of channels in the feature map;

[0011] Subsequently Adaptive 2D average pooling, yielding one feature value per channel. There are a total of These feature values ​​are used to perform a fully connected operation on the values ​​to obtain the weights of the two layers. , :

[0012] To achieve the spliced ​​features Dimensionality reduction and weights Reassign to two feature layers and set up a dimensionality-reduced convolution. weight Multiplying by this dimensionality-reducing convolution, the returned parameters are then concatenated with the features. Perform matrix multiplication and output the final result. and before fusion Same size:

[0013] Step 4: Process the feature map , , , To perform bounding box regression and class classification prediction operations, a two-branch prediction module was designed. This module consists of two branches: a classification head and a regression head. First, it processes the features output from the previous step. Using two non-parameter-shared convolutions and Extracting classification features separately and regression features Then, convolution is used in the regression branch to obtain the predicted target bounding boxes. and spatial probability graph And simultaneously learn the bounding box offset. Variable convolutions are used for subsequent bounding boxes; the classification branch uses fully connected layers. Bundle The channel is reduced to dimensionality , here Knowing the number of categories, we can obtain the logical score of the target category contained within the features of the corresponding current region. Combine the logical score with the previously obtained spatial probability. After normalization, multiplication yields the final classification result. ;

[0014] Step 5: Calculate the bounding box loss and classification loss, perform backpropagation to reduce the loss, and update the network parameters at the same time;

[0015] Step 6: For the unlabeled and unenhanced test image to be detected, perform steps 2 to 4 to obtain the detection results.

[0016] Furthermore, data augmentation is performed on the image training dataset. A local pasting strategy is employed, directly cropping small targets. After a series of operations including rotation, scaling, and affine transformation, the pasting area is defined according to the following formula:

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] in, , They represent the regional boundaries, , The coordinates of the top-left corner of the rectangle's boundary. , The coordinates of the bottom right corner of the rectangle's boundary. , These are the center coordinates of the small target. , It's about the width and height of small goals. That is the magnification factor of the region.

[0022] Furthermore, the highway small target detection method based on convolutional neural networks includes concatenating adjacent feature layers and assigning higher weights to the feature layer containing the small target.

[0023] Furthermore, the highway small target detection method based on convolutional neural networks includes: performing bounding box regression and category classification prediction operations on the feature map, which is divided into two branches: a regression head and a classification head. The regression head uses convolution... After obtaining the bounding box prediction, the classification head is used with a fully connected layer to obtain the classification prediction.

[0024] This application also proposes a small target detection device for highways based on convolutional neural networks, including a data augmentation module. It employs a local pasting strategy to directly crop the small target, and after a series of operations such as rotation, scaling, and affine transformation, the small target is then displayed. center , When defining an area using coordinates as its center, For the first version of the master copy after random transformation One copy instance With small target instance set When the areas do not intersect, The defined region in the original image is pasted back using Poisson blending and simultaneously incorporated into the set. middle;

[0025] Layer attention module, which processes feature maps , , , By performing feature refusion processing, feature maps are obtained. , , , Perform the following operations:

[0026] Adjacent two layers and After aligning to the same size using bilinear interpolation, they are concatenated. ,right Perform adaptive 2D average pooling to obtain the feature value of the number of channels. Through full connectivity Dimension reduction yields two weights , Finally, these two weights , Assigned to and Two layers, yielding the final output. ;

[0027] The dual-branch prediction module performs a feature map analysis. , , , To perform bounding box regression and class classification operations, a Double Head prediction module was designed. This module consists of two branches: a regression head and a classification head. The regression head uses convolution to obtain the predicted bounding boxes of the target objects. and spatial probability graph Simultaneously learn bounding box offsets Used for subsequent variable convolutions on the bounding boxes; the classification head uses fully connected layers. Bundle The channel is reduced to dimensionality , here Knowing the number of categories, we can obtain the logical score of the target category contained within the features of the corresponding current region. Combine the logical score with the previously obtained spatial probability. After normalization, multiplication yields the final classification result. ;

[0028] The parameter update module calculates the bounding box loss and classification loss, performs backpropagation to reduce the loss, and updates the network parameters at the same time.

[0029] The detection module is used to input the image to be detected into the feature extraction module, the layer attention module, and the bi-branch prediction module to obtain the detection result.

[0030] Furthermore, data augmentation is performed on the image training dataset. A local pasting strategy is employed, directly cropping small targets. After a series of operations including rotation, scaling, and affine transformation, the pasting area is defined according to the following formula:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] in, , They represent the regional boundaries, , The coordinates of the top-left corner of the rectangle's boundary. , The coordinates of the bottom right corner of the rectangle's boundary. , These are the center coordinates of the small target. , It's about the width and height of small goals. That is the magnification factor of the region.

[0036] Furthermore, the highway small target detection method based on convolutional neural networks includes concatenating adjacent feature layers and assigning higher weights to the feature layer containing the small target.

[0037] Furthermore, the highway small target detection method based on convolutional neural networks includes: performing bounding box regression and category classification prediction operations on the feature map, which is divided into two branches: a regression head and a classification head. The regression head uses convolution... After obtaining the bounding box prediction, the classification head is used with a fully connected layer to obtain the classification prediction.

[0038] The display module is used to draw small target detection boxes on the image to be detected to indicate the location and category information of the target in the image. Attached Figure Description

[0039] Figure 1 This is a thumbnail of the highway dataset on which this application is based;

[0040] Figure 2 This is a flowchart of the small target detection method for highways based on convolutional neural networks proposed in this application;

[0041] Figure 3 This is a schematic diagram of the attention module. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] This application presents a method for detecting small targets on highways based on convolutional neural networks, which mainly includes the following three steps: First, image data augmentation is performed using a local pasting strategy before training. Then, training begins in batches, with each batch of images processed through a convolutional neural network to obtain feature maps. These feature maps are then regressed to obtain predicted values, which are compared with the true values ​​of the images to calculate the loss. After each batch of training, backpropagation is performed to reduce the loss and update the network parameters. Finally, after all training batches are completed, the pre-trained parameters are loaded into the network to detect unlabeled images. The resulting predicted values ​​are the model's predictions for the images.

[0044] Reference Figure 1 The dataset in this application focuses on small targets within a highway environment. The images were collected from surveillance cameras along a section of a highway in China. After manual selection and annotation, a total of 931 images were obtained, containing 1458 target instances, including 700 small targets, 527 medium targets, and 231 large targets. The images are divided into two parts: a training set and a test set. The training set consists of 651 labeled images, and the test set consists of 280 unlabeled images, used to test the model's generalization ability. Figure 1 As shown, in order to achieve the goal of intelligent road safety management, this dataset is mainly used to detect three types of small target instances on highways: plastic bags, roadblocks, and cardboard boxes. These categories are potential threats to safe driving and require real-time monitoring and intelligent feedback.

[0045] Reference Figure 2 A method for detecting small targets on highways based on convolutional neural networks includes the following steps:

[0046] S1. Obtain an image training dataset with labeled small target detection boxes for highways, and perform data augmentation on the image training dataset using a local pasting strategy. Each image in the obtained image training dataset is labeled with a small target detection box. First, the small targets are cropped, followed by a series of operations including rotation, scaling, and affine transformation; these will not be detailed here.

[0047] This application aims for a small goal. center , Define the area to be pasted in the image with the center as the center:

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] in, , They represent the regional boundaries, , The coordinates of the top-left corner of the rectangle's boundary. , The coordinates of the bottom right corner of the rectangle's boundary. , These are the center coordinates of the small target. , It's about the width and height of small goals. That is the magnification factor of the region.

[0053] At the same time The first version is obtained after the above random transformation of the master version. One copy instance When the copied instance is in the same set as the small target instance When the areas do not intersect:

[0054] ;

[0055] Bundle The defined region in the original image is pasted back using Poisson blending and simultaneously incorporated into the set. In the process, this operation is repeated until the number of small targets pasted in each image reaches the preset value, and finally the labels are updated.

[0056] S2. Pad the image training dataset to a preset size and input it into the ResNet-50-FPN network, outputting feature maps of four different sizes. , , , ;

[0057] This application employs batch training, with a total of 50 batches. During training, the batch size is 2 (i.e., each batch processes 2 images), and the learning rate starts at 0.01. The first two batches use a warm-up method for preheating, and the learning rate is updated using one-dimensional linear interpolation for each batch. After the warm-up phase, the learning rate is set to 0.005. Subsequently, after the 30th and 40th batches, the learning rate is multiplied by 0.1 for adjustment, and stochastic gradient descent is used to guide the loss gradient descent.

[0058] Because the original image was not of standard size, this application scales the original image according to a scaling factor, etc. Then use Fill to The padded image is input into the ResNet-50-FPN network, and after a series of convolutional operations, it is output sequentially starting from a 4x downsampling stride. , , and Four sizes of feature maps , , , The size of the feature map is determined by the ResNet-50-FPN network, which will not be elaborated here.

[0059] S3. Reference Figure 3 The layer attention module includes the following steps:

[0060] feature map , , , Feature processing is performed to obtain feature maps. , , , ,include:

[0061] After aligning the two layers to be merged to the same size, they are spliced ​​together. :

[0062] ;

[0063] in The first output from FPN layer, represent The next layer has feature maps that are half the size of the previous layer, so upsampling is needed to align them. Here, we use bilinear interpolation. After aligning them to the same size, they are spliced ​​together. Operation obtained .

[0064] Subsequently Adaptive Average pooling yields a 1×1 feature value for each channel. There are a total of ×1 such feature values, and perform... → The fully connected operation yields the weights of the two layers:

[0065] ;

[0066] here These are the feature values ​​after pooling. Average pooling refers to taking a single image... The average value of the image. and There are two fully connected layers, which respectively implement the conversion of the original channel from Reduce the dimension to 32, and then reduce the dimension from 32 to 2. and These represent the sigmoid and ReLU activation functions, respectively, which will not be elaborated upon here.

[0067] In order to achieve weight Reassigned to two feature layers while simultaneously processing the concatenated features Dimensionality reduction, set a dimensionality-reducing convolution. weight Multiplying by this dimensionality-reducing convolution returns parameters Then, the spliced ​​features Perform matrix multiplication:

[0068] ;

[0069] Here Represents dimensionality reduction convolution. Represents matrix multiplication. This indicates that group normalization normalizes the parameters, and the final output result is... , and before fusion They are all the same size.

[0070] S4. Feature Map , , , To perform bounding box regression and class classification prediction operations, a DoubleHead prediction module was designed. This module consists of two branches: a regression head and a classification head. The regression head uses convolution to obtain the predicted target bounding boxes. and spatial probability graph Simultaneously learn bounding box offsets Used for subsequent variable convolutions on the bounding boxes; the classification head uses fully connected layers. The Channel dimensionality reduction to , here Knowing the number of categories, we can obtain the logical score of the target category contained within the features of the corresponding current region. Combine the logical score with the previously obtained spatial probability. After normalization, multiplication yields the final classification result. ;

[0071] First, the features output from the layer attention module are split:

[0072] ;

[0073] Here, two non-parameter-shared convolutions are used. and From respectively Output to extract classification features and regression features , This represents the ReLU activation function, which will not be elaborated upon here.

[0074] Then, in the regression branch, we use three output channels with a step size of 2. Convolutions with 4, 1, and 9 respectively yielded the predicted target bounding boxes. Spatial probability graph And the bounding box offset used for subsequent variable convolution of the bounding box. :

[0075] ;

[0076] Here Representative at the Feature offset of each channel A point in a spatial location. The bounding box is adjusted using bilinear interpolation as a learnable parameter, and the feature point sampling is further refined using variable convolution to obtain the final target bounding box. Here, 4 represents the coordinates of the top left and bottom right corners of the bounding box: x1y2x2y2.

[0077] Categorization branches use full connections From the channel Dimensional reduction , here Knowing the number of categories, we can obtain the logical score of the target category contained within the features of the corresponding current region. :

[0078] ;

[0079] Here It is the final classification result, derived from spatial probability. with logical fractions Normalized multiplication followed by root extraction yields the result. This represents the sigmoid normalization operation.

[0080] S5. Calculate the bounding box loss and classification loss, perform backpropagation to reduce the loss, and update the network parameters simultaneously.

[0081] The loss is calculated based on the bounding box and classification prediction (bx1by1bx2by2bc) obtained above, and the label (Bx1By1Bx2By2Bc) obtained in the first step after data augmentation.

[0082] 1. Calculate the regression loss. The Intersection of Union (IoU) is calculated using the predicted bounding box information and the ground truth bounding box information obtained from the labels. IoU is the intersection-union ratio between the predicted and ground truth bounding boxes. The predicted bounding box with the higher IoU value is obtained through NMS post-processing.

[0083] ;

[0084] in For ground truth, For the prediction box, This represents the area where the ground truth bounding box and the predicted bounding box intersect. The area of ​​the combined ground truth bounding box and the predicted bounding box. A higher value indicates a more accurate prediction.

[0085] The loss function is in Based on the previous version, the improved version achieves better convergence during training.

[0086] ;

[0087] in It is the smallest closure rectangle region between the ground truth bounding box and the predicted bounding box.

[0088] 2. Calculate the classification loss using the Binary Cross Entropy Loss (BCE) function:

[0089] ;

[0090] in It is the true category of the sample. It is the predicted category of the sample.

[0091] It should be noted that the calculation of bounding box loss and classification loss is a relatively mature technology in this field, and will not be elaborated here.

[0092] This yields the loss between the predicted and actual values. Before the end of each batch, backpropagation is performed to reduce the loss. Simultaneously, the network parameters are updated, and training for the next batch begins, continuing until all batches of training data have been completed, resulting in a trained backbone network. After all batches of training are finished, all updated parameters are saved in the pth weight file. Before detection begins, simply replace the original weights with the updated weights and repeat training steps S2 to S4.

[0093] Step S6: For the image to be detected, execute steps S2 to S4 to obtain the detection result.

[0094] This application also scales up and fills the image to be detected. The input network takes four sizes of features and outputs feature maps through a series of convolutional operations. Regression and classification of these feature values ​​yields the predicted values, including the category. The bounding box predictions bx1by1bx2by2 are directly output as the detection result without further loss calculation.

[0095] In one embodiment, this application also proposes a highway small target detection device based on a convolutional neural network, comprising:

[0096] Data augmentation module. Employing a local paste strategy, it directly crops small targets, and after a series of operations including rotation, scaling, and affine transformation, it restores the small target to its original state. center , When defining an area using coordinates as its center, For the first version of the master copy after random transformation One copy instance With small target instance set When the areas do not intersect, The defined region in the original image is pasted back using Poisson blending and simultaneously incorporated into the set. middle;

[0097] The feature extraction module pads the image training dataset to a preset size, inputs it into the ResNet-50-FPN network, and outputs feature maps of four different sizes. , , , ;

[0098] Layer attention module, which processes feature maps , , , Feature refusion is performed to obtain feature maps. , , , Perform the following operations:

[0099] Adjacent two layers , After aligning to the same size using bilinear interpolation, they are concatenated. ,right Perform adaptive 2D average pooling to obtain the feature value of the number of channels. Through full connectivity Dimension reduction yields two weights , Finally, these two weights , Assigned to , Two layers, yielding the final output. ;

[0100] The dual-branch prediction module performs a feature map analysis. , , , To perform bounding box regression and class classification operations, a Double Head prediction module was designed. This module consists of two branches: a regression head and a classification head. The regression head uses convolution to obtain the predicted bounding boxes of the targets. and spatial probability graph Simultaneously learn bounding box offsets Used for subsequent variable convolutions on the bounding boxes; the classification head uses fully connected layers. Bundle The channel is reduced to dimensionality Here, given the number of categories, we can obtain the logical score of the target category contained within the features of the corresponding current region. Combine the logical score with the previously obtained spatial probability. After normalization, multiplication yields the final classification result. ;

[0101] The parameter update module calculates the bounding box loss and classification loss, performs backpropagation to reduce the loss, and updates the network parameters at the same time.

[0102] The detection module is used to input the image to be detected into the feature extraction module, the layer attention module, and the bi-branch prediction module to obtain the detection result.

[0103] In a specific example, the highway small target detection method based on convolutional neural networks further includes:

[0104] Specific limitations regarding the highway small target detection device based on convolutional neural networks can be found in the limitations of the highway small target detection method based on convolutional neural networks mentioned above, and will not be repeated here. Each module in the aforementioned highway small target detection device based on convolutional neural networks can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0105] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the network topology layout method in this embodiment of the invention by running the computer program stored in the memory.

[0106] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0107] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0108] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting small targets on highways based on convolutional neural networks, comprising: Step 1: Obtain an image training dataset with labeled small target detection boxes for highways. Perform data augmentation on the image training dataset using a local pasting strategy to directly crop the small targets. After rotation, scaling, and affine operations, the small targets are then displayed. center , To define an area using coordinates as its center, when... The first one after random transformation of the master version One copy instance With small target instance set When the areas do not intersect, The defined region in the original image is pasted back using Poisson blending and simultaneously incorporated into the set. middle; Step 2: Pad the image training dataset to a preset size and input it into the ResNet-50-FPN network to output feature maps of four different sizes. , , , ; Step 3: Transfer the feature map , , , Feature maps are obtained by performing feature re-fusion through layer attention. , , , ,include: First, the two layers to be merged and After aligning to a uniform size, they are stitched together. , here The first output from FPN layer, represent The next layer has feature maps that are half the size of the previous layer, so aligning them requires... Upsampling is performed here using bilinear interpolation. Upsampling is achieved by aligning the samples to the same size and then stitching them together. , here and These are the height and width of the feature map, respectively. The number of channels in the feature map; Subsequently Adaptive 2D average pooling, yielding one feature value per channel. There are a total of These feature values ​​are used to perform a fully connected operation on the values ​​to obtain the weights of the two layers. , : To achieve the spliced ​​features Dimensionality reduction and weights Reassign to two feature layers and set up a dimensionality-reduced convolution. weight Multiplying by this dimensionality-reducing convolution, the returned parameters are then concatenated with the features. Perform matrix multiplication and output the final result. and before fusion Same size: Step 4: Process the feature map , , , To perform bounding box regression and class classification prediction operations, a two-branch prediction module was designed. This module consists of two branches: a classification head and a regression head. First, the features output from the previous step are processed... Using two non-parameter-shared convolutions and Extracting classification features separately and regression features ; Then, convolution is used in the regression branch to obtain the target bounding box predictions. and spatial probability graph And simultaneously learn the bounding box offset. Used for subsequent variable convolution of the bounding box; Categorization branches use full connections Bundle The channel is reduced to dimensionality , here Knowing the number of categories, we can obtain the logical score of the target category contained within the features of the corresponding current region. Combine the logical score with the previously obtained spatial probability. After normalization, multiplication yields the final classification result. ; Step 5: Calculate the bounding box loss and classification loss, perform backpropagation to reduce the loss, and update the network parameters at the same time; Step 6: For the unlabeled and unenhanced test image to be detected, perform steps 2 to 4 to obtain the detection results.

2. The method for detecting small targets on highways based on convolutional neural networks as described in claim 1, characterized in that, Data augmentation is performed on the image training dataset using a local pasting strategy. Small targets are directly cropped, and after a series of operations including rotation, scaling, and affine transformation, the pasting region is defined according to the following formula: ； ； ； ； in, , They represent the regional boundaries, , The coordinates of the top-left corner of the rectangle's boundary. , The coordinates of the bottom right corner of the rectangle's boundary. , These are the center coordinates of the small target. , It's about the width and height of small goals. That is the magnification factor of the region.

3. The method for detecting small targets on highways based on convolutional neural networks as described in claim 1, characterized in that, Adjacent feature layers are concatenated, and higher weights are assigned to the feature layer containing the small target.

4. The method for detecting small targets on highways based on convolutional neural networks as described in claim 1, characterized in that, The feature map is used for bounding box regression and class classification prediction, which consists of two branches: a regression head and a classification head. The regression head uses convolution. After obtaining the bounding box prediction, the classification head is used with a fully connected layer to obtain the classification prediction.

5. A highway small target detection device based on convolutional neural networks, characterized in that, The highway small target detection device based on convolutional neural network includes: The data augmentation module employs a local paste strategy, directly cropping small targets and then performing a series of operations such as rotation, scaling, and affine transformation to present the small targets as... center , To define an area using coordinates as its center, when... The first one after random transformation of the master version One copy instance With small target instance set When the areas do not intersect, The defined region in the original image is pasted back using Poisson blending and simultaneously incorporated into the set. middle; The feature extraction module pads the image training dataset to a preset size, inputs it into the ResNet-50-FPN network, and outputs feature maps of four different sizes. , , , ; Layer attention module, which processes feature maps , , , Feature refusion is performed to obtain feature maps. , , , Perform the following operations: Adjacent two layers and After aligning to the same size using bilinear interpolation, they are concatenated. ,right Perform adaptive 2D average pooling to obtain the feature value of the number of channels. Through full connectivity Dimension reduction yields two weights , Finally, these two weights , Assigned to and Two layers, yielding the final output. ; The dual-branch prediction module performs a feature map analysis. , , , To perform bounding box regression and class classification operations, a DoubleHead prediction module was designed. This module consists of two branches: a regression head and a classification head. The regression head uses convolution to obtain the predicted bounding boxes of the target objects. and spatial probability graph Simultaneously learn bounding box offsets Used for subsequent variable convolutions on the bounding boxes; the classification head uses fully connected layers. Bundle The channel is reduced to dimensionality , here Knowing the number of categories, we can obtain the logical score of the target category contained within the features of the corresponding current region. Combine the logical score with the previously obtained spatial probability. After normalization, multiplication yields the final classification result. ; The parameter update module calculates the bounding box loss and classification loss, performs backpropagation to reduce the loss, and updates the network parameters at the same time. The detection module is used to input the image to be detected into the feature extraction module, the layer attention module, and the bi-branch prediction module to obtain the detection result.

6. The highway small target detection device based on convolutional neural network as described in claim 5, characterized in that, Data augmentation is performed on the image training dataset using a local pasting strategy. Small targets are directly cropped, and after a series of operations including rotation, scaling, and affine transformation, the pasting region is defined according to the following formula: ； ； ； ； in, , They represent the regional boundaries, , The coordinates of the top-left corner of the rectangle's boundary. , The coordinates of the bottom right corner of the rectangle's boundary. , These are the center coordinates of the small target. , It's about the width and height of small goals. That is the magnification factor of the region.

7. The highway small target detection device based on convolutional neural network as described in claim 5, characterized in that, Adjacent feature layers are concatenated, and higher weights are assigned to the feature layer containing the small target.

8. The highway small target detection device based on convolutional neural network as described in claim 5, characterized in that, The feature map is used for bounding box regression and class classification prediction, which consists of two branches: a regression head and a classification head. The regression head uses convolution. After obtaining the bounding box prediction, the classification head is used with a fully connected layer to obtain the classification prediction.

9. The highway small target detection device based on convolutional neural network as described in claim 5, characterized in that, The highway small target detection device based on convolutional neural networks also includes: The display module is used to draw small target detection boxes on the image to be detected to indicate the location and category information of the target in the image.

Images