Real-time target detection method based on multi-scale feature recursion fusion
The multi-scale feature recursive fusion method addresses resource constraints and improves detection accuracy and robustness in real-time target detection, enhancing performance on resource-constrained devices.
Patent Information
- Application Number
- CN202411068354.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-06
AI Technical Summary
Existing real-time target detection technologies face challenges such as high computing resource requirements, limited generalization ability, data dependence, model complexity, and energy consumption, which hinder their application on resource-constrained devices and compromise detection accuracy.
A real-time target detection method utilizing multi-scale feature recursive fusion, employing a resnet101 network with a BiFPN and RPN for feature extraction and candidate box generation, combined with a detection head network, to achieve efficient and robust detection.
The method achieves efficient real-time target detection with improved generalization and robustness, demonstrated by better AP values compared to YOLOv9.
Smart Images

Figure CN119091114A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to target detection technology, in particular to real-time target detection, a recursive feature pyramid network and a candidate frame region generation network, and specifically to a real-time target detection method based on multi-scale feature recursive fusion. Background Art
[0002] The research background of real-time object detection mainly stems from the growing demand for efficient and accurate detection of targets in fields such as intelligent video surveillance and autonomous driving. With the rise of deep learning technology, especially the application of convolutional neural networks, real-time object detection has become possible. However, despite significant technological advances, existing technologies still have some shortcomings and disadvantages, such as high computing resource requirements, which limits their application on resource-constrained devices. In addition, in order to meet the requirements of real-time and accuracy, researchers are constantly developing new algorithms, but these algorithms may sacrifice detection accuracy while pursuing high real-time performance.
[0003] At present, the technologies for real-time target detection mainly include YOLO series, RT-DETR, SSD, EfficientDet and other algorithms. By combining deep learning and efficient network structures, they have achieved high detection accuracy while significantly improving the real-time performance of target detection. However, these technologies still face challenges in practical applications, such as limited generalization ability, data dependence, model complexity, adaptability issues, and energy consumption issues. Transformer-based real-time detectors and lightweight network structures provide new solutions for real-time target detection, but these new methods also need to be further optimized to overcome the limitations of existing technologies and have high requirements for computing resources. These technologies have broad application prospects in practical applications, but they require continuous innovation and improvement to achieve better performance and wider application. Summary of the invention
[0004] The purpose of the present invention is to provide a real-time target detection method based on multi-scale feature recursive fusion to address the deficiencies of the prior art. This method achieves efficient real-time target detection on a real-time target detection dataset and has good generalization and robustness.
[0005] The technical solution for achieving the purpose of the present invention is:
[0006] A real-time target detection method based on multi-scale feature recursive fusion includes the following steps:
[0007] 1) Divide the dataset: Select the public dataset COCO according to the actual task situation, and divide the dataset COCO into a training set and a test set to train and test the model;
[0008] 2) Design a real-time object detection model based on recursive fusion of multi-scale features, including:
[0009] 2-1) Feature extraction: The feature extraction uses the resnet101 network model. The input of the feature extraction is a normal image containing the target, and the output is an image feature map. The specific process is as follows:
[0010] Before extracting features from an image, the target is labeled. The labeling method is as follows: in addition to the three RGB channels representing the image, a mask channel is added to form a four-channel channel. The fourth channel uses different codes to label the bounding boxes of different types of objects. After labeling, the image has 4 channels. The resent101 network is modified to a network that can extract not only 3-channel images but also 4-channel images. The features of the input image after passing through the 4-channel resent101 network will be input into the feature pyramid network BiFPN for multi-scale feature recursive fusion for multi-scale feature recursive fusion;
[0011] 2-2) Multi-scale feature recursive fusion: The input is the features obtained by the 4-channel resent101 network in step 2-1), and the output is the new image features of the feature pyramid network BiFPN after multi-scale feature recursive fusion. The BiFPN network will perform feature fusion on the features obtained after step 2-1) to obtain the features of the same category of targets at different scales. The algorithm process of BiFPN is as follows:
[0012] 2-2-1) After the top-down process of the BiFPN network inputs the features obtained in step 2-1), a downsampling operation is used to extract high-level semantic features, and the high-level semantic features are divided into three scales according to their different sizes;
[0013] 2-2-2) After the bottom-up process of the BiFPN network inputs the features of the three scales obtained in step 2-2-1), upsampling operations are used to extract the low-level global features of the features of the three scales. First, the size of the feature map of the small scale is enlarged to the size of the feature map of the middle scale, and then the size of the feature map of the middle scale is enlarged to the size of the feature map of the large scale. Then, the size of the feature map of the small scale is enlarged to the size of the feature map of the large scale, and finally, the features of the same two scales are fused in pairs;
[0014] 2-2-3) The lateral connection process of the BiFPN network inputs the three features of different scales obtained in step 2-2-1) and the three fused features obtained in step 2-2-2). If the original input features and the output features are in the same layer, an additional edge will be added between the input features and the output features, so that more features can be fused without increasing too much computational cost;
[0015] 2-2-4) The recursive process of the BiFPN network is to regard step 2-2-1) and step 2-2-2) as a bidirectional feature extraction operation process, and repeat the operation process multiple times to achieve higher-level feature fusion.
[0016] 2-3) Candidate region box generation: The features of different scales obtained in step 2-2) are input into the candidate box generation network RPN to obtain the candidate box of the image. The input of the PRN network is the feature map of multi-scale feature recursive fusion, and the output is multiple candidate boxes. The specific implementation of the RPN network is as follows:
[0017] 2-3-1) After RPN inputs the features obtained in step 2-2), it first uses the candidate box generation algorithm AnchorGenerator to generate multiple candidate boxes. The candidate boxes generated by this algorithm include candidate boxes with 5 different scaling ratios and candidate boxes with 3 different height-to-width ratios;
[0018] 2-3-2) After obtaining the candidate boxes, encode them, encode the coordinates (x1, y1, x2, y2) of the candidate boxes into (x, y, w, h), and normalize these coordinates (x, y, w, h) with a mean of 0 and a variance of 1;
[0019] 2-3-3) Then use L1 loss to regress the candidate box and use cross entropy loss to classify the candidate box to determine whether the candidate box contains an object:
[0020] L1 loss:
[0021]
[0022] in,
[0023] y represents the true label data of the i-th image, and y′ represents the predicted label data of the i-th image; cross entropy loss:
[0024] Loss=-(ylogP+(1-y)log(1-p));
[0025] 2-3-4) Finally, the random sampling algorithm Random Sampler is used to randomly sample 256 candidate boxes from the candidate boxes obtained in step 2-3-3) as the final candidate boxes;
[0026] 2-4) Input the candidate box obtained in step 2-3) and the features obtained in step 2-2) into the detection head network to perform boundary regression and category judgment on the candidate box. The detection head network includes:
[0027] 2-4-1) Shared fully connected layer network: The features obtained by recursive fusion of multi-scale features in step 2-2) pass through a shared fully connected network, and then two branches of output are obtained, one for the bounding box regression task and the other for the classification task;
[0028] 2-4-2) ROI network: The ROI network extracts features from the candidate boxes obtained in step 2-3). For each image, 256 final candidate boxes are obtained through a random sampling algorithm, and the ROI network also obtains 256 feature outputs;
[0029] 2-4-3) BBoxHead network: The features obtained by recursive fusion of multi-scale features in step 2-2) and the features obtained by ROI network in step 2-4-2) are used to perform the final bounding box regression and classification tasks, and the final loss is calculated. The loss includes three parts, namely:
[0030] L1 loss for bounding boxes:
[0031]
[0032] Cross entropy loss for classification:
[0033] Loss=-(ylogP+(1-y)log(1-p)),
[0034] Cross Entropy Loss:
[0035] Loss=-(ylogP+(1-y)log(1-p)),
[0036] The three losses are added together to get the loss of the final detection head network;
[0037] After steps 2-1) to 2-4), a real-time target detection model is obtained;
[0038] 3) Obtaining a basic model: Use the public dataset COCO to perform basic training on the real-time object detection model in step 2) to obtain a basic model. The specific training process is as follows:
[0039] 3-1) Label each image in the training set of the public dataset COCO with one or more category targets, and set different batches according to different GPU devices. The batch size represents the number of training sets. If single GPU training, the batch size is set to 2, then the sample size of the training set in each task is 2. Similarly, if 8 GPU training, each GPU is set to a batch size of 2, then the number of training sets is 8*2=16;
[0040] 3-2) Feature extraction training: Input the batch training set obtained in step 3-1) into the resnet101 network to obtain batch image features, which contain features of different categories in the entire image. One image contains multiple categories.
[0041] 3-3) Multi-scale feature recursive fusion training: The batch image features obtained in step 3-2) are input into BiFPN for multi-scale feature recursive fusion to obtain feature information of different scales of the same category. These feature information are used to improve the model's recognition ability for multi-scale images.
[0042] 3-4) Candidate box generation training: The image features of the fusion of features of different scales obtained in step 3-3) are input into the RPN network to obtain candidate boxes related to these feature maps;
[0043] 3-5) Detection head network training: Based on the batch image features containing category information obtained in step 3-3), the target box generated in step 3-4) is trained for the final target detection task, including target box regression and category judgment, and the basic network model is obtained through steps 3-1) to 3-5);
[0044] 4) Obtain the final model: Fine-tune the basic model to obtain the final model, including:
[0045] 4-1) Fine-tuning feature extraction: After dividing the public COCO dataset into a training set and a test set, feature extraction is performed on the images in the training set and the test set in the fine-tuning stage. In the fine-tuning stage, the test set has multiple objects, and the category objects are new categories.
[0046] 4-2) Fine-tune the recursive fusion of multi-scale features: Input the features obtained in step 4-1) into the BiFPN network to obtain multi-scale features and perform recursive fusion;
[0047] 4-3) Fine-tune candidate box generation: Input the features obtained by recursive fusion of multi-scale features in step 4-2) into the RPN network to obtain candidate boxes of relevant features;
[0048] 4-4) Fine-tune the detection head network: Based on the fused features obtained in step 4-2), the candidate boxes generated in step 4-3) are trained for the final target detection task. The final target detection task training includes target box regression and category judgment. After steps 4-1) to 4-4), the final network model is obtained;
[0049] 5) Object detection: Use the final model trained in step 4) to perform object detection on the public dataset COCO.
[0050] This method achieves efficient real-time object detection on real-time object detection datasets with good generalization and robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 Schematic diagram of the method flow of the embodiment;
[0052] Figure 2 Schematic diagram of BiFPN network feature fusion in the embodiment;
[0053] Figure 3 A schematic diagram of generating a candidate frame of an RPN network in an embodiment. DETAILED DESCRIPTION
[0054] The content of the present invention is further described below in conjunction with the drawings and embodiments, but the present invention is not limited thereto.
[0055] Example:
[0056] Reference Figure 1 , a real-time target detection method based on multi-scale feature recursive fusion, comprising the following steps:
[0057] 1) Divide the dataset: Select the public dataset COCO according to the actual task situation. The COCO dataset is a large-scale dataset with an important position in the field of computer vision. It contains more than 330,000 images, with more than 2.5 million target instances annotated, covering 80 object categories. Each image contains one or more object categories. Divide the COCO dataset into a training set and a test set with a ratio of 8:2 to train and test the model.
[0058] 2) Design a real-time object detection model based on recursive fusion of multi-scale features, including:
[0059] 2-1) Feature extraction: The feature extraction uses the resnet101 network model. The input of the feature extraction is a normal image containing the target, and the output is an image feature map. The specific process is as follows:
[0060] Before extracting features from an image, the target is labeled. The labeling method is as follows: in addition to the three RGB channels representing the image, a mask channel is added to form a four-channel channel. The fourth channel uses different codes to label the bounding boxes of different types of objects. After labeling, the image has 4 channels. The resent101 network is modified to a network that can extract not only 3-channel images but also 4-channel images. The features of the input image after passing through the 4-channel resent101 network will be input into the feature pyramid network BiFPN for multi-scale feature recursive fusion for multi-scale feature recursive fusion;
[0061] like Figure 2 As shown, 2-2) Multi-scale feature recursive fusion: The input is the feature obtained by the 4-channel resent101 network in step 2-1), and the output is the new image feature of the feature pyramid network BiFPN after multi-scale feature recursive fusion. The BiFPN network will perform feature fusion on the features obtained after step 2-1) to obtain the features of the same category of targets at different scales. The algorithm process of BiFPN is as follows:
[0062] 2-2-1) After the top-down process of the BiFPN network inputs the features obtained in step 2-1), a downsampling operation is used to extract high-level semantic features, and the high-level semantic features are divided into three scales according to their different sizes;
[0063] 2-2-2) After the bottom-up process of the BiFPN network inputs the features of the three scales obtained in step 2-2-1), upsampling operations are used to extract the low-level global features of the features of the three scales. First, the feature map of the small scale is enlarged to the size of the feature map of the middle scale, and then the size of the feature map of the middle scale is enlarged to the size of the feature map of the large scale. Then, the size of the feature map of the small scale is enlarged to the size of the feature map of the large scale, and finally, the features of the same two scales are fused in pairs;
[0064] 2-2-3) The lateral connection process of the BiFPN network inputs the three features of different scales obtained in step 2-2-1) and the three fused features obtained in step 2-2-2). If the original input features and the output features are in the same layer, an additional edge will be added between the input features and the output features to fuse more features without increasing too much computational cost;
[0065] 2-2-4) The recursive process of the BiFPN network regards step 2-2-1) and step 2-2-2) as a bidirectional feature extraction operation process, and repeats the operation process multiple times to achieve higher-level feature fusion.
[0066] like Figure 3 As shown, 2-3) Candidate region box generation: The features of different scales obtained in step 2-2) are input into the candidate box generation network RPN to obtain the candidate box of the image. The input of the PRN network is the feature map of multi-scale feature recursive fusion, and the output is multiple candidate boxes. The specific implementation of the RPN network is as follows:
[0067] 2-3-1) After RPN inputs the features obtained in step 2-2), it first uses the candidate box generation algorithm AnchorGenerator to generate multiple candidate boxes. The candidate boxes generated by this algorithm include candidate boxes with 5 different scaling ratios and candidate boxes with 3 different height-to-width ratios;
[0068] 2-3-2) After obtaining the candidate boxes, encode them, encode the coordinates (x1, y1, x2, y2) of the candidate boxes into (x, y, w, h), and normalize these coordinates (x, y, w, h) with a mean of 0 and a variance of 1;
[0069] 2-3-3) Then use L1 loss to regress the candidate box and use cross entropy loss to classify the candidate box to determine whether the candidate box contains an object:
[0070] L1 loss:
[0071]
[0072] in,
[0073] y represents the true label data of the i-th image, and y′ represents the predicted label data of the i-th image; cross entropy loss:
[0074] Loss=-(ylogP+(1-y)log(1-p));
[0075] 2-3-4) Finally, the random sampling algorithm Random Sampler is used to randomly sample 256 candidate boxes from the candidate boxes obtained in step 2-3-3) as the final candidate boxes;
[0076] 2-4) Input the candidate box obtained in step 2-3) and the features obtained in step 2-2) into the detection head network to perform boundary regression and category judgment on the candidate box. The detection head network includes:
[0077] 2-4-1) Shared fully connected layer network: The features obtained by recursive fusion of multi-scale features in step 2-2) pass through a shared fully connected network, and then two branches of output are obtained, one for the bounding box regression task and the other for the classification task;
[0078] 2-4-2) ROI network: The function of the ROI network is to extract features from the candidate boxes obtained in step 2-3). For each image, 256 final candidate boxes will be obtained through a random sampling algorithm, and the ROI network will also obtain 256 feature outputs;
[0079] 2-4-3) BBoxHead network: The features obtained by recursive fusion of multi-scale features in step 2-2) and the features obtained by ROI network in step 2-4-2) are used to perform the final bounding box regression and classification tasks, and the final loss is calculated. The loss includes three parts, namely:
[0080] L1 loss for bounding boxes:
[0081]
[0082] Cross entropy loss for classification:
[0083] Loss=-(ylogP+(1-y)log(1-p)),
[0084] Cross Entropy Loss:
[0085] Loss=-(ylogP+(1-y)log(1-p)),
[0086] The three losses are added together to get the loss of the final detection head network;
[0087] After steps 2-1) to 2-4), a real-time target detection model is obtained;
[0088] 3) Obtaining a basic model: Use the public dataset COCO to perform basic training on the real-time object detection model in step 2) to obtain a basic model. The specific training process is as follows:
[0089] 3-1) Label each image of the training set of the public dataset COCO with one or more category targets, and set different batches according to different GPU devices. The batch size represents the number of training sets. If single GPU training, the batch size is set to 2, then the sample size of the training set in each task is 2. Similarly, if 8 GPU training, each GPU is set to a batch size of 2, then the number of training sets is 8*2=16;
[0090] 3-2) Feature extraction training: Input the batch data set obtained in step 3-1) into the 4-channel resnet101 network to obtain batch image features, which contain features of different categories in the entire image. One image contains multiple categories.
[0091] 3-3) Multi-scale feature recursive fusion training: The batch image features obtained in step 3-2) are input into BiFPN for multi-scale feature recursive fusion to obtain feature information of different scales of the same category. These feature information are used to improve the model's recognition ability for multi-scale images.
[0092] 3-4) Candidate box generation training: The image features of the fusion of features of different scales obtained in step 3-3) are input into the RPN network to obtain candidate boxes related to these feature maps;
[0093] 3-5) Detection head network training: Based on the batch image features containing category information obtained in step 3-3), the target box generated in step 3-4) is trained for the final target detection task. The task training includes target box regression and category judgment. After steps 3-1) to 3-5), the basic network model is obtained;
[0094] 4) Obtain the final model: Fine-tune the basic model to obtain the final model, including:
[0095] 4-1) Fine-tuning feature extraction: After dividing the public COCO dataset into training and test sets, feature extraction is performed on the images in the training and test sets in the fine-tuning stage. In the fine-tuning stage, the test set has multiple targets, and the category targets are new categories;
[0096] 4-2) Fine-tune the recursive fusion of multi-scale features: Input the features obtained in step 4-1) into the BiFPN network to obtain multi-scale features and perform recursive fusion;
[0097] 4-3) Fine-tune candidate box generation: Input the features obtained by recursive fusion of multi-scale features in step 4-2) into the RPN network to obtain candidate boxes of relevant features;
[0098] 4-4) Fine-tune the detection head network: Based on the fused features obtained in step 4-2), the candidate boxes generated in step 4-3) are trained for the final target detection task. The final target detection task training includes target box regression and category judgment. After steps 4-1) to 4-4), the final network model is obtained;
[0099] 5) Object detection: Use the final model trained in step 4) to perform object detection on the public dataset COCO.
[0100] In this column, the COCO test set is used to test the detection effect of the network model, and the target detection task indicator is AP. 50:95 , A.P. 50 and AP 75 indicators for evaluation.
[0101] The results of the experiment are as follows:
[0102] YOLOv9 is a representative algorithm for real-time target detection. Ours is used to represent the experimental results of this method, as shown below:
[0103] Method Name Param FLOPs <![CDATA[AP 50:95 ]]> <![CDATA[AP 50 ]]> <![CDATA[AP 75 ]]> YOLOv9 57.3 189.0 55.6 72.8 60.6 Ours 57.3 189.1 56.0 73.2 61.0
[0104] In the table, Param represents the model parameter quantity, and FLOPs represents the number of floating-point operations per second. From the above experimental results, it can be seen that the AP value of this method in each comparative experiment is better than that of YOLOv9.
Claims
1. A real-time target detection method based on recursive fusion of multi-scale features, characterized in that: The steps include: 1) Dataset division: Select the public COCO dataset according to the actual task situation and divide the COCO dataset into training and test sets to train and test the model; 2) Design a real-time object detection model based on recursive fusion of multi-scale features, including: 2-1) Feature extraction: The feature extraction uses the resnet101 network model. The input of the feature extraction is a normal image containing the target, and the output is an image feature map. The specific process is as follows: Before feature extraction from the image, the target is annotated. The annotation method is as follows: in addition to the RGB three channels representing the image, a mask channel is added to form a four-channel channel. The fourth channel uses different encodings to mark the bounding boxes of different types of objects. After the annotation is completed, the image has 4 channels. The resent101 network is modified to be able to extract not only 3-channel images but also 4-channel images. The features obtained after the input image passes through the 4-channel resent101 network will be input into the feature pyramid network BiFPN for multi-scale feature recursive fusion; 2-2) Multi-scale feature recursive fusion: The input is the features obtained by the 4-channel resent101 network in step 2-1), and the output is the new image features obtained by the feature pyramid network BiFPN after multi-scale feature recursive fusion. The BiFPN network fuses the features obtained in step 2-1) to obtain features of the same category of objects at different scales. The BiFPN algorithm process is as follows: 2-2-1) After the top-down process of the BiFPN network inputs the features obtained in step 2-1), a downsampling operation is used to extract high-level semantic features, and the high-level semantic features are divided into three scales according to their different sizes; 2-2-2) After the bottom-up process of the BiFPN network inputs the features of the three scales obtained in step 2-2-1), upsampling operations are used to extract low-level global features of the features of the three scales. The size of the feature map of the small scale is first enlarged to the size of the feature map of the medium scale, the size of the feature map of the medium scale is then enlarged to the size of the feature map of the large scale, and the size of the feature map of the small scale is then enlarged to the size of the feature map of the large scale. Finally, the features of the same two scales are fused pairwise. 2-2-3) The lateral connection process of the BiFPN network inputs the three features of different scales obtained in step 2-2-1) and the three fused features obtained in step 2-2-2). If the input features and output features are in the same layer, an additional edge will be added between the input features and the output features to fuse more features; 2-2-4) The recursive process of the BiFPN network is to regard steps 2-2-1) and 2-2-2) as a bidirectional feature extraction operation process, and repeat the operation process multiple times to achieve higher-level feature fusion; 2-3) Candidate Region Box Generation: The features of different scales obtained in step 2-2) are input into the candidate box generation network RPN to obtain candidate boxes of the image. The input of the PRN network is the feature map of recursively fused multi-scale features, and the output is multiple candidate boxes. The specific implementation of the RPN network is as follows: 2-3-1) After the RPN inputs the features obtained in step 2-2), it first uses the candidate box generation algorithm AnchorGenerator to generate multiple candidate boxes. The candidate boxes generated by this algorithm include candidate boxes with 5 different scaling ratios and candidate boxes with 3 different aspect ratios; 2-3-2) After obtaining the candidate boxes, encode them and encode the coordinates (x1, y1, x2, y2) of the candidate boxes into (x, y, w, h). At the same time, these coordinates (x, y, w, h) are normalized with a mean of 0 and a variance of 1. 2-3-3) Then use L1 loss to regress the candidate box and use cross entropy loss to classify the candidate box to determine whether the candidate box contains an object: L1 loss: in, y represents the true label data of the i-th image, and y′ represents the predicted label data of the i-th image; Cross Entropy Loss: Loss=-(ylogP+(1-y)log(1-p)); 2-3-4) Finally, the random sampling algorithm Random Sampler is used to randomly sample 256 candidate boxes from the candidate boxes obtained in step 2-3-3) as the final candidate boxes; 2-4) Input the candidate box obtained in step 2-3) and the features obtained in step 2-2) into the detection head network to perform boundary regression and category judgment on the candidate box. The detection head network includes: 2-4-1) Shared fully connected layer network: The features obtained by recursive fusion of multi-scale features in step 2-2) are passed through a shared fully connected network, and then two branches of output are obtained, one for the bounding box regression task and the other for the classification task; 2-4-2) ROI network: The ROI network extracts features from the candidate boxes obtained in step 2-3). For each image, a random sampling algorithm is used to obtain 256 final candidate boxes, and the ROI network also outputs 256 features. 2-4-3) BBoxHead Network: Use the features obtained by the recursive fusion of multi-scale features in step 2-2) and the features obtained by the ROI network in step 2-4-2) to perform the final bounding box regression and classification tasks, and calculate the final loss. The loss consists of three parts, namely: L1 loss for bounding boxes: Cross entropy loss for classification: Loss=-(ylogP+(1-y)log(1-p)), Cross Entropy Loss: Loss=-(ylogP+(1-y)log(1-p)), The three losses are added together to obtain the loss of the final detection head network; After steps 2-1) to 2-4), a real-time target detection model is obtained; 3) Obtaining a basic model: Use the public COCO dataset to perform basic training on the real-time object detection model in step 2) to obtain a basic model. The specific training process is as follows: 3-1) Label each image in the training set of the public COCO dataset with one or more target categories. Different batch sizes are set based on the GPU device. The batch size represents the number of training sets. If training on a single GPU, the batch size is set to 2, so the number of training set samples in each task is 2. Similarly, if training on 8 GPUs, the batch size is set to 2 for each GPU, so the number of training sets is 8*2=16. 3-2) Feature extraction training: The batch training set obtained in step 3-1) is input into the 4-channel ResNet101 network to obtain batch image features. The image features contain features of different categories in the entire image. One image contains multiple categories. 3-3) Multi-scale feature recursive fusion training: The batch image features obtained in step 3-2) are input into BiFPN for multi-scale feature recursive fusion to obtain feature information of different scales containing the same category. This feature information is used to improve the model's recognition ability for multi-scale images; 3-4) Candidate box generation training: The image features obtained by fusion of different scale features in step 3-3) are input into the RPN network to obtain candidate boxes related to these feature maps; 3-5) Detection head network training: Based on the batch image features containing category information obtained in step 3-3), the target bounding box generated in step 3-4) is trained for the final target detection task. The task training includes target bounding box regression and category judgment. After steps 3-1) to 3-5), the basic network model is obtained; 4) Obtaining the final model: Fine-tune the base model to obtain the final model, including: 4-1) Fine-tuning feature extraction: After dividing the public COCO dataset into training and test sets, feature extraction is performed on the images in the training and test sets during the fine-tuning phase. In the fine-tuning phase, the test set has multiple objects, and the category targets are new categories; 4-2) Fine-tuning multi-scale feature recursive fusion: Input the features obtained in step 4-1) into the BiFPN network to obtain multi-scale features and perform recursive fusion; 4-3) Fine-tuning candidate box generation: The features obtained by recursive fusion of multi-scale features in step 4-2) are input into the RPN network to obtain candidate boxes with relevant features; 4-4) Fine-tune the detection head network: Based on the fused features obtained in step 4-2), the candidate boxes generated in step 4-3) are trained for the final object detection task. The final object detection task training includes target box regression and category judgment. After steps 4-1) to 4-4), the final network model is obtained; 5) Object Detection: Use the final model trained in step 4) to perform object detection on the public dataset COCO.
Citation Information
Patent Citations
Metalearning small sample image target detection method based on multi-scale feature fusion
CN116704260A
Aerial image small target detection method based on improved YOLOX-S
CN117079163A
Multi-scale target real-time detection method based on hierarchical semantics
CN118429720A
Transformer-based multi-scale pedestrian re-identification method
US20240161531A1
