Real-time target detection method based on multi-dimensional attention mechanism

Abstract

The invention discloses a real-time target detection method based on a multi-dimensional attention mechanism. The method comprises the following steps: 1) dividing a data set; the method comprises the steps of (1) designing a real-time target detection model based on a multi-dimensional attention mechanism, (2) obtaining a basic model, (4) obtaining a final model and (5) carrying out target detection.The method achieves efficient real-time target detection on a real-time target detection data set and has good generalization and robustness.

Description

Technical Field

[0001] The present invention relates to target detection technology, and in particular to real-time target detection, a multi-dimensional attention mechanism and a candidate frame region generation network, and specifically to a real-time target detection method based on a multi-dimensional attention mechanism. Background Art

[0002] The development of real-time object detection technology is mainly due to the increasing demand for fast and accurate object recognition in scenarios such as intelligent video surveillance and autonomous driving. Deep learning, especially the introduction of convolutional neural networks, provides a technical basis for achieving this goal. Despite significant progress, existing technologies still face challenges in terms of computing resource requirements, balance between real-time and accuracy, and application on resource-constrained devices.

[0003] At present, the key technologies in the field of real-time object detection include the YOLO series, RT-DETR, SSD and EfficientDet, etc. These technologies combine deep learning with optimized network architecture to improve detection speed while maintaining high detection accuracy. However, these technologies still face some challenges in actual deployment, such as insufficient generalization of the model, dependence on training data, model complexity, adaptability and energy consumption.

[0004] Recently, Transformer-based detection algorithms and lightweight network designs have brought new perspectives to real-time object detection. These emerging methods have shown potential in improving computational efficiency, but they also need further optimization to address the limitations of existing technologies and reduce the high requirements for computing resources. Although these technologies have shown potential in a variety of application scenarios, continuous innovation and technology iteration are still needed to achieve better performance and a wider range of applications. Summary of the invention

[0005] The purpose of the present invention is to provide a real-time target detection method based on a multi-dimensional attention mechanism to address the deficiencies of the prior art. This method achieves efficient real-time target detection on a real-time target detection dataset and has good generalization and robustness.

[0006] The technical solution for achieving the purpose of the present invention is:

[0007] A real-time target detection method based on a multi-dimensional attention mechanism comprises the following steps:

[0008] 1) Divide the dataset: Select the public dataset COCO or PASCAL VOC according to the actual task situation, and divide the dataset COCO or PASCAL VOC into training set and test set to train and test the model;

[0009] 2) Design a real-time object detection model based on a multi-dimensional attention mechanism, including:

[0010] 2-1) Feature extraction: The feature extraction uses the CSPDarknet network model. The input of the feature extraction is a normal image containing the target, and the output is an image feature map. The specific process is as follows:

[0011] Before extracting features from the image, the target is annotated. The annotation method is as follows: in addition to the RGB three channels representing the image, a mask channel is added to form a four-channel channel. The fourth channel uses different codes to annotate the bounding boxes of different types of objects. After the annotation is completed, the image has 4 channels. The CSPDarknet network is modified to a network that can extract not only 3-channel images but also 4-channel images. The features obtained after the input image passes through the 4-channel CSPDarknet network will be input into the channel attention-based feature extraction module CSPAM for channel feature extraction;

[0012] 2-2) Channel feature extraction: The input is the features obtained by the 4-channel CSPDarknet network in step 2-1), and the output is the new image features after the channel attention-based feature extraction module CSPAM. CSPAM extracts channel features from the features obtained after step 2-1) to obtain channel features of the same category of targets. The algorithm process of CSPAM is as follows:

[0013] 2-2-1) Global average pooling process of CSPAM After inputting the features obtained in step 2-1), a global average pooling operation is used to extract high-level global features;

[0014] 2-2-2) After the high-level global features obtained in step 2-2-1) are input into the local convolution process of CSPAM, a convolution kernel of size 1 is used to extract low-level semantic features;

[0015] 2-2-3) The feature fusion process of CSPAM inputs the features obtained by the 4-channel CSPDarknet network in step 2-1) and the features obtained in step 2-2-2), and fuses these two features to obtain the channel features;

[0016] 2-3) Spatial feature extraction: The input is the channel feature extracted in step 2-2), and the output is the new image feature after the feature extraction module SFPM based on spatial attention. SFPM extracts spatial features from the features obtained in step 2-2) to obtain the spatial features of the same category of targets. The algorithm process of SFPM is as follows:

[0017] 2-3-1) After the features obtained in step 2-2) are input into the global pooling process of SFPM, a global pooling operation is used to extract high-level global features;

[0018] 2-3-2) After the local convolution process of SFPM inputs the features obtained in step 2-2), a convolution kernel of size 1 is used to extract low-level semantic features;

[0019] 2-3-3) The feature aggregation process of SFPM inputs the features obtained in step 2-3-1) and aggregates the features to obtain new features;

[0020] 2-3-4) The feature splicing process of SFPM inputs the feature obtained in step 2-3-2) and the feature obtained in step 2-3-3), and splices these two features to obtain a new feature;

[0021] 2-4) Candidate region box generation: The features obtained in step 2-3) are input into the candidate box generation network RPN to obtain the candidate box of the image. The input of the PRN network is the feature map extracted from multi-dimensional features, and the output is multiple candidate boxes. The specific implementation of the RPN network is as follows:

[0022] 2-4-1) After RPN inputs the features obtained in step 2-3), it first uses the candidate box generation algorithm AnchorGenerator to generate multiple candidate boxes. The candidate boxes generated by this algorithm include candidate boxes with 3 different height-to-width ratios;

[0023] 2-4-2) After obtaining the candidate boxes, encode them, encode the coordinates (x1, y1, x2, y2) of the candidate boxes into (x, y, w, h), and normalize these coordinates (x, y, w, h) with a mean of 0 and a variance of 1;

[0024] 2-4-3) Then use L1 loss to regress the candidate box and use cross entropy loss to classify the candidate box to determine whether the candidate box contains an object:

[0025] L1 loss:

[0026]

[0027] in,

[0028] y represents the true label data of the i-th image, and y′ represents the predicted label data of the i-th image; cross entropy loss:

[0029] Loss=-(ylogP+(1-y)log(1-p));

[0030] 2-4-4) Finally, the random sampling algorithm Random Sampler is used to randomly sample 256 candidate boxes from the candidate boxes obtained in step 2-4-3) as the final candidate boxes;

[0031] 2-5) Input the candidate box obtained in step 2-4) and the features obtained in step 2-3) into the detection head network to perform boundary regression and category judgment on the candidate box. The detection head network includes:

[0032] 2-5-1) Shared fully connected layer network: The features obtained in step 2-3) are passed through a shared fully connected network, and then the output of two branches is obtained, one for the bounding box regression task and the other for the classification task;

[0033] 2-5-2) ROI network: The ROI network extracts features from the candidate boxes obtained in step 2-4). For each image, 256 final candidate boxes will be obtained through a random sampling algorithm, and the ROI network will also obtain 256 feature outputs;

[0034] 2-5-3) BBoxHead network: Use the features obtained in step 2-3) and the features obtained in step 2-5-2) ROI network to perform the final bounding box regression and classification tasks, and calculate the final loss. The loss includes three parts, namely:

[0035] L1 loss for bounding boxes:

[0036]

[0037] Cross entropy loss for classification:

[0038] Loss=-(yLogP+(1-y)log(1-p)),

[0039] Cross Entropy Loss:

[0040] Loss=-(ylogP+(1-y)log(1-p)),

[0041] The three losses are added together to get the loss of the final detection head network;

[0042] After steps 2-1) to 2-5), a real-time target detection model is obtained;

[0043] 3) Obtaining a basic model: Use the public dataset COCO or PASCAL VOC to perform basic training on the real-time object detection model in step 2) to obtain a basic model. The specific training process is as follows:

[0044] 3-1) Label each image in the training set divided by the public dataset COCO or PASCAL VOC with one or more category targets, and set different batches according to different GPU devices. The batch size represents the number of training sets. If single GPU training, the batch size is set to 2, then the sample size of the training set in each task is 2. Similarly, if 8 GPU training, each GPU is set to a batch size of 2, then the number of training sets is 8*2=16;

[0045] 3-2) Feature extraction training: Input the batch training set obtained in step 3-1) into the 4-channel CSPDarknet network to obtain batch image features, which contain features of different categories in the entire image. One image contains multiple categories.

[0046] 3-3) Channel feature extraction training: The batch image features obtained in step 3-2) are input into the channel attention-based feature extraction module CSPAM to extract channel features, and the channel features of the same category of targets are obtained. These feature information is used to improve the model's ability to recognize images;

[0047] 3-4) Spatial feature extraction training: The batch image channel features obtained in step 3-3) are input into the spatial attention-based feature extraction module SFPM to extract spatial features, and the spatial features of the same category of targets are obtained. These feature information is used to improve the model's ability to recognize images;

[0048] 3-5) Candidate box generation training: The image features obtained in step 3-4) are input into the RPN network to obtain candidate boxes related to these feature maps;

[0049] 3-6) Detection head network training: Based on the batch image features containing category information obtained in step 3-4), the target box generated in step 3-5) is trained for the final target detection task. The task training includes target box regression and category judgment. After steps 3-1) to 3-6), the basic network model is obtained;

[0050] 4) Obtain the final model: Fine-tune the basic model to obtain the final model, including:

[0051] 4-1) Fine-tuning feature extraction: After dividing the public dataset COCO or PASCAL VOC into a training set and a test set, feature extraction is performed on the images in the training set and the test set in the fine-tuning stage. In the fine-tuning stage, the test set has multiple objects, and the category target is a new category.

[0052] 4-2) Fine-tuning channel feature extraction: Input the features obtained in step 4-1) into the channel attention-based feature extraction module CSPAM to obtain channel features;

[0053] 4-3) Fine-tuning spatial feature extraction: Input the features obtained in step 4-2) into the spatial attention-based feature extraction module SFPM to obtain spatial features;

[0054] 4-4) Fine-tune candidate box generation: Input the multi-dimensional features obtained in step 4-3) into the RPN network to obtain candidate boxes with relevant features;

[0055] 4-5) Fine-tune the detection head network: Based on the multi-dimensional features obtained in step 4-3), the candidate boxes generated in step 4-4) are trained for the final target detection task. The final target detection task training includes the regression of the target box and the judgment of the category. After steps 4-1) to 4-5), the final network model is obtained;

[0056] 5) Object detection: Use the final model trained in step 4) to perform object detection on the public datasets COCO or PASCAL VOC.

[0057] This method achieves efficient real-time object detection on real-time object detection datasets with good generalization and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the process flow of the embodiment method;

[0059] Figure 2 A schematic diagram of generating a candidate frame of an RPN network in an embodiment. DETAILED DESCRIPTION

[0060] The content of the present invention is further described below in conjunction with the drawings and embodiments, but the present invention is not limited thereto.

[0061] Example:

[0062] Reference Figure 1 , a real-time target detection method based on a multi-dimensional attention mechanism, comprising the following steps:

[0063] 1) Divide the dataset: Select the public dataset COCO or PASCAL VOC according to the actual task situation, and divide the dataset COCO or PASCAL VOC into training set and test set to train and test the model;

[0064] 2) Design a real-time object detection model based on a multi-dimensional attention mechanism, including:

[0065] 2-1) Feature extraction: The feature extraction uses the CSPDarknet network model. The input of the feature extraction is a normal image containing the target, and the output is an image feature map. The specific process is as follows:

[0066] Before extracting features from the image, the target is annotated. The annotation method is as follows: in addition to the RGB three channels representing the image, a mask channel is added to form a four-channel channel. The fourth channel uses different codes to annotate the bounding boxes of different types of objects. After the annotation is completed, the image has 4 channels. The CSPDarknet network is modified to a network that can extract not only 3-channel images but also 4-channel images. The features obtained after the input image passes through the 4-channel CSPDarknet network will be input into the channel attention-based feature extraction module CSPAM for channel feature extraction;

[0067] 2-2) Channel feature extraction: The input is the features obtained by the 4-channel CSPDarknet network in step 2-1), and the output is the new image features after the channel attention-based feature extraction module CSPAM. CSPAM extracts channel features from the features obtained after step 2-1) to obtain channel features of the same category of targets. The algorithm process of CSPAM is as follows:

[0068] 2-2-1) Global average pooling process of CSPAM After inputting the features obtained in step 2-1), a global average pooling operation is used to extract high-level global features;

[0069] 2-2-2) After the high-level global features obtained in step 2-2-1) are input into the local convolution process of CSPAM, a convolution kernel of size 1 is used to extract low-level semantic features;

[0070] 2-2-3) The feature fusion process of CSPAM inputs the features obtained by the 4-channel CSPDarknet network in step 2-1) and the features obtained in step 2-2-2), and fuses these two features to obtain the channel features;

[0071] 2-3) Spatial feature extraction: The input is the feature obtained by the channel feature extraction in step 2-2), and the output is the new image feature after the feature extraction module SFPM based on spatial attention. SFPM extracts the spatial feature of the feature obtained after step 2-2) to obtain the spatial features of the same category of targets. The algorithm process of SFPM is as follows:

[0072] 2-3-1) After the features obtained in step 2-2) are input into the global pooling process of SFPM, a global pooling operation is used to extract high-level global features;

[0073] 2-3-2) After the local convolution process of SFPM inputs the features obtained in step 2-2), a convolution kernel of size 1 is used to extract low-level semantic features;

[0074] 2-3-3) The feature aggregation process of SFPM inputs the features obtained in step 2-3-1) and aggregates the features to obtain new features;

[0075] 2-3-4) The feature splicing process of SFPM inputs the feature obtained in step 2-3-2) and the feature obtained in step 2-3-3), and splices these two features to obtain a new feature;

[0076] 2-4) Candidate region box generation: The features obtained in step 2-3) are input into the candidate box generation network RPN to obtain the candidate box of the image. The input of the PRN network is the feature map extracted from multi-dimensional features, and the output is multiple candidate boxes, such as Figure 2 As shown, the specific implementation of the RPN network is as follows:

[0077] 2-4-1) After RPN inputs the features obtained in step 2-3), it first uses the candidate box generation algorithm AnchorGenerator to generate multiple candidate boxes. The candidate boxes generated by this algorithm include candidate boxes with 3 different height-to-width ratios;

[0078] 2-4-2) After obtaining the candidate boxes, encode them, encode the coordinates (x1, y1, x2, y2) of the candidate boxes into (x, y, w, h), and normalize these coordinates (x, y, w, h) with a mean of 0 and a variance of 1;

[0079] 2-4-3) Then use L1 loss to regress the candidate box and use cross entropy loss to classify the candidate box to determine whether the candidate box contains an object:

[0080] L1 loss:

[0081]

[0082] in,

[0083] y represents the true label data of the i-th image, and y′ represents the predicted label data of the i-th image; cross entropy loss:

[0084] Loss=-(ylogP+(1-y)log(1-p));

[0085] 2-4-4) Finally, the random sampling algorithm Random Sampler is used to randomly sample 256 candidate boxes from the candidate boxes obtained in step 2-4-3) as the final candidate boxes;

[0086] 2-5) Input the candidate box obtained in step 2-4) and the features obtained in step 2-3) into the detection head network to perform boundary regression and category judgment on the candidate box. The detection head network includes:

[0087] 2-5-1) Shared fully connected layer network: The features obtained in step 2-3) are passed through a shared fully connected network, and then the output of two branches is obtained, one for the bounding box regression task and the other for the classification task;

[0088] 2-5-2) ROI network: The ROI network extracts features from the candidate boxes obtained in step 2-4). For each image, 256 final candidate boxes will be obtained through a random sampling algorithm, and the ROI network will also obtain 256 feature outputs;

[0089] 2-5-3) BBoxHead network: Use the features obtained in step 2-3) and the features obtained in step 2-5-2) ROI network to perform the final bounding box regression and classification tasks, and calculate the final loss. The loss includes three parts, namely:

[0090] L1 loss for bounding boxes:

[0091]

[0092] Cross entropy loss for classification:

[0093] Loss=-(ylogP+(1-y)log(1-p)),

[0094] Cross Entropy Loss:

[0095] Loss=-(ylogP+(1-y)log(1-p)),

[0096] The three losses are added together to get the loss of the final detection head network;

[0097] After steps 2-1) to 2-5), a real-time target detection model is obtained;

[0098] 3) Obtaining a basic model: Use the public dataset COCO or PASCAL VOC to perform basic training on the real-time object detection model in step 2) to obtain a basic model. The specific training process is as follows:

[0099] 3-1) Label each image in the training set divided by the public dataset COCO or PASCAL VOC with one or more category targets, and set different batches according to different GPU devices. The batch size represents the number of training sets. If single GPU training, the batch size is set to 2, then the sample size of the training set in each task is 2. Similarly, if 8 GPU training, each GPU is set to a batch size of 2, then the number of training sets is 8*2=16;

[0100] 3-2) Feature extraction training: Input the batch training set obtained in step 3-1) into the 4-channel CSPDarknet network to obtain batch image features, which contain features of different categories in the entire image. One image contains multiple categories.

[0101] 3-3) Channel feature extraction training: The batch image features obtained in step 3-2) are input into the channel attention-based feature extraction module CSPAM to extract channel features, and the channel features of the same category of targets are obtained. These feature information is used to improve the model's ability to recognize images;

[0102] 3-4) Spatial feature extraction training: The batch image channel features obtained in step 3-3) are input into the spatial attention-based feature extraction module SFPM to extract spatial features, and the spatial features of the same category of targets are obtained. These feature information is used to improve the model's ability to recognize images;

[0103] 3-5) Candidate box generation training: The image features obtained in step 3-4) are input into the RPN network to obtain candidate boxes related to these feature maps;

[0104] 3-6) Detection head network training: Based on the batch image features containing category information obtained in step 3-4), the target box generated in step 3-5) is trained for the final target detection task. The task training includes target box regression and category judgment. After steps 3-1) to 3-6), the basic network model is obtained;

[0105] 4) Obtain the final model: Fine-tune the basic model to obtain the final model, including:

[0106] 4-1) Fine-tuning feature extraction: After dividing the public dataset COCO or PASCAL VOC into a training set and a test set, feature extraction is performed on the images in the training set and the test set in the fine-tuning stage. In the fine-tuning stage, the test set has multiple objects, and the category target is a new category.

[0107] 4-2) Fine-tuning channel feature extraction: Input the features obtained in step 4-1) into the channel attention-based feature extraction module CSPAM to obtain channel features;

[0108] 4-3) Fine-tuning spatial feature extraction: Input the features obtained in step 4-2) into the spatial attention-based feature extraction module SFPM to obtain spatial features;

[0109] 4-4) Fine-tune candidate box generation: Input the multi-dimensional features obtained in step 4-3) into the RPN network to obtain candidate boxes with relevant features;

[0110] 4-5) Fine-tune the detection head network: Based on the multi-dimensional features obtained in step 4-3), the candidate boxes generated in step 4-4) are trained for the final target detection task. The final target detection task training includes the regression of the target box and the judgment of the category. After steps 4-1) to 4-5), the final network model is obtained;

[0111] 5) Object detection: Use the final model trained in step 4) to perform object detection on the public datasets COCO or PASCAL VOC.

[0112] In this column, the COCO test set is used to test the detection effect of the network model, and the target detection task indicator is AP. 50 and AP 75 indicators for evaluation.

[0113] The results of the experiment are as follows:

[0114] YOLOX and YOLOv7 are representative algorithms for real-time target detection. Ours is used to represent the experimental results of this method, as shown below:

[0115] Method Name Param FLOPs FPS <![CDATA[AP 50 ]]> <![CDATA[AP 75 ]]> YOLOX 99.1 281.9 58 69.6 55.7 YOLOv7 70.4 360.0 84 72.6 60.1 Ours 87.0 296.6 82 73.1 60.6

[0116] In the table, Param represents the model parameter quantity, FLOPs represents the number of floating-point operations per second, and FPS represents the number of frames transmitted per second. From the above experimental results, it can be seen that the AP value of this method in each comparative experiment is better than that of YOLOX.

Claims

1. A real-time target detection method based on a multi-dimensional attention mechanism, characterized in that: The steps include: 1) Divide the dataset: Select the public dataset COCO or PASCAL VOC according to the actual task situation, and divide the dataset COCO or PASCAL VOC into training set and test set to train and test the model; 2) Design a real-time object detection model based on a multi-dimensional attention mechanism, including: 2-1) Feature extraction: The feature extraction uses the CSPDarknet network model. The input of the feature extraction is a normal image containing the target, and the output is an image feature map. The specific process is as follows: Before extracting features from the image, the target is annotated. The annotation method is as follows: in addition to the RGB three channels representing the image, a mask channel is added to form a four-channel channel. The fourth channel uses different codes to annotate the bounding boxes of different types of objects. After the annotation is completed, the image has 4 channels. The CSPDarknet network is modified to a network that can extract not only 3-channel images but also 4-channel images. The input image passes through the 4-channel CSPDarknet network to obtain features, which will be input into the channel attention-based feature extraction module CSPAM for channel feature extraction; 2-2) Channel feature extraction: The input is the features obtained by the 4-channel CSPDarknet network in step 2-1), and the output is the new image features extracted by the channel attention-based feature extraction module CSPAM. CSPAM performs channel feature extraction on the features obtained after step 2-1) to obtain the channel features of the same category of targets. The algorithm process of CSPAM is as follows: 2-2-1) Global average pooling process of CSPAM After inputting the features obtained in step 2-1), a global average pooling operation is used to extract high-level global features; 2-2-2) After the high-level global features obtained in step 2-2-1) are input into the local convolution process of CSPAM, a convolution kernel of size 1 is used to extract low-level semantic features; 2-2-3) The feature fusion process of CSPAM inputs the features obtained by the 4-channel CSPDarknet network in step 2-1) and the features obtained in step 2-2-2), and fuses these two features to obtain the channel features; 2-3) Spatial feature extraction: The input is the channel feature obtained by fusion in step 2-2), and the output is the new image feature after the feature extraction module SFPM based on spatial attention. SFPM extracts spatial features from the features obtained after step 2-2) to obtain the spatial features of the same category of targets. The algorithm process of SFPM is as follows: 2-3-1) After the features obtained in step 2-2) are input into the global pooling process of SFPM, a global pooling operation is used to extract high-level global features; 2-3-2) After the local convolution process of SFPM inputs the features obtained in step 2-2), a convolution kernel of size 1 is used to extract low-level semantic features; 2-3-3) The feature aggregation process of SFPM is to input the features obtained in step 2-3-1) and aggregate the features to obtain new features; 2-3-4) The feature splicing process of SFPM inputs the feature obtained in step 2-3-2) and the feature obtained in step 2-3-3), and splices these two features to obtain a new feature; 2-4) Candidate region box generation: The features obtained in step 2-3) are input into the candidate box generation network RPN to obtain the candidate box of the image. The input of the PRN network is the feature map extracted from multi-dimensional features, and the output is multiple candidate boxes. The specific implementation of the RPN network is as follows: 2-4-1) After RPN inputs the features obtained in step 2-3), it first uses the candidate box generation algorithm AnchorGenerator to generate multiple candidate boxes. The candidate boxes generated by this algorithm include candidate boxes with 3 different height-to-width ratios; 2-4-2) After obtaining the candidate boxes, encode them, encode the coordinates (x1, y1, x2, y2) of the candidate boxes into (x, y, w, h), and normalize these coordinates (x, y, w, h) with a mean of 0 and a variance of 1; 2-4-3) Then use L1 loss to regress the candidate box and use cross entropy loss to classify the candidate box to determine whether the candidate box contains an object: L1 loss: in, y represents the true label data of the i-th image, and y′ represents the predicted label data of the i-th image; Cross Entropy Loss: Loss=-(y log P+(1-y)log(1-p)); 2-4-4) Finally, the random sampling algorithm Random Sampler is used to randomly sample 256 candidate boxes from the candidate boxes obtained in step 2-4-3) as the final candidate boxes; 2-5) Input the candidate box obtained in step 2-4) and the features obtained in step 2-3) into the detection head network to perform boundary regression and category judgment on the candidate box. The detection head network includes: 2-5-1) Shared fully connected layer network: The features obtained in step 2-3) are passed through a shared fully connected network, and then the output of two branches is obtained, one for the bounding box regression task and the other for the classification task; 2-5-2) ROI network: The ROI network extracts features from the candidate boxes obtained in step 2-4). For each image, 256 final candidate boxes will be obtained through a random sampling algorithm, and the ROI network will also obtain 256 feature outputs; 2-5-3) BBoxHead network: Use the features obtained in step 2-3) and the features obtained in step 2-5-2) ROI network to perform the final bounding box regression and classification tasks, and calculate the final loss. The loss includes three parts, namely: L1 loss for bounding boxes: Cross entropy loss for classification: Loss=-(y log P+(1-y)log(1-p)), Cross Entropy Loss: loss=-(y log P+(1-y)log(1-p)), The three losses are added together to get the loss of the final detection head network; After steps 2-1) to 2-5), a real-time target detection model is obtained; 3) Obtaining a basic model: Use the public dataset COCO or PASCAL VOC to perform basic training on the real-time object detection model in step 2) to obtain a basic model. The specific training process is as follows: 3-1) Label each image in the training set divided by the public dataset COCO or PASCAL VOC with one or more category targets, and set different batches according to different GPU devices. The batch size represents the number of training sets. If single GPU training, the batch size is set to 2, then the sample size of the training set in each task is 2. Similarly, if 8 GPU training, each GPU is set to a batch size of 2, then the number of training sets is 8*2=16; 3-2) Feature extraction training: Input the batch training set obtained in step 3-1) into the 4-channel CSPDarknet network to obtain batch image features, which contain features of different categories in the entire image. One image contains multiple categories. 3-3) Channel feature extraction training: The batch image features obtained in step 3-2) are input into the channel attention-based feature extraction module CSPAM to extract channel features, and the channel features of the same category of targets are obtained. These feature information is used to improve the model's ability to recognize images; 3-4) Spatial feature extraction training: The batch image channel features obtained in step 3-3) are input into the spatial attention-based feature extraction module SFPM to extract spatial features, and the spatial features of the same category of targets are obtained. These feature information is used to improve the model's ability to recognize images; 3-5) Candidate box generation training: The image features obtained in step 3-4) are input into the RPN network to obtain candidate boxes related to these feature maps; 3-6) Detection head network training: Based on the batch image features containing category information obtained in step 3-4), the target box generated in step 3-5) is trained for the final target detection task. The task training includes target box regression and category judgment. After steps 3-1) to 3-6), the basic model is obtained; 4) Obtain the final model: Fine-tune the basic model to obtain the final model, including: 4-1) Fine-tune feature extraction: After dividing the public dataset COCO or PASCAL VOC into a training set and a test set, feature extraction is performed on the images in the training set and the test set in the fine-tuning stage, wherein the test set has multiple targets in the fine-tuning stage, and the category target is a new category; 4-2) Fine-tuning channel feature extraction: Input the features obtained in step 4-1) into the channel attention-based feature extraction module CSPAM to obtain channel features; 4-3) Fine-tuning spatial feature extraction: Input the features obtained in step 4-2) into the spatial attention-based feature extraction module SFPM to obtain spatial features; 4-4) Fine-tune candidate box generation: Input the multi-dimensional features obtained in step 4-3) into the RPN network to obtain candidate boxes with relevant features; 4-5) Fine-tune the detection head network: Based on the multi-dimensional features obtained in step 4-3), the candidate boxes generated in step 4-4) are trained for the final target detection task. The final target detection task training includes the regression of the target box and the judgment of the category. After steps 4-1) to 4-5), the final network model is obtained; 5) Object detection: Use the final model trained in step 4) to perform object detection on the public datasets COCO or PASCAL VOC.

Images