A FCOS network optimization method and system based on feature fusion and attention mechanism
By introducing feature fusion and attention mechanisms into the FCOS network, the problems of low detection accuracy and high computing resource consumption in multi-scale object detection are solved, and more efficient multi-scale object detection is achieved, which improves detection accuracy and computing efficiency.
Patent Information
- Application Number
- CN202411195885.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-29
AI Technical Summary
The existing multi-scale object detection technology has low accuracy and high computing resources when detecting small targets, especially on resource-constrained devices, and too many background areas lead to serious category imbalance problems.
Feature fusion and attention mechanism are introduced in the FCOS network, and by adding spatial attention and channel attention mechanisms, the information fusion capability of feature pyramids is enhanced and the feature expression and detection capability of multi-scale targets is improved. The specific method includes adding an attention mechanism between the feature pyramid and the detection head, determining the attention weight through maximum and mean pooling, softmax operations, and multiplying it with the original feature layer to obtain the final output feature map.
The accuracy and efficiency of multi-scale image object detection have been improved, the information fusion capability of the network has been enhanced, the accuracy and computing efficiency of small object detection have been improved, and the detection accuracy has been increased by 3.2% on the MS COCO dataset.
Smart Images

Figure CN118711040B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of target detection network, and in particular to an FCOS network optimization method and system based on feature fusion and attention mechanism. Background Art
[0002] Multi-scale object detection is a key problem in computer vision, which requires the algorithm to be able to simultaneously detect objects of different sizes in an image. With the development of deep learning technology, multi-scale object detection technology has made significant progress, but it still faces some challenges and difficulties. At present, deep learning-based object detection methods can be mainly divided into anchor-based methods and anchor-free methods. Typical anchor-based methods include Faster R-CNN, SSD, etc., while anchor-free methods include YOLO, FCOS, etc. In order to detect objects of different sizes, many methods construct feature pyramids to improve the detection ability of small objects by detecting on feature maps of different scales. The attention mechanism is introduced into object detection to enhance the model's attention to different parts of the object, thereby improving the detection performance. Small objects in multi-scale objects are the difficulty in detection, which is reflected in the fact that small objects occupy fewer pixels in the image, making it difficult to extract enough features and the detection accuracy is low; in practical applications, the background area (excluding the object) is far more than the foreground object, causing serious class imbalance. Multi-scale object detection usually requires high computational resources, which limits its application on resource-constrained devices.
[0003] The FCOS network has the following advantages in detecting multi-scale targets:
[0004] 1. Anchor-Free: FCOS is an anchor-free detection method that does not require pre-defined anchor boxes, thereby reducing the adjustment of hyperparameters and simplifying the training process.
[0005] 2. Feature Pyramid: FCOS uses feature pyramid to capture features of different scales, which enables it to effectively detect objects of different sizes.
[0006] 3. Pixel-by-pixel prediction: FCOS directly predicts the target category and bounding box for each pixel in the feature map. This method can provide more refined detection results.
[0007] 4. Center-ness: FCOS introduces a Center-ness branch to evaluate the distance between the predicted box and the target center, which helps to improve the detection accuracy of small targets.
[0008] End-to-end detection: FCOS is an end-to-end object detection framework that does not require an additional candidate region extraction network, which further improves the detection speed.
[0009] In summary, the FCOS network has shown its unique advantages in multi-scale target detection, especially in processing small target detection and computational efficiency, but further research and improvement are still needed to address the difficulties of small target detection. Summary of the invention
[0010] The purpose of the present invention is to provide a FCOS network optimization method and system based on feature fusion and attention mechanism to solve the deficiencies in the prior art, enhance the information fusion capability of the network and the feature expression of multi-scale targets, and improve the detection capability of multi-scale image targets.
[0011] An embodiment of the present application provides a FCOS network optimization method based on feature fusion and attention mechanism, the method comprising:
[0012] The feature fusion method of the original fully convolutional single-stage object detection FCOS network is optimized;
[0013] The spatial attention mechanism and the channel attention mechanism are added between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight.
[0014] Optionally, the optimization process of the feature fusion method includes:
[0015] The first preset feature layer is added to the feature pyramid of the FCOS network, the second preset feature layer is deleted, and a bottom-up information fusion path is added.
[0016] Optionally, the feature pyramid includes: P3, P4, P5, P6 and P7 feature layers, the first preset feature layer is the P2 feature layer, and the second preset feature layer is the P7 feature layer;
[0017] Among them, the P2 feature layer is downsampled and combined with the original P3 feature layer to obtain the feature layer P3 that integrates the bottom feature information;
[0018] Downsample the feature layer P3 and combine it with the original P4 feature layer to obtain the feature layer P4 that integrates the bottom feature information;
[0019] Downsample the feature layer P4 and combine it with the original P5 feature layer to obtain the feature layer P5 that integrates the bottom feature information;
[0020] The feature layer P5 is downsampled to obtain the feature layer P6.
[0021] Optionally, determining the spatial attention weight and the channel attention weight according to the original feature layer output by the feature pyramid, and obtaining the final output of the feature pyramid according to the original feature layer, the spatial attention weight and the channel attention weight, includes:
[0022] The original feature layer output by the feature pyramid is processed by maximum pooling and mean pooling respectively;
[0023] Perform softmax operation on the feature vectors obtained after pooling to obtain spatial attention weights and channel attention weights;
[0024] Multiply the original feature layer by the spatial attention weight and the channel attention weight respectively to obtain the first feature map with added spatial attention and the second feature map with added channel attention;
[0025] The first feature map and the second feature map are fused to obtain a third feature map that integrates the spatial attention and channel attention mechanisms;
[0026] The original feature map is connected with the third feature map to obtain the final output of the feature pyramid, which is used as the final feature layer sent to the detection head.
[0027] Optionally, the calculation formula of the first feature map is:
[0028]
[0029] Among them, the Represents the first feature map after adding the spatial attention mechanism, Represents the original feature layer output by the feature pyramid. represents the maximum pooling, express operate.
[0030] Optionally, the calculation formula of the second feature map is:
[0031]
[0032] Among them, the Represents the feature layer after adding the channel attention mechanism. Represents mean pooling.
[0033] Optionally, the calculation formula of the final feature layer sent to the detection head is:
[0034]
[0035] Among them, the is the final feature layer.
[0036] Another embodiment of the present application provides a FCOS network optimization system based on feature fusion and attention mechanism, the system comprising:
[0037] The optimization module is used to optimize the feature fusion method of the original fully convolutional single-stage object detection FCOS network;
[0038] A module is added to add a spatial attention mechanism and a channel attention mechanism between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight.
[0039] Yet another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when running.
[0040] Yet another embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the methods described above.
[0041] Compared with the prior art, the present invention provides an FCOS network optimization method based on feature fusion and attention mechanism, which optimizes the feature fusion method of the original full convolution single-stage target detection FCOS network; between the feature pyramid and the detection head of the optimized FCOS network structure, a spatial attention mechanism and a channel attention mechanism are added, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight, thereby enhancing the information fusion capability of the network and the feature expression of multi-scale targets, and improving the detection capability of multi-scale image targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A hardware structure block diagram of a computer terminal for an FCOS network optimization method based on feature fusion and attention mechanism provided in an embodiment of the present invention;
[0043] Figure 2 A flowchart of a FCOS network optimization method based on feature fusion and attention mechanism provided in an embodiment of the present invention;
[0044] Figure 3A schematic diagram of the overall structure of an optimized FCOS network provided by an embodiment of the present invention;
[0045] Figure 4 A network structure diagram of a feature pyramid in an optimized FCOS network provided by an embodiment of the present invention;
[0046] Figure 5 A network structure diagram of an attention mechanism added to an optimized FCOS network provided by an embodiment of the present invention;
[0047] Figure 6 A schematic diagram of the detection effect of an optimized FCOS network in practical application provided by an embodiment of the present invention;
[0048] Figure 7 A structural diagram of an FCOS network optimization system based on feature fusion and attention mechanism provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0049] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, but should not be construed as limiting the present invention.
[0050] The embodiment of the present invention first provides a FCOS network optimization method based on feature fusion and attention mechanism, which can be applied to electronic devices such as computer terminals, specifically ordinary computers, etc.
[0051] The following describes it in detail by taking running on a computer terminal as an example. Figure 1 The hardware structure block diagram of a computer terminal of a FCOS network optimization method based on feature fusion and attention mechanism provided by an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0052] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / modules corresponding to the FCOS network optimization method based on feature fusion and attention mechanism in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
[0053] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.
[0054] See also Figure 2-Figure 6 , an embodiment of the present invention provides a FCOS network optimization method based on feature fusion and attention mechanism, which may include the following steps:
[0055] S201, optimizing the feature fusion method of the original full convolution single-stage target detection FCOS network;
[0056] Specifically, the optimization process of the feature fusion method may include:
[0057] The first preset feature layer is added to the feature pyramid of the FCOS network, the second preset feature layer is deleted, and a bottom-up information fusion path is added.
[0058] Specifically, the feature pyramid may include: P3, P4, P5, P6 and P7 feature layers, the first preset feature layer is the P2 feature layer, the P2 feature layer has richer position information than the P3 feature layer, and can improve the positioning accuracy of small targets, and the second preset feature layer is the P7 feature layer;
[0059] Among them, the P2 feature layer is downsampled and combined with the original P3 feature layer to obtain the feature layer P3 that integrates the bottom feature information; the feature layer P3 is downsampled and combined with the original P4 feature layer to obtain the feature layer P4 that integrates the bottom feature information; the feature layer P4 is downsampled and combined with the original P5 feature layer to obtain the feature layer P5 that integrates the bottom feature information; the feature layer P5 is downsampled to obtain the feature layer P6.
[0060] The Feature Pyramid of the original FCOS network uses P3, P4, P5, P6 and P7 feature layers. The present invention deletes the P7 feature layer and adds the P2 feature layer based on the Feature Pyramid structure of the original network; adds a bottom-up information fusion path, downsamples the P2 feature layer and combines it with the original P3 feature layer to obtain the feature layer P3 that fuses the bottom feature information, performs the same operation on P3 to obtain the P4 and P5 feature layers respectively, and finally downsamples the P5 feature layer to obtain the P6 feature layer.
[0061] S202, adding a spatial attention mechanism and a channel attention mechanism between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight.
[0062] Specifically, according to the original feature layer output by the feature pyramid, the spatial attention weight and the channel attention weight are determined, and according to the original feature layer, the spatial attention weight and the channel attention weight, the final output of the feature pyramid is obtained, which may include the following steps:
[0063] The original feature layer output by the feature pyramid is processed by maximum pooling and mean pooling respectively;
[0064] Perform softmax operation on the feature vectors obtained after pooling to obtain spatial attention weights and channel attention weights;
[0065] Multiply the original feature layer by the spatial attention weight and the channel attention weight respectively to obtain the first feature map with added spatial attention and the second feature map with added channel attention;
[0066] The first feature map and the second feature map are fused to obtain a third feature map that integrates the spatial attention and channel attention mechanisms;
[0067] The original feature map is connected with the third feature map to obtain the final output of the feature pyramid, which is used as the final feature layer sent to the detection head.
[0068] Specifically, the calculation formula of the first characteristic map may be:
[0069]
[0070] Among them, the Represents the first feature map after adding the spatial attention mechanism, Represents the original feature layer output by the feature pyramid. represents the maximum pooling, express operate.
[0071] Specifically, the calculation formula of the second characteristic map may be:
[0072]
[0073] Among them, the Represents the feature layer after adding the channel attention mechanism. Represents mean pooling.
[0074] Specifically, the calculation formula of the final feature layer sent to the detection head is:
[0075]
[0076] Among them, the is the final feature layer.
[0077] In practical applications, the spatial attention mechanism and channel attention mechanism include: performing maximum pooling and mean pooling on the feature layers output by the improved FeaturePyramid, performing softmax operations on the feature vectors obtained after pooling, obtaining spatial attention weights and channel attention weights, weighting the weights with the output of FeaturePyramid, taking the maximum value at each position to obtain the final feature map and connecting it with the original feature layer through a residual structure to obtain the feature map after feature enhancement.
[0078] Among them, the spatial attention mechanism includes: performing maximum pooling on the feature map output by the improved Feature Pyramid, taking the maximum value of all channels at each spatial position in the feature layer, obtaining a maximum value feature map with a channel of 1 and the same length and width as the original feature layer, performing a softmax operation on the maximum value feature map, and obtaining a spatial attention weight map (spatial attention weight) whose values are normalized between 0 and 1, and multiplying the original feature layer by the spatial attention weight map to obtain a feature map with the spatial attention mechanism added (the first feature map).
[0079] The channel attention mechanism includes: first, average pooling the feature map output by the feature pyramid, taking the average value of all spatial positions of each channel in the feature layer to obtain a one-dimensional vector, and then fully connecting and softmaxing the one-dimensional vector to obtain a channel weight vector. According to the channel weight vector, the original feature map is weighted to obtain a channel attention weight map (channel attention weight) whose values are normalized between 0 and 1. The original feature layer is multiplied by the channel attention weight map to obtain a feature map (second feature map) with the channel attention mechanism added.
[0080] For the feature layer after adding spatial attention and channel attention, take the larger value of the same position in the two layers to obtain the feature map (third feature map) that integrates the two attention mechanisms, and connect the original feature map output by the improved feature pyramid with the above fused feature map through the residual connection channel to obtain the feature layer that is finally sent to the detection head. Figure 3 As shown, Represents the size of the current convolutional layer, which can be visualized as , , the rest are the same; Backbone represents the backbone network for feature extraction, FEN represents the feature enhancement network, Head represents the detection head, Represents the number of channels, Regression represents regression, Center-ness represents central entropy, and Classification represents classification. Indicates the size and number of channels of the convolutional layer.
[0081] Exemplarily, an application scenario includes:
[0082] Step 1: Get the MS COCO dataset, which contains images and corresponding annotation information, covering a variety of object categories. Preprocess the image dataset, including scaling, cropping, normalization, and other operations. At the same time, convert the annotation information into the format required for model training, including Bounding Box coordinates and object categories.
[0083] Step 2: Select the FCOS network model pre-trained on ImageNet as the base model and modify the network structure, such as Figure 4As shown in the figure, Conv1(BN+ReLu) means that after the first convolution layer (Conv1), batch normalization (BN) is first applied, and then the ReLU activation function is applied. Conv2_x means convolution layer 2, Conv2d means two-dimensional convolution, and the rest are similar; Conv2d means two-dimensional convolution, k1, s1 means the convolution kernel size is 1×1, the step size of the convolution operation is 1, Up-sample means upsampling, k3, s1 means the convolution kernel size is 3×3, the step size of the convolution operation is 1, and down-sample means downsampling. Delete the P7 feature layer in the original network and add the P2 feature layer. Taking the input image scale of 640×640×3 as an example, the scale of the feature extraction network C2 in the backbone is 320×320×64. The length and width from C3 to C5 are reduced by 1 times, and the number of channels is doubled. After a 1×1 convolution kernel, the number of channels from C2 to C5 is unified to 256. Downsample the C5 feature map and take the maximum value of C4 bitwise to obtain an intermediate feature map of size 40×40×256. The same operation is performed to obtain the intermediate feature maps of C4, C3, and C2. Upsample the intermediate feature map of C2 and combine it with the intermediate feature map of C3 to obtain a feature pyramid output P3 of scale 80×80×256. Repeat the above operation to obtain P4 and P5. Downsample P5 to obtain P6. The intermediate feature map of C2 is directly output to obtain P2. The spatial attention mechanism and channel attention mechanism are performed on the feature layers P2 to P6 in turn. The spatial attention mechanism has an effect on the feature maps. Take the maximum value of the 1×256 vector at each position in the , and get the width and length The same, the maximum value feature map with a channel number of 1, the feature map is softmax normalized, and each position of the normalized feature map is compared with Multiplying at the same position obtains the feature map with added spatial attention mechanism.
[0084] like Figure 5 As shown, fc represents the fully connected operation, and They represent the feature layers after maximum pooling and mean pooling respectively. represents the feature map after taking the maximum value of Pi1 and Pi2 at the same position in the feature map, and max represents the maximum value. The mean of the feature map in each channel is obtained to obtain a 1×256-dimensional vector. The vector is fully connected and softmaxed to obtain a 1×256-dimensional vector normalized to between 0 and 1. The vector is then combined by channel and The feature maps in each channel are multiplied to obtain the feature maps with the channel attention mechanism. Finally, the feature maps of the two attention mechanisms are obtained, and the maximum value of the same position in the feature map is taken bit by bit, and the residual structure is combined with They are connected to form the final feature map that is input to the detection head.
[0085] Step 3: The loss function during training is defined as follows:
[0086]
[0087] in represents focal loss; represents IOU loss; Represents the number of positive samples. The value is set to 1 to balance The weight of Represents a point on the feature map The classification score of is a positive sample, Indicates the true classification label corresponding to the point; Indicate point The distance to the four sides of the prediction box , Indicates the distance from the point to the ground truth bounding box; the summation symbol indicates the calculation of all points on the feature map P2 to P6; represents the indicator function, when The function value is 1 when , otherwise it is 0.
[0088] Step 4: Initialize the model using the pre-trained model parameters, use SGD to optimize the model, initialize the model with a learning rate of 0.01, and an iteration number of 10,000. When the iteration number reaches 6,000 and 8,000, the learning rate will be 0.001 and 0.0001 respectively. Set the batch size during training to 16, monitor the model's loss value and performance indicators, and adjust them as needed.
[0089] Step 5: Evaluate model performance using the validation set: During the training process, use the reserved validation set to evaluate the performance of the model on new data, using MAP (mean Average Precision) as the evaluation indicator.
[0090] Step 6: Adjust hyperparameters such as learning rate, batch size, and number of training rounds to improve model performance; use data augmentation techniques to increase the diversity of training data and improve the generalization ability of the model.
[0091] Step 7: After the training is completed, save the model weights to obtain the trained and optimized FCOS network model for multi-scale target detection. The detection effect in practical applications can be shown as follows: Figure 6 As shown, the box represents the detected target area.
[0092] The FCOS network multi-scale target detection method based on feature fusion and attention mechanism of the present invention is adopted. By improving the Feature Pyramid and adding the attention mechanism, the information fusion ability of the model and the feature expression of multi-scale targets are enhanced. As shown in Table 1, the accuracy of this method on the MS COCO dataset is improved by 3.2%, which proves the effectiveness of this method and provides a reference for multi-scale target detection.
[0093] Table 1 Comparative experimental data
[0094]
[0095] By improving the Feature Pyramid in the existing FCOS network, the P7 feature layer in the Feature Pyramid is deleted, the P2 feature layer is added, and a bottom-up information fusion path is introduced. The P2 feature layer is downsampled to obtain a feature layer with the same scale as P3 and fused with the P3 feature layer. The specific fusion method is to add the same position to obtain the feature layer P3 that integrates the underlying information. The P4 and P5 feature layers are fused in the same way, and the P5 feature layer is downsampled to obtain the P6 feature layer. An attention mechanism is added to the feature maps P2 to P6 that integrate the underlying feature information. The feature maps are respectively subjected to the spatial attention mechanism and the channel attention mechanism to obtain the corresponding weights. The larger value of the same position in the two weighted feature maps is taken to obtain the weight map that integrates the two attention mechanisms. The original feature map is weighted according to the weight map and connected to the original feature map through the residual structure to obtain the feature layer after the attention mechanism is added.
[0096] Based on the above, the improved Feature Pyramid improves the ability to integrate the position information of the underlying small targets, and the attention mechanism strengthens the information expression of the key features in the feature layer. Therefore, the improved FCOS network has enhanced multi-scale target detection capabilities.
[0097] It can be seen that the feature fusion method of the original fully convolutional single-stage target detection FCOS network is optimized; the spatial attention mechanism and the channel attention mechanism are added between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight, thereby enhancing the network's information fusion capability and the feature expression of multi-scale targets, and improving the detection capability of multi-scale image targets.
[0098] Another embodiment of the present invention provides a FCOS network optimization system based on feature fusion and attention mechanism, see Figure 7 , the system may include:
[0099] The optimization module 701 is used to optimize the feature fusion mode of the original full convolution single-stage target detection FCOS network;
[0100] Adding module 702 is used to add a spatial attention mechanism and a channel attention mechanism between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight.
[0101] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0102] Specifically, in this embodiment, the above storage medium may be configured to store a computer program for performing the following steps:
[0103] S201, optimizing the feature fusion method of the original full convolution single-stage target detection FCOS network;
[0104] S202, adding a spatial attention mechanism and a channel attention mechanism between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight.
[0105] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0106] Specifically, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0107] Specifically, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0108] S201, optimizing the feature fusion method of the original full convolution single-stage target detection FCOS network;
[0109] S202, adding a spatial attention mechanism and a channel attention mechanism between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight.
[0110] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings. The above is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the drawings. Any changes made according to the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which still do not exceed the spirit covered by the description and drawings, should be within the protection scope of the present invention.
Claims
1. A FCOS network optimization method based on feature fusion and attention mechanism, It is characterized in that The method comprises: Acquire an image data set, and preprocess the image data set; The feature fusion method of the original fully convolutional single-stage object detection FCOS network is optimized; A spatial attention mechanism and a channel attention mechanism are added between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight; The method of determining the spatial attention weight and the channel attention weight according to the original feature layer output by the feature pyramid, and obtaining the final output of the feature pyramid according to the original feature layer, the spatial attention weight and the channel attention weight, includes: The original feature layer output by the feature pyramid is processed by maximum pooling and mean pooling respectively; Perform softmax operation on the feature vectors obtained after pooling to obtain spatial attention weights and channel attention weights; Multiply the original feature layer by the spatial attention weight and the channel attention weight respectively to obtain the first feature map with added spatial attention and the second feature map with added channel attention; The first feature map and the second feature map are fused to obtain a third feature map that integrates the spatial attention and channel attention mechanisms; Connect the original feature map with the third feature map to obtain the final output of the feature pyramid, which is used as the final feature layer sent to the detection head; Initialize the current FCOS network model using the pre-trained model parameters, perform training based on the image dataset, evaluate the model performance using the validation set, and adjust the model hyperparameters; After the training is completed, the model weights are saved to obtain the trained and optimized FCOS network model for multi-scale object detection.
2. The method according to claim 1, It is characterized in that The optimization process of the feature fusion method includes: The first preset feature layer is added to the feature pyramid of the FCOS network, the second preset feature layer is deleted, and a bottom-up information fusion path is added.
3. The method according to claim 2, It is characterized in that The feature pyramid includes: P3, P4, P5, P6 and P7 feature layers, the first preset feature layer is the P2 feature layer, and the second preset feature layer is the P7 feature layer; Among them, the P2 feature layer is downsampled and combined with the original P3 feature layer to obtain the feature layer P3 that integrates the bottom feature information; Downsample the feature layer P3 and combine it with the original P4 feature layer to obtain the feature layer P4 that integrates the bottom feature information; Downsample the feature layer P4 and combine it with the original P5 feature layer to obtain the feature layer P5 that integrates the bottom feature information; The feature layer P5 is downsampled to obtain the feature layer P6.
4. The method according to claim 1, It is characterized in that The calculation formula of the first characteristic map is: P m =P i ·softmax(maxPooling(P i )) Among them, the P m represents the first feature map after adding the spatial attention mechanism, the P i It represents the original feature layer output by the feature pyramid, the maxPooling represents the maximum value pooling, and the softmax represents the softmax operation.
5. The method according to claim 4, It is characterized in that The calculation formula of the second characteristic map is: P a =P i ·softmax(avgPooling(P i )) Among them, the P a It represents the feature layer after adding the channel attention mechanism, and avgPooling represents mean pooling.
6. The method according to claim 5, It is characterized in that The calculation formula of the final feature layer sent to the detection head is: P=max(P m ,P a )+P i Wherein, the P is the final feature layer.
7. A FCOS network optimization system based on feature fusion and attention mechanism, It is characterized in that The system is used to obtain an image data set and preprocess the image data set; the system comprises: The optimization module is used to optimize the feature fusion method of the original fully convolutional single-stage object detection FCOS network; Add a module for adding a spatial attention mechanism and a channel attention mechanism between the feature pyramid and the detection head of the optimized FCOS network structure, wherein the spatial attention weight and the channel attention weight are determined according to the original feature layer output by the feature pyramid, and the final output of the feature pyramid is obtained according to the original feature layer, the spatial attention weight and the channel attention weight; The method of determining the spatial attention weight and the channel attention weight according to the original feature layer output by the feature pyramid, and obtaining the final output of the feature pyramid according to the original feature layer, the spatial attention weight and the channel attention weight, includes: The original feature layer output by the feature pyramid is processed by maximum pooling and mean pooling respectively; Perform softmax operation on the feature vectors obtained after pooling to obtain spatial attention weights and channel attention weights; Multiply the original feature layer by the spatial attention weight and the channel attention weight respectively to obtain the first feature map with added spatial attention and the second feature map with added channel attention; The first feature map and the second feature map are fused to obtain a third feature map that integrates the spatial attention and channel attention mechanisms; Connect the original feature map with the third feature map to obtain the final output of the feature pyramid, which is used as the final feature layer sent to the detection head; The system is also used to: initialize the current FCOS network model using the pre-trained model parameters, perform training based on the image data set, evaluate model performance using the validation set, and adjust model hyperparameters; After the training is completed, the model weights are saved to obtain the trained and optimized FCOS network model for multi-scale object detection.
8. A storage medium, It is characterized in that The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 6 when executed.
9. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.
