A method for constructing a weighted feature pyramid network for target detection
By using Gaussian matrix and exponential function to calculate weights in the feature pyramid network and adjusting the differences between feature layers, the problem of insufficient accuracy in detecting small targets is solved, and higher detection accuracy and Smaller computational costs.
Patent Information
- Application Number
- CN202111620313.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-28
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-12-28
AI Technical Summary
The existing feature pyramid network cannot effectively impart the weight of resolution and semantic information when detecting challenging images, resulting in insufficient accuracy when identifying small targets in object detection.
By introducing Gaussian matrix and exponential function to calculate weights in the feature pyramid network, the differences between feature layers are adjusted to give different weights to resolution information and semantic information, and a feature pyramid with weights is formed to improve the accuracy of target detection. .
Improves the accuracy of object detection, enables detection of small targets that are not detected by traditional feature pyramid networks, and obtains higher detection scores at smaller computational costs.
Smart Images

Figure CN114494808B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communication technology, and in particular to a method for constructing a weighted feature pyramid network for target detection. Background Art
[0002] With the progress of the times, people's convenience and efficiency of life are gradually improving, and the field of computer vision in artificial intelligence plays an important role in improving our quality of life. For example, face scanning payment, identity authentication using facial movements, etc., which are often used in daily life, in addition, face scanning at train stations, vehicle violation detection on roads, and electronic eyes are also closely related to the target detection technology in the field of computer vision. In order to make these scientific and technological achievements better serve mankind, how to improve the performance of these functions has become the focus of target detection technology. Target detection technology has been a hot topic in the field of computer vision research in recent years. It is the core part of intelligent monitoring systems and various computer disciplines, and plays a vital role in subsequent tasks such as image segmentation. Therefore, if the accuracy of the results obtained by target detection technology is not high or the target cannot be identified, it will have a significant impact on the living environment and quality of many people, and even cause some potential social problems. For example: false detection or failure to detect contraband will bring unpredictable harm, and face recognition errors may also cause Alipay to be impersonated by others, and face recognition failure or recognition errors may also cause criminals to go unpunished or wrongly accuse innocent people.
[0003] Object detection reduces the consumption of human capital through computer vision and has important display significance. The task of object detection is to find all the objects of interest in the image, determine their categories and locations, that is, it includes two subtasks: object positioning and object classification. The quality of object detection has a lot to do with the extracted features, and the extraction of feature information is affected by factors such as complex environment, similar objects, and small objects, which make object detection technology more difficult. In the past few years, deep convolutional networks have made great progress in object detection. However, like traditional object detection, deep convolutional networks also face severe challenges in detecting and locating multiple objects in a large range. Feature pyramid network is a component of object detection proposed by scholars to solve the above problems. Its role is to obtain better features, so that the model can detect objects at multiple scales, overcome these difficulties to make object detection more accurate predictions, which can not only improve the accuracy of object detection, but also lay a good foundation for subsequent tasks such as face recognition. People can use these technologies more confidently, making life more efficient and convenient.
[0004] In object detection, Feature Pyramid Networks (FPN) is one of the representative models that combines low-resolution generative pyramids. Through top-down paths and lateral connections, semantically strong features have high resolution and semantically weak features. FPN can recognize objects with different scales, thereby improving scale robustness. FPN is widely used in object detection in deep convolutional networks. For example, FPN helps Fast Region-Convolutional Neural Network (Fast R-CNN) achieve better performance. FPN helps the Region Proposal Network (RPN) obtain candidate boxes and provide feature maps for Fast R-CNN.
[0005] Several variants of FPN have been proposed for feature extraction, semantic segmentation, and other applications. For example, Neural Architecture Search Feature Pyramid Networks (NAS-FPN) is one of the FPN-based methods, which designs the search space to cover multi-scale feature representations and all possible cross-scale connections. Panoramic FPN is also a FPN-based method, which achieves state-of-the-art performance on instance, semantic segmentation, and other common tasks. However, all these methods give the same weight to resolution information and semantic information in object detection, and thus fail to detect certain objects on challenging images. To address this issue, an increasing number of recent works have proposed different methods to generate feature pyramids.
[0006] Computer vision plays an important role in the field of artificial intelligence. Reducing labor costs through computer vision has important practical significance. Because of this, computer vision has penetrated into every corner of our lives. For example, we are familiar with face recognition, vehicle detection, contraband detection at railway stations, robot navigation, intelligent video surveillance, industrial detection, aerospace and other application fields. The above visual tasks can all be attributed to target detection. Therefore, if there is a recognition error in the target detection process, it may cause serious consequences. Therefore, the speed and accuracy of target detection are increasingly valued by people, hoping to obtain high accuracy at a low computational cost.
[0007] In order to improve the accuracy of target detection and better detect small objects, Feature Pyramid Network (FPN) came into being, which has become the most common target detection model, such as Fast R-CNN, Faster R-CNN, Mask R-CNN, etc. All use the feature pyramid model. FPN uses a top-down path and lateral connections, and semantically strong features have high resolution and semantically weak features. FPN can recognize objects with different scales, thereby improving scale robustness. FPN is widely used in target detection in deep convolutional networks. Weighted Feature Pyramid Network (WFPN) generates multi-scale feature layers based on Feature Pyramid Network (FPN). WFPN constructs a feature pyramid by combining two layers through a top-down path and lateral connections. One layer is semantically strong but has a lower resolution, so WFPN upsamples this layer and then combines it with another layer with a higher resolution. WFPN uses a Gaussian kernel function to calculate weights and sets these different weight values in the lateral connection layer when WFPN creates the feature pyramid. Therefore, the features generated by WFPN have higher resolution and stronger semantics. Compared with FPN, WFPN has better accuracy and effectiveness.
[0008] 1. Traditional feature extraction
[0009] Feature extraction refers to the use of computers to extract image information and determine whether each image point belongs to an image feature. The result of feature extraction is to divide the points on the image into different subsets, which usually belong to isolated points, continuous curves or continuous regions. Traditional feature extraction methods include Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradient (HOG) and Oriented FAST and Rotated BRIEF (ORB).
[0010] SIFT finds feature points at different scales and calculates the direction of feature points. It finds some very prominent feature points, such as corner points, edge points, dark highlights and bright black points, which are not affected by light, affine transformation, noise and other factors.
[0011] HOG is used to calculate and compute the gradient direction histogram of the local area of the image to form features. HOG features combined with the Support Vector Machine (SVM) classifier have been widely used in image recognition, especially in pedestrian detection, with good performance.
[0012] ORB is based on fast corner feature point detection. It is scale and rotation invariant, as well as noise and perspective affine invariant. ORB is increasingly widely used in feature description scenarios due to its good performance.
[0013] The feature expression ability of traditional methods is not as good as that of deep learning. It is natural to replace traditional methods with deep learning.
[0014] 2. Deep Convolutional Neural Networks
[0015] In recent years, deep neural networks (DNN) have made significant progress in image classification and recognition. Some object detection methods have emerged, such as Region-Convolutional Neural Network (R-CNN), Fast Region-Convolutional Neural Network (Fast R-CNN), and Faster Region-Convolutional Neural Network (Faster R-CNN).
[0016] R-CNN is an early method of using DNN for object detection. The core idea is to use DNN to extract features of each candidate box in the image and apply SVM for classification. The classification result is a preliminary detection result, which is then used to obtain a more accurate bounding box using features of a deep convolutional neural network (DCNN) and another SVM regression model. Fast R-CNN is a model based on R-CNN that makes the training and testing process faster. Faster R-CNN uses a convolutional neural network to directly obtain the candidate box, so there is no additional time cost to obtain the candidate box. Although these methods have achieved good performance in object detection, multi-scale detection still requires better performance, especially for small objects.
[0017] 3. Method using Gaussian kernel function
[0018] In recent years, many data fusion methods use Gaussian kernel functions to calculate weights. For example, the Color-Directional-Region Merging (CDRM) method uses RGB-D images. CDRM uses a rotated bi-half Gaussian function to calculate the dissimilarity between two adjacent pixels. Based on the calculated dissimilarity, CDRM constructs an undirected weighted Region Adjacency Graph (RAG). CDRM uses a rotated bi-half Gaussian function to calculate the weight value, which enables the experiment to achieve better performance in image segmentation. In this paper, the weighted average of the pixel neighborhood is used to replace the pixel value of the point in the Gaussian filter, and the weight of each neighborhood pixel increases or decreases monotonically with the distance between the point and the center point. The Gaussian kernel function is a single-valued function. Therefore, the Gaussian kernel allows us to pay less attention to the edges of the image.
[0019] The shortcomings of traditional target detection methods such as Scale Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG) and ORB (ORB) are: weak feature expression ability, not as good as deep learning.
[0020] Disadvantages of deep convolutional neural networks: These methods have achieved good performance in object detection, but multi-scale detection still requires better performance, especially for small objects.
[0021] Therefore, a method for constructing a weighted feature pyramid network for target detection has become an urgent problem to be solved. Summary of the invention
[0022] The purpose of the present invention is to retain the original scale robustness advantage of the Feature Pyramid Network (FPN), and use weights to evaluate the influence of resolution information and semantic information when creating a feature pyramid; thereby improving target detection accuracy.
[0023] To achieve the above object, the present invention provides a technical solution: a method for constructing a weighted feature pyramid network for target detection, comprising the following steps:
[0024] Step 1: Get the feature layer of the model:
[0025] The lowest feature layer in the feature pyramid network is used, which is represented as {C2, C3, C4, C5} in the feature pyramid network; a 1×1 convolution layer is added to C5, and the result obtained after convolution is represented as feature layer D5; a 3×3 convolution layer is added to D5, and the result obtained after convolution is represented as feature layer P5; an upsampling operation is performed on P5, and the result is represented as D4; the number of channels of C4 is reduced, feature layer C4 and feature layer D4 are fused, and a 3×3 convolution layer is used to obtain feature layer P4; a 1×1 convolution layer is added to C4, and the result obtained after convolution is represented as feature layer P5; a 3×3 ... 1×1 convolutional layers; perform an upsampling operation on P4 and represent the result as feature layer D3; reduce the number of channels of C3, fuse feature layers C3 and D3, and use a 3×3 convolutional layer to obtain feature layer P3; attach a 1×1 convolutional layer to feature layer C2; attach an upsampling process on P3 and represent the result as feature layer D2; reduce the number of channels of C2, fuse feature layers C2 and D2, and use a 3×3 convolutional layer to obtain feature layer P2; finally obtain the feature layers: {C2, C3, C4, C5} and {D2, D3, D4};
[0026] Step 2: Create a Gaussian matrix:
[0027] Normalize the data: Since the data set contains multiple feature layers, feature layer C4 and feature layer D4 are used to calculate the weights;
[0028] Since the feature layers C4 and D4 have the same shape and size, C4 is selected as the basic feature layer to design the Gaussian matrix; the width and height of the feature layer C4 are represented by W and H respectively, so the coordinates of the center of the matrix are ((H-1) / 2, (W-1) / 2);
[0029]
[0030] Formula (1) is the Gaussian kernel formula; where r and c are the horizontal and vertical coordinates corresponding to each point in the matrix; σ represents the smoothness parameter of the function, which plays a role in controlling the radial range of the function;
[0031] Step 3. Calculate the difference between C4 and D4:
[0032] Since the Gaussian matrix is generated based on the shapes of feature layers C4 and D4, the shape of the Gaussian matrix is the same as that of the two layers; a 1×1 convolutional layer is attached to the Gaussian matrix to make it have the same number of channels as feature layers C4 and D4; then the obtained Gaussian matrix is convolved with feature layers C4 and D4 respectively, replacing the size of the entire layer with the size of the center value of each layer; the value obtained by convolving feature layer C4 with the Gaussian matrix is x, and the value obtained by convolving feature layer D4 with the Gaussian matrix is y;
[0033] The difference between feature layers C4 and D4 is denoted as a, then a = |xy|;
[0034] Step 4: Calculate weights:
[0035] The difference a between the feature layer C4 and the D4 layer minus a value is used as the input of the exp function. This value is set to ω, and the result becomes the input of the exp function.
[0036] b=e (a-ω ) (2)
[0037] The weights are calculated based on the ratio of a and b, and the resulting weights are expressed as ω1 and ω2;
[0038] ω1 = a / (a + b) (3)
[0039] ω2 = b / (a + b) (4)
[0040] Step 5: Calculate the feature pyramid with weights:
[0041] {F4,F3,F2}=ω1*{C4,C3,C2}+ω2*{D4,D3,D2} (5)
[0042] According to formula (5), {F2, F3, F4} is obtained. Feature layer F5 is obtained by performing a 1×1 convolution on feature layer F4. Feature layer F6 is obtained by downsampling feature layer F5, thereby obtaining {F2, F3, F4, F5, F6}. {F2, F3, F4, F5, F6} is used as the input of the RPN network in target detection, and {F2, F3, F4, F5} is the final feature pyramid with weights.
[0043] The advantages of the present invention compared with the prior art are: the present invention assigns weights on the basis of the feature pyramid, thereby achieving a higher detection accuracy; it can detect small targets that are not detected by the traditional feature pyramid, and also achieves better detection effects than the traditional feature pyramid. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is a weight-based feature pyramid network structure diagram.
[0045] Figure 2 It is a flowchart for calculating the difference between feature layers C4 and D4.
[0046] Figure 3 This is a qualitative comparison chart between the detection results of the weighted feature pyramid model and the traditional feature pyramid model.
[0047] Figure 4 It is the graph of exp function. DETAILED DESCRIPTION
[0048] The following is a further detailed description of a method for constructing a weighted feature pyramid network for target detection according to the present invention in conjunction with the accompanying drawings.
[0049] Combined with Figure 1-4 The specific implementation process of the method for constructing a weighted feature pyramid network for target detection in the present invention is as follows:
[0050] A method for constructing a weighted feature pyramid network for target detection comprises the following steps:
[0051] Step 1: Get the feature layer of the model:
[0052] The lowest feature layer in the feature pyramid network is used, which is represented as {C2, C3, C4, C5} in the feature pyramid network; a 1×1 convolution layer is added to C5, and the result obtained after convolution is represented as feature layer D5; a 3×3 convolution layer is added to D5, and the result obtained after convolution is represented as feature layer P5; an upsampling operation is performed on P5, and the result is represented as D4; the number of channels of C4 is reduced, feature layer C4 and feature layer D4 are fused, and a 3×3 convolution layer is used to obtain feature layer P4; a 1×1 convolution layer is added to C4, and the result obtained after convolution is represented as feature layer P5; a 3×3 ... 1×1 convolutional layers; perform an upsampling operation on P4 and represent the result as feature layer D3; reduce the number of channels of C3, fuse feature layers C3 and D3, and use a 3×3 convolutional layer to obtain feature layer P3; attach a 1×1 convolutional layer to feature layer C2; attach an upsampling process on P3 and represent the result as feature layer D2; reduce the number of channels of C2, fuse feature layers C2 and D2, and use a 3×3 convolutional layer to obtain feature layer P2; finally obtain the feature layers: {C2, C3, C4, C5} and {D2, D3, D4};
[0053] Step 2: Create a Gaussian matrix:
[0054] Due to the large amount of data, the data is normalized to reduce the amount of calculation. Considering that the data set used for the experiment contains multiple feature layers, feature layer C4 and feature layer D4 are used to calculate the weights. These two feature layers are selected because they have small shapes and contain less data, which can reduce the calculation time.
[0055]
[0056] The above formula is a traditional Gaussian kernel formula, where x′ represents the center of the matrix, x represents the current position, and σ represents the smoothness parameter of the function, which plays a role in controlling the radial range of the function. It is modified and the modified function is used to generate a Gaussian matrix.
[0057] Since the feature layers C4 and D4 have the same shape and size, C4 is selected as the basic feature layer to design the Gaussian matrix; the width and height of the feature layer C4 are represented by W and H respectively, so the coordinates of the center of the matrix are ((H-1) / 2, (W-1) / 2);
[0058] According to the properties of the Gaussian kernel function, changing the σ value is actually adjusting the influence of the surrounding pixels on the current pixel. The larger the σ value, the more dispersed the layout of the elements in the matrix is, so the weight of each part of the matrix is not much different, so the values of each part in the generated Gaussian matrix are roughly the same, similar to the average matrix. On the contrary, the smaller the σ value, the higher its concentration, the proportion of the middle elements is much greater than the proportion of the edge elements, resulting in the value of the central element in the obtained Gaussian matrix being much larger than the value of the edge element. On the one hand, it is hoped that the Gaussian kernel function can reduce the influence of the edge on the image; on the other hand, it is not desired to overly ignore the influence of the edge on the image. Therefore, a relatively large σ value is selected.
[0059]
[0060] Formula (1) is the Gaussian kernel formula; where r and c are the horizontal and vertical coordinates corresponding to each point in the matrix; σ represents the smoothness parameter of the function, which plays a role in controlling the radial range of the function;
[0061] Step 3. Calculate the difference between C4 and D4:
[0062] Since the Gaussian matrix is generated based on the shapes of feature layers C4 and D4, the shape of the Gaussian matrix is the same as that of the two layers; a 1×1 convolutional layer is attached to the Gaussian matrix to make it have the same number of channels as feature layers C4 and D4; then the obtained Gaussian matrix is convolved with feature layers C4 and D4 respectively, replacing the size of the entire layer with the size of the center value of each layer; the value obtained by convolving feature layer C4 with the Gaussian matrix is x, and the value obtained by convolving feature layer D4 with the Gaussian matrix is y;
[0063] The difference between feature layers C4 and D4 is denoted as a, then a = |xy|;
[0064] Step 4: Calculate weights:
[0065] The larger the values of x and y obtained in step 3, the more information is contained in the corresponding layer. If x is greater than y, a larger weight is assigned to the feature layer set {C2, C3, C4} corresponding to feature layer C4, and a smaller weight is assigned to the feature layer set {D2, D3, D4} corresponding to feature layer D4; otherwise, a smaller weight is assigned to the feature layer set {C2, C3, C4} corresponding to feature layer C4, and a larger weight is assigned to the feature layer set {D2, D3, D4} corresponding to feature layer D4.
[0066] In the present invention, it is hoped that the greater the difference between the feature layers C4 and D4, the greater the difference in the corresponding weight values. To achieve this goal, the exp function is selected.
[0067] Combined with Figure 4 , the properties of the exp function just meet this requirement. In order to make the exp function more demanding, the difference a between the feature layer C4 and the D4 layer minus a value is used as the input of the exp function. This value is set as ω, and the result becomes the input of the exp function; the size of the ω value is set according to the experimental data.
[0068] b=e (a-ω ) (2)
[0069] The weights are calculated based on the ratio of a and b, assigning larger weights to layers containing more information. The resulting weights are denoted as ω1 and ω2;
[0070] ω1 = a / (a + b) (3)
[0071] ω2=b / (a+b) (4)
[0072] Step 5: Calculate the feature pyramid with weights:
[0073] {F4,F3,F2}=ω1*{C4,C3,C2}+ω2*{D4,D3,D2} (5)
[0074] According to formula (5), {F2, F3, F4} is obtained. Feature layer F5 is obtained by performing a 1×1 convolution on feature layer F4. Feature layer F6 is obtained by downsampling feature layer F5, thereby obtaining {F2, F3, F4, F5, F6}. {F2, F3, F4, F5, F6} is used as the input of the RPN network in target detection, and {F2, F3, F4, F5} is the final feature pyramid with weights.
[0075] The final algorithm flow for generating feature pyramids in the weight-based feature pyramid model is as follows:
[0076] Input: {C5, C4, C3, C2}
[0077] Output: Final feature pyramid
[0078]
[0079] Return the final feature pyramid {F2, F3, F4}
[0080] Combined with Figure 3 ,Comparing the application of the model provided by the present invention with FPN, it can be seen that the present invention has better performance than FPN.
[0081] The weighted feature pyramid and traditional models were applied to Pascal VOC2007 to compare the results, and the results are shown in Table 1.
[0082] Table 1: Comparison of the effects of weighted feature pyramid and traditional models applied to Pascal VOC2007
[0083]
[0084] According to Table 1, using WFPN in Faster R-CNN improves mAP100 from 50.9 to 51.8, and mAP1K from 79.1 to 79.5, while increasing the computation time cost slightly. Although calculating weights based on each layer is better than calculating weights based on C4 and D4 layers, for example, WFPN improves mAP100 from 50.9 to 52.3, the computation time is longer. In order to balance computation efficiency and accuracy, only weights based on C4 and D4 layers are calculated.
[0085] Combined with Figure 3 As can be seen from Table 1, the present invention achieves good results when used in Fast R-CNN and detects objects in the image that cannot be detected by FPN. These results show that it is important to consider the impact of resolution information and semantic information on different images.
[0086] The present invention uses the PASCAL dataset produced in 2007, including 5011 train images and 4952 test images, to compare with the traditional feature pyramid network algorithm to evaluate the algorithm. mAP is used to compare the results generated by different methods.
[0087] It can be seen from Table 1 that the weighted feature pyramid (using each feature layer) improves mAP10K from 50.9 to 52.3, which is 0.3 higher than the result of calculating weights based only on feature layers C4 and D4, but it takes 12 seconds longer. Therefore, considering that the method of calculating weights based on each feature layer has higher results and longer time consumption than the method of calculating weights based only on feature layers C4 and D4, the present invention hopes to obtain high accuracy with relatively less time consumption in order to better calculate the relationship between time and accuracy. Therefore, the weighted feature pyramid model based on feature layers C4 and D4, which consumes relatively less time and has higher accuracy, is finally selected as the final result model.
[0088] On the Pascal VOC2007 dataset, compared with the traditional feature pyramid model, the weighted feature pyramid model applied in the Fast R-CNN model achieved higher detection accuracy than several other comparison models, and detected small targets that were not detected by the traditional feature pyramid. It also achieved higher detection scores than the traditional feature pyramid. These results show that it is necessary to consider the impact of the resolution information and semantic information content of different images on target detection, which directly proves the feasibility of the weighted feature pyramid model. At the same time, the good results on both datasets also show that the weighted feature pyramid model has a certain versatility for different datasets.
[0089] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.
Claims
1. A method for constructing a weighted feature pyramid network for target detection, characterized in that: The following steps are involved: Step 1: Get the feature layer of the model: Use the lowest feature layer in the feature pyramid network, which is represented as {C2, C3, C4, C5} in the feature pyramid network; add a 1×1 convolution layer to C5, and represent the result after convolution as feature layer D5; A 3×3 convolutional layer is added to D5, and the result obtained after convolution is represented as feature layer P5; Perform an upsampling operation on P5 and represent the result as D4; reduce the number of channels of C4, fuse feature layer C4 and feature layer D4, and use a 3×3 convolution layer to obtain feature layer P4; Add a 1×1 convolutional layer to C4; perform an upsampling operation on P4 and represent the result as feature layer D3; Reduce the number of channels of C3, fuse feature layer C3 and feature layer D3, and use a 3×3 convolution layer to obtain feature layer P3; A 1×1 convolutional layer is added to the feature layer C2; an upsampling process is added to P3, and the result is represented as the feature layer D2; Reduce the number of channels of C2, fuse feature layer C2 and feature layer D2, and use a 3×3 convolutional layer to obtain feature layer P2; finally, we get the feature layers: {C2, C3, C4, C5} and {D2, D3, D4}; Step 2: Create a Gaussian matrix: Normalize the data: Since the data set contains multiple feature layers, feature layer C4 and feature layer D4 are used to calculate the weights; Since the feature layers C4 and D4 have the same shape and size, C4 is selected as the basic feature layer to design the Gaussian matrix; the width and height of the feature layer C4 are represented by W and H respectively, so the coordinates of the center of the matrix are ((H-1) / 2, (W-1) / 2); Formula (1) is the Gaussian kernel formula; where r and c are the horizontal and vertical coordinates corresponding to each point in the matrix; σ represents the smoothness parameter of the function, which plays a role in controlling the radial range of the function; Step 3. Calculate the difference between C4 and D4: Since the Gaussian matrix is generated based on the shapes of feature layers C4 and D4, the shape of the Gaussian matrix is the same as that of the two layers; a 1×1 convolutional layer is attached to the Gaussian matrix to make it have the same number of channels as feature layers C4 and D4; then the obtained Gaussian matrix is convolved with feature layers C4 and D4 respectively, replacing the size of the entire layer with the size of the center value of each layer; the value obtained by convolving feature layer C4 with the Gaussian matrix is x, and the value obtained by convolving feature layer D4 with the Gaussian matrix is y; The difference between feature layers C4 and D4 is denoted as a, then a = |xy|; Step 4: Calculate weights: The difference a between the feature layer C4 and the D4 layer minus a value is used as the input of the exp function. This value is set to ω, and the result becomes the input of the exp function. b=e (a-ω ) (2) The weights are calculated based on the ratio of a and b, and the resulting weights are expressed as ω1 and ω2; ω1 = a / (a + b) (3) ω2 = b / (a + b) (4) Step 5: Calculate the feature pyramid with weights: {F4, F3, F2} = ω1*{C4, C3, C2} + ω2*{D4, D3, D2} (5) According to formula (5), {F2, F3, F4} is obtained, and then the final feature pyramid with weights is obtained.
