A feature fusion method and target detection network based on gating mechanism

Abstract

The present invention relates to a feature fusion method and target detection network based on a gating mechanism, comprising: obtaining low-level detail features and high-level semantic features of a detection target image; using the low-level detail features and high-level semantic features as input features of a gating fusion module; splicing the low-level detail features and high-level semantic features, performing a convolution operation, and obtaining an update gating signal and a reset gating signal using a first activation function; filtering the low-level detail features using the reset gating signal to obtain the filtered low-level detail features; splicing the filtered low-level detail features with the high-level semantic features, performing a convolution operation, and obtaining the current candidate features using a second activation function; selectively fusing the low-level detail features and the current candidate features using the update gating signal to obtain a fused feature, and using the fused feature for target detection. The present invention improves the accuracy of target detection.

Description

Technical Field

[0001] The present invention relates to the field of feature fusion, and in particular to a feature fusion method and a target detection network based on a gating mechanism. Background Art

[0002] In the task of using convolutional neural networks for target detection, the depth of the convolutional neural network is crucial to the performance of the model. When the number of convolutional neural network layers is increased, the convolutional neural network can extract more complex feature patterns, so when the model is deeper, better results can be achieved in theory. However, in practice, as the depth of the convolutional neural network increases, the accuracy of convolutional neural network detection becomes saturated or even decreases. The residual network solves this problem by using short-circuit connections, but the residual network performs congruent mapping and passes redundant features to the subsequent convolutional layers. The redundant information will interfere with the detection and affect the detection of objects. In addition, the existing feature fusion method does not take into account the selectivity of features and cannot fully utilize the low-level detail features and high-level semantic features, resulting in a low accuracy rate for target detection. Summary of the invention

[0003] The purpose of the present invention is to provide a feature fusion method and a target detection network based on a gating mechanism to solve the problem of low accuracy of target detection using convolutional neural networks in the prior art.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A feature fusion method based on a gating mechanism, comprising:

[0006] Acquire low-level detail features and high-level semantic features of the detection target image; the low-level detail features and the high-level semantic features are used as input features of the gated fusion module;

[0007] The low-level detail features and the high-level semantic features are concatenated, a convolution operation is performed, and an update gating signal and a reset gating signal are obtained by using a first activation function; the first activation function is a sigmoid function;

[0008] Filtering the low-level detail features using the reset gating signal to obtain filtered low-level detail features;

[0009] The filtered low-level detail features are concatenated with the high-level semantic features, a convolution operation is performed, and a second activation function is used to obtain the current candidate features; the second activation function is a Tanh function;

[0010] Selectively fuse the low-level detailed features and the current candidate features using the updated gating signal to obtain a fused feature, which is used for object detection.

[0011] Optionally, the splicing of the low-level detailed features and the high-level semantic features, performing a convolution operation, and obtaining an updated gating signal and a reset gating signal using a first activation function specifically includes:

[0012] Using the formula z = σ(W z , (f n , f n-1 )) to splice the low-level detailed features and the high-level semantic features, perform a convolution operation, and obtain the updated gating signal;

[0013] Using the formula r = σ(W r , (f n , f n-1 )) to splice the low-level detailed features and the high-level semantic features, perform a convolution operation, and obtain the reset gating signal;

[0014] Among them, σ is the first activation function, z is the updated gating signal, 0 < z < 1, r is the reset gating signal, 0 < r < 1; f n is the high-level semantic feature, f n-1 is the low-level detailed feature; W z and W r are the weights of the corresponding convolution.

[0015] Optionally, filtering the low-level detailed features using the reset gating signal to obtain filtered low-level detailed features specifically includes:

[0016] Using the formula f n-1′ = f n-1 ⊙r to filter the low-level detailed features to obtain filtered low-level detailed features;

[0017] Among them, f n-1’ is the filtered low-level detailed feature, ⊙ represents element-wise multiplication in the matrix. When the reset gating signal approaches 0, the low-level detailed features are discarded, and when the reset gating signal approaches 1, the low-level detailed features are retained.

[0018] Optionally, splicing the filtered low-level detailed features with the high-level semantic features, performing a convolution operation, and obtaining the current candidate feature using a second activation function specifically includes:

[0019] Using the second activation function tanh and the formula f n′ = tanh(W h (fn , f n-1′ )) obtain the current candidate feature;

[0020] Among them, W h is the weight of the corresponding convolution, f n’ is the current candidate feature.

[0021] Optionally, the selectively fusing the low-level detail feature and the current candidate feature using the updated gating signal to obtain a fused feature specifically includes:

[0022] Using formula f GF =z⊙f n-1 +(1-z)⊙f n′ Get fusion features;

[0023] Among them, f GF Represents fusion features.

[0024] A target detection network based on a gating mechanism, comprising a trunk network and a branch network;

[0025] Detecting a target image as input to the target detection network;

[0026] The backbone network includes a plurality of first feature fusion modules; the plurality of first feature fusion modules are connected in sequence, and the pooling layer of the first feature fusion module at the current level is connected to the first convolutional layer of the first feature fusion module at the next level; the first feature fusion module is used to perform feature fusion on the detection target image to obtain fusion features;

[0027] The branch network includes a cascade module, a prediction module, a second feature fusion module and a third feature fusion module; the trunk network is connected to the cascade module and the second feature fusion module respectively; the cascade module is connected to the third feature fusion module; the prediction module is connected to the second feature fusion module and the third feature fusion module respectively; the cascade module is used to cascade the fusion features output by the trunk network and the high-level semantic features output by the first feature fusion module of the penultimate level of the trunk network to obtain the cascaded fusion features; the second feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the fusion features output by the trunk network to obtain the second fusion features; the third feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the cascaded fusion features to obtain the third fusion features; the prediction module is used to predict the multi-scale detection target image.

[0028] Optionally, the first feature fusion module specifically includes: a first convolution layer, a second convolution layer, a pooling layer and a first gated fusion module;

[0029] The input end of the first gated fusion module is connected to the first convolutional layer and the second convolutional layer respectively, and the output end of the first gated fusion module is connected to the pooling layer; the pooling layer of the last-level first feature fusion module of the backbone network is connected to the third convolutional layer of the second feature fusion module; the first convolutional layer is used to extract the low-level detail features of the detection target image; the second convolutional layer is used to extract the high-level semantic features of the detection target image; the first gated fusion module is used to filter the low-level detail features, and fuse the filtered low-level detail features with the high-level semantic features; the pooling layer is used to reduce the parameters of the fusion features; the first-level first feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the detection target image; each level of the first feature fusion module except the first-level first feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the fusion features output by the previous level first feature fusion module.

[0030] Optionally, the second feature fusion module specifically includes: a third convolution layer, a fourth convolution layer and a second gated fusion module; the input end of the second gated fusion module is connected to the third convolution layer and the fourth convolution layer respectively, and the output end of the second gated fusion module is connected to the prediction module;

[0031] The third feature fusion module specifically includes: a fifth convolution layer, a sixth convolution layer and a third gated fusion module; the input end of the third gated fusion module is connected to the fifth convolution layer and the sixth convolution layer respectively, and the output end of the third gated fusion module is connected to the prediction module;

[0032] The third convolutional layer is used to extract low-level detail features of the fused features output by the backbone network; the fifth convolutional layer is used to extract low-level detail features of the fused features after cascading; the fourth convolutional layer is used to extract high-level semantic features of the fused features output by the backbone network; the sixth convolutional layer is used to extract high-level semantic features of the fused features after cascading; the second gated fusion module is used to filter out the low-level detail features of the fused features output by the backbone network, and to fuse the filtered low-level detail features of the fused features output by the backbone network and the high-level semantic features of the fused features output by the backbone network; the third gated fusion module is used to filter out the low-level detail features of the fused features after cascading, and to fuse the filtered low-level detail features of the cascaded fused features and the high-level semantic features of the cascaded fused features.

[0033] Optionally, the input end of the cascade module is respectively connected to the pooling layer of the last-level first feature fusion module of the backbone network and the second convolution layer of the second-to-last-level first feature fusion module; the output end of the cascade module is connected to the fifth convolution layer of the third fusion module.

[0034] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0035] The feature fusion method based on the gating mechanism provided by the present invention screens the low-level detail features by updating the gating signal and resetting the gating signal, filters out redundant low-level detail features, avoids passing the redundant low-level detail features to the subsequent convolutional layers, and avoids the problem that the redundant low-level detail features interfere with the detection. The high-level semantic features and the low-level detail features can be better integrated, thereby improving the accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 A flow chart of a feature fusion method based on a gating mechanism provided by the present invention;

[0038] Figure 2 A schematic diagram of a target detection network based on a gating mechanism provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the gated fusion module provided by the present invention.

[0040] Explanation of symbols: 1-first convolutional layer; 2-second convolutional layer; 3-first gated fusion module; 4-pooling layer; 5-third convolutional layer; 6-fourth convolutional layer; 7-second gated fusion module; 8-fifth convolutional layer; 9-sixth convolutional layer; 10-third gated fusion module; 11-cascade module; 12-prediction module; 13-reset gate; 14-update gate. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The purpose of the present invention is to provide a feature fusion method and a target detection network based on a gating mechanism to solve the problem of low accuracy of target detection using convolutional neural networks in the prior art.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] In the target detection task, the convolutional layer uses the expressive power of the convolutional neural network to learn features based on the image data of the training set, and optimizes the learned weights through back propagation to achieve target detection. Generally speaking, the low-level convolutional layers extract detailed features such as the texture of the object, while the high-level convolutional layers extract high-level semantic features.

[0045] The low-level convolutional layers include too much useless information, while the high-level convolutional layers lack detailed information about the object. Therefore, the gated fusion module is embedded in the feature extraction network. The gated fusion module is used to screen the features extracted by the convolutional layer, filter out the noise irrelevant to target detection, and pass the features that are more effective for detecting the target to the subsequent feature layers, forming a fusion of different features, which has both high-level semantic features and low-level detailed features, enhances the representation ability of target features, and thus improves the accuracy of target detection.

[0046] Figure 3 The schematic diagram of the gated fusion module provided by the present invention is as follows: Figure 3 As shown in Figure 1, the gated fusion module has two inputs, which can be the features of adjacent convolutional layers or the features of interval convolutional layers. Taking adjacent convolutional layers as an example to introduce the working mechanism of the gated fusion module, suppose the feature after a certain convolutional layer is f n-1 , the feature after the next convolutional layer is f n .

[0047] Figure 1 A flow chart of a feature fusion method based on a gating mechanism provided by the present invention, such as Figure 1 and Figure 3 As shown, a feature fusion method based on a gating mechanism includes:

[0048] Step 101: Obtain low-level detail features and high-level semantic features of a detection target image. The low-level detail features and the high-level semantic features are used as input features of a gated fusion module.

[0049] Step 102: concatenate the low-level detail features and the high-level semantic features, perform a convolution operation, and use a first activation function to obtain an update gating signal and a reset gating signal. The first activation function is a sigmoid function.

[0050] Step 102 specifically includes:

[0051] Using the formula z = σ(W z , (f n , f n-1 )) to splice the low-level detail features and the high-level semantic features, perform a convolution operation, and obtain the updated gating signal.

[0052] Using the formula r = σ(W r , (f n , f n-1 )) to splice the low-level detail features and the high-level semantic features, perform a convolution operation, and obtain the reset gating signal.

[0053] Among them, σ is the first activation function, z is the updated gating signal, 0 < z < 1, r is the reset gating signal, 0 < r < 1; f n is the high-level semantic feature, f n-1 is the low-level detail feature; W z and W r are the weights of the corresponding convolution.

[0054] Step 103: Use the reset gating signal to filter the low-level detail features to obtain the filtered low-level detail features.

[0055] Step 103 specifically includes:

[0056] Through the reset gate 13, use the formula f n-1′ = f n-1 ⊙r to filter the low-level detail features to obtain the filtered low-level detail features. Among them, f n-1’ is the filtered low-level detail feature, ⊙ represents element-wise multiplication in the matrix. When r is close to 0, the low-level detail features are discarded, and when r is close to 1, the low-level detail features are retained.

[0057] Step 104: Splice the filtered low-level detail features and the high-level semantic features, perform a convolution operation, and use the second activation function to obtain the current candidate features. The second activation function is the Tanh function.

[0058] Step 104 specifically includes:

[0059] Through the update gate 14, use the second activation function tanh and the formula f n′ = tanh(W h (f n , f n-1′ )) to obtain the current candidate features; where W h is the weight of the corresponding convolution, f n’is the current candidate feature. n’ It helps the gated fusion module to remember information for a longer period of time.

[0060] Step 105: selectively fuse the low-level detail features and the current candidate features using the updated gating signal to obtain fused features, which are used for target detection.

[0061] The step 105 specifically includes:

[0062] Using formula f GF =z⊙f n-1 +(1-z)⊙f n′ Get the fusion feature; among them, f GF Represents fusion features.

[0063] z⊙f n-1 Represents the low-level detail features f related to detection n-1 The choice of can also be seen as the discarding of irrelevant information, (1-z)⊙f n′ Indicates how many current candidate features should be added. When the weight z is used to adjust the low-level detail features f of the previous stage n-1 When forgetting, the current candidate feature f is selected with a weight of (1-z) n′ Make up for it and maintain a "balanced" state.

[0064] Figure 2 The schematic diagram of a target detection network based on a gating mechanism provided by the present invention is as follows: Figure 2 As shown, a target detection network based on a gating mechanism includes a trunk network and a branch network. A detected target image is used as an input of the target detection network.

[0065] The backbone network includes multiple first feature fusion modules; the multiple first feature fusion modules are connected in sequence, and the pooling layer of the first feature fusion module at the current level is connected to the first convolutional layer of the first feature fusion module at the next level; the first feature fusion module is used to perform feature fusion on the detection target image to obtain fused features.

[0066] The first feature fusion module specifically includes: a first convolutional layer 1, a second convolutional layer 2, a pooling layer 4 and a first gated fusion module 3.

[0067] The input end of the first gated fusion module 3 is connected to the first convolutional layer 1 and the second convolutional layer 2 respectively, and the output end of the first gated fusion module 3 is connected to the pooling layer 4; the pooling layer 4 of the last-level first feature fusion module of the backbone network is connected to the third convolutional layer 5 of the second feature fusion module; the first convolutional layer 1 is used to extract the low-level detail features of the detection target image; the second convolutional layer 2 is used to extract the high-level semantic features of the detection target image; the first gated fusion module 3 is used to filter the low-level detail features, and fuse the filtered low-level detail features with the high-level semantic features; the pooling layer 4 is used to reduce the parameters of the fusion features; the first-level first feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the detection target image; each level of the first feature fusion module except the first-level first feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the fusion features output by the previous level first feature fusion module.

[0068] The branch network includes a cascade module 11, a prediction module 12, a second feature fusion module and a third feature fusion module; the trunk network is connected to the cascade module 11 and the second feature fusion module respectively; the cascade module 11 is connected to the third feature fusion module; the prediction module 12 is connected to the second feature fusion module and the third feature fusion module respectively; the cascade module 11 is used to cascade the fusion features output by the trunk network and the high-level semantic features output by the first feature fusion module of the penultimate level of the trunk network to obtain the cascaded fusion features; the second feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the fusion features output by the trunk network to obtain the second fusion features; the third feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the cascaded fusion features to obtain the third fusion features; the prediction module 12 is used to predict the multi-scale detection target image.

[0069] The second feature fusion module specifically includes: a third convolutional layer 5, a fourth convolutional layer 6 and a second gated fusion module 7; the input end of the second gated fusion module 7 is respectively connected to the third convolutional layer 5 and the fourth convolutional layer 6, and the output end of the second gated fusion module 7 is connected to the prediction module 12.

[0070] The third feature fusion module specifically includes: a fifth convolutional layer 8, a sixth convolutional layer 9 and a third gated fusion module 10; the input end of the third gated fusion module 10 is respectively connected to the fifth convolutional layer 8 and the sixth convolutional layer 9, and the output end of the third gated fusion module 10 is connected to the prediction module 12.

[0071] The third convolutional layer 5 is used to extract low-level detail features of the fused features output by the backbone network; the fifth convolutional layer 8 is used to extract low-level detail features of the fused features after cascading; the fourth convolutional layer 6 is used to extract high-level semantic features of the fused features output by the backbone network; the sixth convolutional layer 9 is used to extract high-level semantic features of the fused features after cascading; the second gated fusion module 7 is used to filter out the low-level detail features of the fused features output by the backbone network, and to fuse the filtered low-level detail features of the fused features output by the backbone network and the high-level semantic features of the fused features output by the backbone network; the third gated fusion module 10 is used to filter out the low-level detail features of the fused features after cascading, and to fuse the filtered low-level detail features of the cascaded fused features and the high-level semantic features of the cascaded fused features.

[0072] The input end of the cascade module 11 is respectively connected to the pooling layer 4 of the last-level first feature fusion module of the backbone network and the second convolution layer 2 of the second-to-last-level first feature fusion module; the output end of the cascade module 11 is connected to the fifth convolution layer 8 of the third fusion module.

[0073] In order to verify the effectiveness of the gated fusion module, the gated fusion module is combined with the Yolov3-Tiny target detection framework to construct a target detection network based on the gated mechanism. Figure 2As shown, the network input is an RGB image, and the first convolution layer 1 is used to extract the features of the object to obtain low-level detail features. The low-level detail features are passed through the second convolution layer 2 to obtain the corresponding high-level semantic features. The low-level detail features and high-level semantic features are used as two inputs and input to the gated fusion module. The fusion method of the above steps is used for fusion, and the obtained fusion features enter the pooling layer 4. After some processing, some unnecessary fusion feature parameters are filtered out. Similarly, the gated fusion module is added between the first convolution layer 1 and the second convolution layer 2 of the second-to-last level of the backbone network and the first convolution layer 1 and the second convolution layer 2 of the last level of the backbone network for feature fusion. The Yolov3-Tiny structure has two scale branches. Another branch is separated from the second convolution layer 2 of the second-to-last level and cascaded with the pooling layer 4 of the last level of the backbone network to detect objects of larger scale. A gated fusion module is added between the third convolutional layer 5 and the fourth convolutional layer 6 and the corresponding branch position to realize information fusion. The multi-layer gated fusion structure autonomously selects and adaptively stores the features of different feature layers, better integrates the low-level detail features and high-level semantic features, thereby forming a fusion feature with stronger expression ability, which helps to improve the performance of target detection. The prediction module 12 uses a feature pyramid structure to fuse feature information of two different scales for prediction. The results generated by the convolutional layer generate different feature maps through two scale branches, predict targets of different sizes, and output the position of the object box, the category of the object, and the corresponding probability.

[0074] Dataset: Target detection and recognition dataset. The dataset contains 20 categories of targets. Each image in the dataset corresponds to an XML-type label file, which indicates the category of the object in the image, the coordinates of the upper left corner and the lower right corner of the actual frame of the object.

[0075] Evaluation indicators: The mean average precision (mAP) is used as the evaluation indicator. The average precision AP refers to the indicator of one category. For multiple categories, the mAP is obtained by averaging the AP of each category.

[0076] The existing feature fusion method does not take into account the selectivity of features and cannot make full use of low-level detail features and high-level semantic features. The gated fusion module uses the reset gate signal and the update gate signal learning weight to control the information that needs to be retained or discarded. GF =z⊙f n-1 +(1-z)⊙f n′ It realizes the adaptive fusion of the output features of the previous stage and the input features of another stage. It selects the detail features of the low layer, retains the detail features that are more effective for detecting the target, and fuses them with the high layer features to enhance the expressiveness of the features.

[0077] Inspired by the selective memory function of recurrent neural networks, a gated fusion module is designed to filter the features extracted by the convolutional layer, pass the features that are effective for target detection to the next layer, and add gated fusion modules at different positions of the detection network to form a multi-layer fusion structure, which not only filters out redundant features, but also better integrates low-level detail features related to detection with high-level semantic features, thereby enhancing the expressiveness of features for objects. Experiments have shown that the average accuracy of the detection network with a gated fusion module is 5% higher than that without a gated fusion module.

[0078] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0079] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A feature fusion method based on a gating mechanism, characterized in that: include: Acquire low-level detail features and high-level semantic features of the detection target image; the low-level detail features and the high-level semantic features are used as input features of the gated fusion module; The low-level detail features and the high-level semantic features are concatenated, a convolution operation is performed, and an update gating signal and a reset gating signal are obtained by using a first activation function; the first activation function is a sigmoid function; The step of concatenating the low-level detail features and the high-level semantic features, performing a convolution operation, and obtaining an update gating signal and a reset gating signal using a first activation function specifically includes: Using the formula z=σ(W z ,(f n ,f n-1 )) concatenating the low-level detail features and the high-level semantic features, performing a convolution operation, and obtaining the update gating signal; Using the formula r = σ(W r ,(f n ,f n-1 )) concatenating the low-level detail features and the high-level semantic features, performing a convolution operation, and obtaining the reset gating signal; Among them, σ is the first activation function, z is the update gate signal, 0 < z < 1, r is the reset gate signal, 0 < r < 1; f n is the high-level semantic feature, f n-1 is the low-level detail feature; W z and W r are the weights of the corresponding convolution; Filtering the low-level detail features using the reset gating signal to obtain filtered low-level detail features; The filtered low-level detail features are concatenated with the high-level semantic features, a convolution operation is performed, and a second activation function is used to obtain the current candidate features; the second activation function is a Tanh function; The low-level detail feature and the current candidate feature are selectively fused using the updated gating signal to obtain a fused feature, and the fused feature is used for target detection.

2. A feature fusion method based on a gating mechanism according to claim 1, characterized in that: The filtering of the low-level detail features by using the reset gating signal to obtain the filtered low-level detail features specifically includes: Using formula f n-1' =f n-1 ⊙r filtering out the low-level detail features to obtain filtered low-level detail features; Among them, f n-1’ is the low-level detail feature after filtering, ⊙ represents the multiplication of the corresponding elements in the matrix, when the reset gating signal is close to 0, the low-level detail feature is discarded, and when the reset gating signal is close to 1, the low-level detail feature is retained.

3. A feature fusion method based on a gating mechanism according to claim 2, characterized in that: The step of concatenating the filtered low-level detail features with the high-level semantic features, performing a convolution operation, and obtaining the current candidate features using a second activation function specifically includes: Using the second activation function tanh and formula f n' =tanh(W h (f n ,f n-1' )) obtain the current candidate feature; Among them, W h is the weight of the corresponding convolution, f n’ is the current candidate feature.

4. The feature fusion method based on the gating mechanism according to claim 3 is characterized in that: The selective fusion of the low-level detail feature and the current candidate feature using the updated gating signal to obtain the fused feature specifically includes: Using formula f GF =z⊙f n-1 +1-z⊙f n' Get fusion features; Among them, f GF Represents fusion features.

5. A target detection method based on a gating mechanism, characterized in that: The target detection method based on the gating mechanism is implemented by using a target detection network, and the target detection network includes a trunk network and a branch network; Detecting a target image as input to the target detection network; The backbone network includes a plurality of first feature fusion modules; the plurality of first feature fusion modules are connected in sequence, and the pooling layer of the first feature fusion module at the current level is connected to the first convolutional layer of the first feature fusion module at the next level; the first feature fusion module is used to perform feature fusion on the detection target image to obtain fusion features; The branch network includes a cascade module, a prediction module, a second feature fusion module and a third feature fusion module; the trunk network is connected to the cascade module and the second feature fusion module respectively; the cascade module is connected to the third feature fusion module; the prediction module is connected to the second feature fusion module and the third feature fusion module respectively; the cascade module is used to cascade the fusion features output by the trunk network and the high-level semantic features output by the first feature fusion module of the penultimate level of the trunk network to obtain the cascaded fusion features; the second feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the fusion features output by the trunk network to obtain the second fusion features; the third feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the cascaded fusion features to obtain the third fusion features; the prediction module is used to predict the multi-scale detection target image; The target detection network based on the gating mechanism is used to implement the feature fusion method based on the gating mechanism according to any one of claims 1 to 4, and the feature fusion method based on the gating mechanism includes: Obtain low-level detail features and high-level semantic features of the detection target image; The low-level detail features and the high-level semantic features are concatenated, a convolution operation is performed, and an update gating signal and a reset gating signal are obtained by using a first activation function; the first activation function is a sigmoid function; The step of concatenating the low-level detail features and the high-level semantic features, performing a convolution operation, and obtaining an update gating signal and a reset gating signal using a first activation function specifically includes: Using the formula z=σ(W z ,f n ,f n-1 )) concatenating the low-level detail features and the high-level semantic features, performing a convolution operation, and obtaining the update gating signal; Using the formula r = σ(W r ,f n ,f n-1 )) concatenating the low-level detail features and the high-level semantic features, performing a convolution operation, and obtaining the reset gating signal; Among them, σ is the first activation function, z is the update gate signal, where 0 < z < 1, r is the reset gate signal, where 0 < r < 1; f n is the high-level semantic feature, and f n-1 is the low-level detail feature; W z and W r are the weights of the corresponding convolution; Filtering the low-level detail features using the reset gating signal to obtain filtered low-level detail features; The filtered low-level detail features are concatenated with the high-level semantic features, a convolution operation is performed, and a second activation function is used to obtain the current candidate features; the second activation function is a Tanh function; The low-level detail feature and the current candidate feature are selectively fused using the updated gating signal to obtain a fused feature, and the fused feature is used for target detection.

6. The target detection method based on the gating mechanism according to claim 5, characterized in that: The first feature fusion module specifically includes: a first convolutional layer, a second convolutional layer, a pooling layer and a first gated fusion module; The input end of the first gated fusion module is connected to the first convolutional layer and the second convolutional layer respectively, and the output end of the first gated fusion module is connected to the pooling layer; the pooling layer of the last-level first feature fusion module of the backbone network is connected to the third convolutional layer of the second feature fusion module; the first convolutional layer is used to extract the low-level detail features of the detection target image; the second convolutional layer is used to extract the high-level semantic features of the detection target image; the first gated fusion module is used to filter the low-level detail features, and fuse the filtered low-level detail features with the high-level semantic features; the pooling layer is used to reduce the parameters of the fusion features; the first-level first feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the detection target image; each level of the first feature fusion module except the first-level first feature fusion module is used to perform feature fusion on the low-level detail features and high-level semantic features of the fusion features output by the previous level first feature fusion module.

7. The target detection method based on the gating mechanism according to claim 6, characterized in that: The second feature fusion module specifically includes: a third convolution layer, a fourth convolution layer and a second gated fusion module; the input end of the second gated fusion module is connected to the third convolution layer and the fourth convolution layer respectively, and the output end of the second gated fusion module is connected to the prediction module; The third feature fusion module specifically includes: a fifth convolution layer, a sixth convolution layer and a third gated fusion module; the input end of the third gated fusion module is connected to the fifth convolution layer and the sixth convolution layer respectively, and the output end of the third gated fusion module is connected to the prediction module; The third convolutional layer is used to extract low-level detail features of the fused features output by the backbone network; the fifth convolutional layer is used to extract low-level detail features of the fused features after cascading; the fourth convolutional layer is used to extract high-level semantic features of the fused features output by the backbone network; the sixth convolutional layer is used to extract high-level semantic features of the fused features after cascading; the second gated fusion module is used to filter out the low-level detail features of the fused features output by the backbone network, and to fuse the filtered low-level detail features of the fused features output by the backbone network and the high-level semantic features of the fused features output by the backbone network; the third gated fusion module is used to filter out the low-level detail features of the fused features after cascading, and to fuse the filtered low-level detail features of the cascaded fused features and the high-level semantic features of the cascaded fused features.

8. The target detection method based on the gating mechanism according to claim 7, characterized in that: The input end of the cascade module is respectively connected to the pooling layer of the last-level first feature fusion module of the backbone network and the second convolution layer of the second-to-last-level first feature fusion module; the output end of the cascade module is connected to the fifth convolution layer of the third fusion module.

Images