A visual tracking method based on feature enhancement and feature fusion

Abstract

A visual tracking method based on feature enhancement and feature fusion, comprising the following steps: S1: inputting a template image and a search image, using a ResNet‑50 network to perform feature extraction respectively, saving feature maps obtained in the first four stages of the feature extraction process, and recording them as conv1, conv2, conv3 and conv4 respectively; S2: performing channel adjustment on the conv4 feature map of the template image and the conv4 feature map of the search image to obtain results Adjust A and Adjust B; S3: passing the conv2 feature map of the template image and the conv2 feature map of the search image through a feature enhancement module respectively, and adding the results after feature enhancement to Adjust A and Adjust B respectively; S4: performing a deep cross-correlation operation on the two feature maps obtained from S3 to obtain a response map, and selecting a feature point with the largest response value from the response map as a candidate response window RoW; S5: performing feature fusion on the candidate response window RoW through a refinement module, and finally outputting a mask and a position frame of the target to be tracked; and the target is tracked through the above steps.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning visual tracking, and in particular to a visual tracking method based on feature enhancement and feature fusion. Background Art

[0002] As one of the key issues in the field of computer vision, visual tracking plays a key role in many applications, such as intelligent monitoring, autonomous driving, and sports analysis. The purpose of tracking is to predict the position and size of the target in each subsequent frame of the video based on the given initial frame target box. Although existing tracking methods have achieved certain results, there are still many challenges in the face of target changes or interference from similar objects.

[0003] The patent document with publication number CN115393395A discloses a target tracking method based on a twin network. This method increases the accuracy of classification information by weighting channel entropy values, thereby improving the accuracy of target position. However, this method ignores the importance of the feature extraction backbone network, and cannot fully capture the key information of the target during feature extraction, and the tracking accuracy and robustness are limited in complex scenarios. The patent document with publication number CN115588030A discloses a visual target tracking method and device based on a twin network. This method uses a ResNet network to extract deeper semantic information, thereby enhancing the network's capabilities. However, in complex tracking scenarios, it cannot effectively deal with background interference, thereby reducing tracking accuracy and robustness.

[0004] Therefore, the present invention proposes a visual tracking method based on feature enhancement and feature fusion to overcome the technical defects of the above-mentioned prior art. Summary of the invention

[0005] The purpose of the present invention is to solve the technical problems of low tracking accuracy and robustness caused by target changes or interference from similar objects during visual tracking, and thus a visual tracking method based on feature enhancement and feature fusion is proposed.

[0006] The technical solution adopted by the present invention is:

[0007] A visual tracking method based on feature enhancement and feature fusion, comprising the following steps:

[0008] S1: Input the template image and the search image, use the ResNet-50 network to extract features respectively, and save the feature maps obtained in the first four stages of the feature extraction process, which are recorded as conv1, conv2, conv3 and conv4 respectively;

[0009] S2: Perform channel adjustment on the conv4 feature map of the template image and the conv4 feature map of the search image to obtain the results Adjust A and Adjust B;

[0010] S3: Pass the conv2 feature map of the template image and the conv2 feature map of the search image through the feature enhancement module respectively, and add the enhanced results to Adjust A and Adjust B respectively;

[0011] S4: Perform a deep cross-correlation operation on the two feature maps obtained in step S3 to obtain a response map, and select the feature point with the largest response value from the response map as a candidate response window RoW;

[0012] S5: The candidate response window RoW is subjected to feature fusion by the refinement module, and finally the mask and position box of the target to be tracked are output;

[0013] The above steps are used to track the target.

[0014] In step S3, the constructed feature enhancement module structure is:

[0015] Input feature map → non-local mean operation → convolution downsampling operation → output feature map.

[0016] The non-local mean operation construction method is:

[0017] (1) The correlation is determined by calculating the weighted sum between each position in the input feature map and all other positions. The formula is:

[0018]

[0019] Where x is the input feature map, y i is the feature value of the output i position, i, j represent the position index of a pixel at a certain point on the feature map, x i represents the vector of the same dimension as x at position i, x j represents the vector of the same dimension as x at position j, C(x) is the normalization term used to normalize the function f(x i ,x j ) sums all values, f(x i ,x j ) is the similarity calculation function, used to calculate x i and x j The similarity between two vectors, g(x j ) Use the eigenvalue calculation function g(x) to calculate x j The characteristic value of

[0020] (2)f(x i ,xj ) uses the embedded Gaussian function to calculate x i and x j The similarity between two vectors is given by:

[0021]

[0022] where θ(x) and φ(x) represent different embedding functions, θ(x i ) is used to calculate x i The eigenvalue of φ(x i ) is used to calculate x i The eigenvalue of θ(x i ) T ·φ(x i ) means that θ(x i ) is transposed and then compared with φ(x i ) to perform dot multiplication to calculate similarity;

[0023] (3) C(x) is the normalization term used to normalize the function f(x i ,x j )Sum all the values, the formula is:

[0024]

[0025] (4) g(x), θ(x) and φ(x) are eigenvalue calculation functions, which are obtained through convolution mapping in the neural network. That is, the input feature map x is multiplied by the corresponding weight W to obtain the output result. The formula is:

[0026] g(x)=W g x;

[0027] θ(x)=W θ x;

[0028] φ(x)=W φ x;

[0029] Among them, W g , W θ and W φ They represent the learnable weight matrices in the corresponding convolution.

[0030] The implementation process of non-local mean operation in neural network is:

[0031] (1) For a given input feature map X, its dimension is C×H×W (number of channels×height×width);

[0032] (2) X undergoes convolution of multiple branches, and the number of channels is changed to C', resulting in a result of C'×HW;

[0033] (3) The first and third branch matrices are transposed to obtain HW×C', and the matrix is ​​multiplied with the first and second branches to obtain a HW×HW matrix, which is then normalized by softmax.

[0034] (4) Perform matrix multiplication on the normalized result and the matrix HW×C' of the third branch to obtain the result HW×C';

[0035] (5) Transpose the matrix HW×C' to get C'×HW. Use a 1×1 convolution kernel to restore the channel to the original C and get a C×H×W feature map, which has the same shape as the input feature map.

[0036] (6) Finally, add the two together to get the output Z with dimension C×H×W.

[0037] In step S5, the specific steps of RoW performing feature fusion through the refinement module are as follows:

[0038] Search image feature map conv1→search image feature map conv2→search image feature map conv3;

[0039] Search image feature map conv1→mask feature fusion module MFFM;

[0040] Search image feature map conv1→mask feature fusion module MFFM;

[0041] Search image feature map conv1→mask feature fusion module MFFM;

[0042] RoW→deconvolution deconv→ordinary convolution→matrix addition→feature Figure 2 times upsampling → normal convolution → matrix addition → feature Figure 2 times upsampling → normal convolution → matrix addition → feature Figure 2 Upsample by times → output tracking results.

[0043] The structure of the mask feature fusion module MFFM is:

[0044] Input feature map → CBAM attention mechanism → Deformable convolution (DCN) → Deformable convolution (DCN) → Output feature map.

[0045] The CBAM attention mechanism mainly consists of two serially connected channel attention branches and spatial attention branches, and its construction method is as follows:

[0046] (1) For the input feature map, it first goes through average pooling AvgPool and maximum pooling MaxPool respectively, and the result is described as and Then, through the same multi-layer perceptron MLP and Sigmoid activation function (denoted by σ), the channel weights are learned, and the expression is:

[0047]

[0048] (2) The result obtained in (1) is further subjected to average pooling and maximum pooling, and the result is described as and Then, through n×n dilated convolution and Sigmoid activation function, the weights of each point in the space are learned, and finally a weight-optimized feature map is generated to complete all operations. The expression is:

[0049]

[0050] The deformable convolution DCN construction method is:

[0051] The input feature map is non-uniformly sampled by learning the predicted offsets through ordinary convolution conv, and then a new feature map is output. The implementation process is expressed as:

[0052]

[0053] Among them, y(p) is the feature value of the output position p, x(p) is the feature of the input feature map x at position p, K is the number of convolution kernels, and w k is the weight of the k position, p k To pre-set the offset, Δp k is the learnable offset, Δm k is the adjustment coefficient, ranging from [0,1].

[0054] Compared with the prior art, the present invention has the following technical effects:

[0055] 1) The present invention designs a feature enhancement module, which significantly enhances the feature strength of the target location, especially in the face of complex situations such as target occlusion, scale change and deformation. The module includes non-local mean operation and convolution downsampling, which effectively enhances the feature contrast by capturing global information, thereby improving the robustness of tracking and making the algorithm adaptable to target tracking tasks in different scenarios.

[0056] 2) The present invention designs a mask feature fusion module, which combines the CBAM attention mechanism and deformable convolution to better capture the target channel and spatial features, while adaptively learning the shape and contour information of the target, which helps to improve the accuracy of target segmentation and thus improve the accuracy of tracking.

[0057] 3) The method designed in the present invention was tested on the visual tracking open challenge datasets VOT2016 and VOT2018. The experimental results showed that the EAO values ​​reached 0.470 and 0.420, the Robustness reached 0.182 and 0.234, and the Accuracy reached 0.637 and 0.612, respectively, surpassing most of the existing tracking methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0059] Figure 1 It is the overall network structure diagram of the present invention;

[0060] Figure 2 This is a network structure diagram of the feature enhancement module (FEM) used in the present invention;

[0061] Figure 3 This is a structural diagram of the refinement module used in the present invention;

[0062] Figure 4 This is a network structure diagram of the mask feature fusion module (MFFM) used in the present invention;

[0063] Figure 5 This is the schematic diagram of the CBAM attention mechanism;

[0064] Figure 6 This is the schematic diagram of the deformable convolution (DCN);

[0065] Figure 7 This is a visual comparison chart of some tracking results. DETAILED DESCRIPTION

[0066] A visual tracking method based on feature enhancement and feature fusion, comprising the following steps:

[0067] S1: Input the template image and the search image, use the ResNet-50 network to extract features respectively, and save the feature maps obtained in the first four stages of the feature extraction process, which are recorded as conv1, conv2, conv3 and conv4 respectively;

[0068] S2: Perform channel adjustment on the conv4 feature map of the template image and the conv4 feature map of the search image to obtain the results Adjust A and Adjust B;

[0069] S3: Pass the conv2 feature map of the template image and the conv2 feature map of the search image through the feature enhancement module respectively, and add the enhanced results to Adjust A and Adjust B respectively;

[0070] S4: Perform a deep cross-correlation operation on the two feature maps obtained from S3 to obtain a response map, and select the feature point with the largest response value from the response map as the candidate response window RoW;

[0071] S5: The candidate response window RoW is subjected to feature fusion by the refinement module, and finally the mask and position box of the target to be tracked are output;

[0072] The above steps are used to track the target.

[0073] The overall network structure diagram of steps S1 to S5 is as follows Figure 1 shown.

[0074] In step S3, the constructed feature enhancement module is as follows Figure 2 As shown, its structure is:

[0075] Input feature map → non-local mean operation → convolution downsampling operation with kernel size of 3 and stride of 2 → output feature map.

[0076] The non-local means operation construction method used is:

[0077] (1) The correlation is determined by calculating the weighted sum between each position in the input feature map and all other positions. The formula is:

[0078]

[0079] Where x is the input feature map, y i is the feature value of the output i position, i, j represent the position index of a pixel at a certain point on the feature map, x i represents the vector of the same dimension as x at position i, x j represents the vector of the same dimension as x at position j, C(x) is the normalization term used to normalize the function f(x i ,x j ) sums all values, f(x i ,x j ) is the similarity calculation function, used to calculate x i and x j The similarity between two vectors, g(x j ) Use the eigenvalue calculation function g(x) to calculate x j The characteristic value of

[0080] (2)f(x i ,x j ) uses the embedded Gaussian function to calculate x i and x j The similarity between two vectors is given by:

[0081]

[0082] where θ(x) and φ(x) represent different embedding functions, θ(x i ) is used to calculate x i The eigenvalue of φ(x i ) is used to calculate x i The eigenvalue of θ(x i ) T ·φ(x i ) means that θ(x i ) is transposed and then compared with φ(x i ) to perform dot multiplication to calculate similarity;

[0083] (3) C(x) is the normalization term used to normalize the function f(x i ,x j )Sum all the values, the formula is:

[0084]

[0085] (4) g(x), θ(x) and φ(x) are eigenvalue calculation functions, which are obtained through convolution mapping in the neural network. That is, the input feature map x is multiplied by the corresponding weight W to obtain the output result. The formula is:

[0086] g(x)=W g x;

[0087] θ(x)=W θ x;

[0088] φ(x)=W φ x;

[0089] Among them, W g , W θ and W φ They represent the learnable weight matrices in the corresponding convolution.

[0090] The implementation process of non-local mean operation in neural network is:

[0091] (1) For a given input feature map X, its dimension is C×H×W (number of channels×height×width);

[0092] (2) X passes through three 1×1 convolution kernels respectively, and the number of channels is changed to C', and the result of C'×HW is obtained;

[0093] (3) The first and third branch matrices are transposed to obtain HW×C', and the matrix is ​​multiplied with the first and second branches to obtain a HW×HW matrix, which is then normalized by softmax.

[0094] (4) Perform matrix multiplication on the normalized result and the matrix HW×C' of the third branch to obtain the result HW×C';

[0095] (5) Transpose the matrix HW×C' to get C'×HW. Use a 1×1 convolution kernel to restore the channel to the original C and get a C×H×W feature map, which has the same shape as the input feature map.

[0096] (6) Finally, add the two together to get the output Z with dimension C×H×W.

[0097] In step S5, RoW performs feature fusion through the refinement module, and its network structure is as follows: Figure 3 As shown, the specific implementation steps are:

[0098] Search image feature map conv1→search image feature map conv2→search image feature map conv3;

[0099] Search image feature map conv1→mask feature fusion module MFFM;

[0100] Search image feature map conv1→mask feature fusion module MFFM;

[0101] Search image feature map conv1→mask feature fusion module MFFM;

[0102] RoW→deconvolution deconv→ordinary convolution→matrix addition→feature Figure 2 times upsampling → normal convolution → matrix addition → feature Figure 2 times upsampling → normal convolution → matrix addition → feature Figure 2 Upsample by times → output tracking results.

[0103] Mask feature fusion module MFFM Figure 4 As shown, its structure is:

[0104] Input feature map → CBAM attention mechanism → Deformable convolution (DCN) with a convolution kernel size of 3×3 → Deformable convolution (DCN) with a convolution kernel size of 3×3 → Output feature map.

[0105] The CBAM attention mechanism mainly consists of two serial channel attention branches and spatial attention branches. The network structure is as follows: Figure 5 As shown, the construction method is:

[0106] (1) For the input feature map, it is first subjected to average pooling (AvgPool) and maximum pooling (MaxPool), and the result is described as and Then, through the same multi-layer perceptron (MLP) and Sigmoid activation function (denoted by σ), the channel weights are learned, expressed as:

[0107]

[0108] (2) The result obtained in (1) is further subjected to average pooling and maximum pooling, and the result is described as and Then, through a 3×3 dilated convolution and Sigmoid activation function, the weights of each point in the space are learned, and finally a weight-optimized feature map is generated to complete the entire operation. The expression is:

[0109]

[0110] The network structure of Deformable Convolution (DCN) is as follows Figure 6 As shown, the construction method is:

[0111] The input feature map is non-uniformly sampled by learning the predicted offsets through ordinary convolution conv, and then a new feature map is output. The implementation process is expressed as:

[0112]

[0113] Among them, y(p) is the feature value of the output position p, x(p) is the feature of the input feature map x at position p, K is the number of convolution kernels, and w k is the weight of the k position, p k To pre-set the offset, Δp k is the learnable offset, Δm k is the adjustment coefficient, ranging from [0,1].

[0114] The tracking network designed by the present invention is trained using a comprehensive dataset consisting of COCO, YouTube-VOS, ImageNetVID and ImageNet-Det. The training process uses a pre-trained model based on SiamMask, an SGD optimizer, a momentum of 0.9, and a weight decay of 10e -5 , batch size is 8, training is performed for 20 epochs, of which the first 5 epochs are warm-up phases, and the learning rate is increased from 10 -3 Increase to 5×10 -3 , and the logarithmically decreases to 2.5×10 -3 .

[0115] In order to verify the specific effect of the tracking method designed by the present invention, the trained model was experimented on the VOT2016 and VOT2018 datasets, using accuracy, robustness and EAO (Expected Average Overlap) as evaluation indicators. Accuracy A measures the overlap between the predicted box and the true bounding box in the tracking process. The larger the value, the higher the accuracy; robustness measures the stability of the tracker. The smaller the value, the stronger the stability of the tracker; EAO combines accuracy and robustness and is the most important evaluation indicator for measuring the tracker. The larger the value, the better the performance. The test results are compared with the current mainstream and representative tracking methods. The comparison results on VOT2016 are shown in Table 1. Compared with SiamMask, the present invention (Ours) has improved by 1.5% in accuracy, 3.2% in robustness, and 3.7% in EAO. Compared with SiamRPN++, the present invention has improved by 1.8% in robustness and 0.6% in EAO. In general, the present invention has the highest EAO and the best robustness and maintains a good accuracy, which shows that the present invention not only accurately tracks the position of the target but also maintains good stability.

[0116] Figure 7 The tracking visualization comparison results of the present invention and other representative tracking methods on the VOT2016 dataset are shown. When the target moves quickly, SiamFC may lose or mistrack the target. When the target is occluded or moves quickly, the tracking frame of SiamMask sometimes cannot accurately lock the target. The tracking method proposed in the present invention can still correctly track the target when dealing with target deformation, flipping and occlusion, and maintains a good fit with the target, which is very close to the true value frame.

[0117] Table 1 Experimental results of different tracking methods on VOT2016

[0118]

[0119]

[0120] The comparison results on VOT2018 are shown in Table 2. In terms of accuracy, the present invention achieved the highest score, which is 0.3% higher than SiamMask and 1.2% higher than SiamRPN++. It achieved the best score of 0.420 on EAO, the most important evaluation index of VOT, which is 0.5% higher than SiamRPN++ and 4% higher than SiamMask. It is also better than other methods, which shows that the overall performance of the method proposed in the present invention is good in the tracking process.

[0121] Table 2 Experimental results of different tracking methods on VOT2018

[0122]

[0123] In order to verify the effects of the feature enhancement module and the mask feature fusion module designed in the present invention, an ablation experiment was carried out based on SiamMask as the baseline. The ablation experiment results on VOT2016 are shown in Table 3, and the ablation experiment results on VOT2018 are shown in Table 4.

[0124] Table 3 Ablation experiment results on VOT2016

[0125]

[0126] Table 4 Ablation experiment results on VOT2016

[0127]

[0128] After adding FEM to the model, EAO increased by 1.4% and 1.5% on the VOT2016 and VOT2018 datasets, respectively. After adding FEM and MFFM to the original model, the best results were achieved, with EAO increased by 3.7% and 4.0%, accuracy increased by 1.5% and 0.3%, and robustness increased by 3.2% and 4.2% on the three datasets, respectively, indicating that the added modules are effective and compatible, and can significantly improve the overall tracking accuracy and robustness of the model. It is also noted that after adding the two modules, the overall test speed of the model has dropped slightly, but it can still maintain an average speed of more than 90 frames per second, indicating that the added modules do not bring particularly huge computational overhead.

[0129] In summary, the present invention proposes a visual tracking method based on feature enhancement and feature fusion, and conducts comparative experiments and ablation experiments on multiple tracking data sets. The experimental results show that the tracking method proposed in the present invention has achieved leading performance in the three indicators of accuracy, robustness and EAO, which verifies the effectiveness of the tracking method proposed in the present invention.

Claims

1. A visual tracking method based on feature enhancement and feature fusion, characterized in that: It includes the following steps: S1: Input the template image and the search image, use the ResNet-50 network to extract features respectively, save the feature maps obtained in the first four stages of the feature extraction process, and record them as the conv1 feature map of the template image, the conv2 feature map of the template image, the conv3 feature map of the template image, the conv4 feature map of the template image, and the search image feature map conv1(1), the search image feature map conv2(2), the search image feature map conv3(3), and the search image feature map conv4(4); S2: Perform channel adjustment on the conv4 feature map of the template image and the feature map conv4(4) of the search image to obtain the results Adjust A and Adjust B; S3: Pass the conv2 feature map of the template image and the search image feature map conv2(2) through the feature enhancement module respectively, and add the enhanced results to Adjust A and Adjust B respectively; S4: Perform a deep cross-correlation operation on the two feature maps obtained in step S3 to obtain a response map, and select the feature point with the largest response value from the response map as a candidate response window RoW; S5: The candidate response window RoW is subjected to feature fusion by the refinement module, and finally the mask and position box of the target to be tracked are output; The above steps are used to track the target.

2. The method according to claim 1, characterized in that In step S3, the constructed feature enhancement module structure is: Input feature map → non-local mean operation → convolution downsampling operation → output feature map.

3. The method according to claim 2, characterized in that The non-local means operation construction method used is: (1) The correlation is determined by calculating the weighted sum between each position in the input feature map and all other positions. The formula is: Where x is the input feature map, y i is the feature value of the output i position, i, j represent the position index of a pixel at a certain point on the feature map, x i represents the vector of the same dimension as x at position i, x j represents the vector of the same dimension as x at position j, C(x) is the normalization term used to normalize the function f(x i ,x j )Sum all values, f(x i ,x j ) is the similarity calculation function, used to calculate x i and x j The similarity between two vectors, g(x j ) Use the eigenvalue calculation function g(x) to calculate x j The characteristic value of (2)f(x i ,x j ) uses the embedded Gaussian function to calculate x i and x j The similarity between two vectors is given by: where θ(x) and φ(x) represent different embedding functions, θ(x i ) is used to calculate x i The eigenvalue of φ(x i ) is used to calculate x i The eigenvalue of θ(x i ) T ·φ(x i ) means that θ(x i ) is transposed and then compared with φ(x i ) to perform dot multiplication to calculate similarity; (3) C(x) is the normalization term used to normalize the function f(x i ,x j )Sum all the values, the formula is: (4) g(x), θ(x) and φ(x) are eigenvalue calculation functions, which are obtained through convolution mapping in the neural network. That is, the input feature map x is multiplied by the corresponding weight W to obtain the output result. The formula is: g(x)=W g ·x; θ(x)=W θ ·x; φ(x)=W φ ·x; Among them, W g , W θ and W φ They represent the learnable weight matrices in the corresponding convolution.

4. The method according to claim 3, characterized in that The implementation process of non-local mean operation in neural network is: (1) For a given input feature map X, its dimension is C×H×W (number of channels×height×width); (2) X undergoes convolution of multiple branches, and the number of channels is changed to C', resulting in a result of C'×HW; (3) The first and third branch matrices are transposed to obtain HW×C', and the matrix is ​​multiplied with the first and second branches to obtain a HW×HW matrix, which is then normalized by softmax. (4) Perform matrix multiplication on the normalized result and the matrix HW×C' of the third branch to obtain the result HW×C'; (5) Transpose the matrix HW×C' to get C'×HW. Use a 1×1 convolution kernel to restore the channel to the original C and get a C×H×W feature map, which has the same shape as the input feature map. (6) Finally, add the two together to get the output Z with dimension C×H×W.

5. The method according to claim 1, characterized in that In step S5, the specific steps of RoW performing feature fusion through the refinement module are as follows: Search image feature map conv1(1) → search image feature map conv2(2) → search image feature map conv3(3); Search image feature map conv1(1) → first mask feature fusion module MFFM(4); Search image feature map conv2(2) → second mask feature fusion module MFFM(5); Search image feature map conv3(3) → third mask feature fusion module MFFM(6); RoW(7)→deconvolution(8)→ordinary convolution(9)→matrix addition(10)→feature map double upsampling(11)→ordinary convolution(12)→matrix addition(13)→feature map double upsampling(14)→ordinary convolution(15)→matrix addition(16)→feature map double upsampling(17)→output tracking result(18).

6. The method according to claim 5, characterized in that The structure of the mask feature fusion module MFFM is: Input feature map → CBAM attention mechanism → Deformable convolution (DCN) → Deformable convolution (DCN) → Output feature map.

7. The method according to claim 6, characterized in that The CBAM attention mechanism used mainly includes two serially connected channel attention branches and spatial attention branches, and its construction method is as follows: (1) For the input feature map, it first goes through average pooling AvgPool and maximum pooling MaxPool respectively, and the result is described as and Then, through the same multi-layer perceptron MLP and Sigmoid activation function (denoted by σ), the channel weights are learned, and the expression is: (2) The result obtained in (1) is further subjected to average pooling and maximum pooling, and the result is described as and Then, through n×n dilated convolution and Sigmoid activation function, the weights of each point in the space are learned, and finally a weight-optimized feature map is generated to complete all operations. The expression is:

8. The method according to claim 6, characterized in that The deformable convolution DCN construction method used is: The input feature map is non-uniformly sampled by learning the predicted offsets through ordinary convolution conv, and then a new feature map is output. The implementation process is expressed as: Among them, y(p) is the feature value of the output position p, x(p) is the feature of the input feature map x at position p, K is the number of convolution kernels, and w k is the weight of the k position, p k To pre-set the offset, Δp k is the learnable offset, Δm k is the adjustment coefficient, ranging from [0,1].

Images