Lightweight high-resolution human pose estimation method based on channel spatial weighting

Abstract

The application provides a lightweight high-resolution human pose estimation method based on channel space weighting, comprising the following steps: step one: constructing a Lite CSW-HRNet network structure, calculating weights by using single resolution and channel space features of all resolutions, and simultaneously achieving the effect of exchanging information; step two: single resolution weight calculation, connecting two weight calculation modules in parallel, and then multiplying the weight map by the input feature map for adaptive feature refinement. Compared with other lightweight pose estimation networks, the application has higher pose estimation accuracy and a lighter model. In the core single resolution and cross resolution weight calculation process, the weight map is calculated independently and in parallel along the channel and space, and maximum pooling and average pooling are used to fully retain the original features. In the space weight calculation, convolution is used to increase the receptive field to aggregate wider spatial context information.

Description

TECHNICAL FIELD

[0001] The present application relates to a human pose estimation method, in particular to a lightweight high-resolution human pose estimation method based on channel space weighting, and belongs to the field of computer science and technology. BACKGROUND

[0002] Human pose estimation aims to obtain human skeleton key point coordinates from image input, and is one of the most challenging and research directions in the field of computer vision. In recent years, human pose estimation technology has developed rapidly and is widely used in action recognition, human-computer interaction and medical rehabilitation fields. The current human pose estimation algorithm is mostly based on neural networks such as ResNet, Hourglass, HRNet and generative adversarial network. ResNet introduces a residual module, uses a residual structure with a skip connection, and alleviates the network degradation problem caused by increasing depth. The Hourglass network can better extract multi-scale feature information of the target by cascading multiple hourglass modules. HRNet maintains high-resolution representation throughout the process by connecting high-resolution to low-resolution subnetworks in parallel and repeatedly performing multi-scale fusion. The generative adversarial network combines human skeleton prior information, improves the correctness of skeleton key point prediction through adversarial training, and further improves the estimation accuracy of each part of the body position.

[0003] Although most of the current human pose estimation methods can achieve high detection accuracy, they are often implemented through complex network structures, which leads to a large number of model parameters and large amount of calculation, and requires high cost for training and mobile terminal deployment. Although Lite-HRNet has achieved a good balance between accuracy and complexity, there is still a problem of insufficient weight coefficient learning, which is caused by the following reasons: the importance of human skeleton key points in spatial position is not considered; average pooling greatly suppresses the diversity of features, resulting in a large amount of feature loss; after pooling, the fully connected layer reduces and increases the dimension of the channel, which has an adverse effect on learning the dependency relationship between channels. Therefore, a lightweight high-resolution human pose estimation method based on channel space weighting is proposed. SUMMARY

[0004] Therefore, the present application provides a lightweight high-resolution human pose estimation method based on channel space weighting to solve or alleviate the technical problems existing in the prior art, and at least provides a beneficial choice.

[0005] The technical scheme of the embodiment of the present application is implemented as follows: the lightweight high-resolution human pose estimation method based on channel space weighting comprises the following steps:

[0006] Step one: construct Lite CSW-HRNet network structure, calculate weights by using single resolution and channel space features of all resolutions, and achieve the effect of exchanging information at the same time;

[0007] Step two: single resolution weight calculation, connect two weight calculation modules in parallel, then multiply the weight map by the input feature map to perform adaptive feature refinement;

[0008] Step three: channel weight calculation, use the relationship between the channels of the features to generate a channel weight map;

[0009] Step four: spatial weight calculation, first perform mean pooling and maximum pooling operations along the channel axis of the input feature map, and then concatenate the generated descriptors along the channel axis to generate an effective spatial feature descriptor;

[0010] Step five: cross-resolution weight calculation, use adaptive average pooling to convert each parallel resolution to the size of the smallest resolution, and subsequent calculations are performed on the small resolution, so the calculation complexity is very small.

[0011] Further preferably, in step one, the network structure of Lite CSW-HRNet still follows Lite-HRNet, first stage1 takes 1 / 4 of the input image resolution as the input of the network, and then the formation process of stage is to add the lowest resolution of the previous stage to the current stage as a new branch in parallel, and fuse each resolution with each other.

[0012] Further preferably, in step one, in Lite CSW-HRNet, the original conditional channel weighting is called channel space weighting; the weight calculation formula of the s-th resolution branch is:

[0013]

[0014] where W s is a three-dimensional tensor with size C s ×W s ×H s , denotes element-wise multiplication;

[0015] By using single resolution and channel space features of all resolutions to calculate weights, and at the same time achieve the effect of exchanging information.

[0016] Further preferably, in step two, given a feature map as input, a 1D channel weight map and a 2D spatial weight map are calculated according to the calculation formula respectively.

[0017] F c '=M c (F)

[0018] F s '=M s (F)

[0019] The two weight calculation modules are connected in parallel, and then the weight map is multiplied by the input feature map to perform adaptive feature refinement, and F” is the final refined output:

[0020]

[0021] wherein represents element-wise multiplication; during multiplication, the weight values are broadcast accordingly: the channel weight values are broadcast along the spatial dimension, and vice versa.

[0022] Further preferably, in step two, the calculation process is divided into two parallel sub-processes: channel weight calculation and spatial weight calculation, wherein F is the input feature map, F c ' is the generated channel weight, F s ' is the generated spatial weight, and F” is the final refined output.

[0023] Further preferably, in step three, a one-dimensional convolution is introduced to replace the fully connected layer to implement the local cross-channel interaction strategy, and an adaptive channel dimension function is applied to determine the size of the one-dimensional convolution kernel, so as to more efficiently train the network, improve the accuracy, and reduce the parameter amount. In order to capture the local cross-channel interaction, only the interaction between each channel and its k adjacent channels is considered. Taking the average pooling branch as an example, the weight ω i ,

[0024]

[0025] wherein represents k adjacent channels of F , represents the connection weight between the k adjacent channels and the next layer, at this time, the parameters in the average pooling branch have a total of kxC, and C represents the number of channels of the input feature map. In order to further reduce the complexity of the network and improve the efficiency, the parameters between all connection channels are shared, and ω i is generated by the calculation formula:

[0026]

[0027] compared with the first ω i , becomes wj After the channel connection parameter is shared, the second ω i It is easily implemented by a fast one-dimensional convolution with a kernel size of k, and the parameter amount is only k.

[0028] w = σ(C1D k (y))

[0029] In the above formula, C1D represents one-dimensional convolution, k represents the number of convolution kernels, and k is related to the channel dimension C. There may be a certain mapping relationship between k and C k is in a nonlinear proportion with C, so it is a feasible choice to assume the mapping relationship between k and C by an exponential function, that is,

[0030]

[0031] where y and b are two hyperparameters, generally set to 1 and 2, and since the channel dimension C is often set to an integer power of 2, the previous formula is replaced by

[0032]

[0033] where |t| odd represents the nearest odd number to t. Therefore, given the channel dimension C, the convolution kernel size k can be adaptively determined in the above manner.

[0034] Further preferably, in step three, the channel weight map is generated by utilizing the inter-channel relationship of the features. First, the average pooling and maximum pooling operations are used to aggregate the spatial information of the feature maps to generate two different channel descriptors and represent the average pooled features and the maximum pooled features, respectively. Then, a one-dimensional convolution with a kernel length of k is used to aggregate the information of the k channels in the channel neighborhood. The two features after convolution are added element by element, and the channel weight map is generated by Sigmoid function operation Subsequently, the generated channel weight is broadcasted along the two dimensions in space to and multiplied by the corresponding elements of the input feature map to obtain the feature map after injecting the channel weight. The channel weight calculation process can be represented as follows:

[0035]

[0036] where σ represents the Sigmoid function, represents the one-dimensional convolution operation with a kernel size of k.

[0037] Further preferably, in step four, the spatial weight calculation is performed as a supplement to the channel weight calculation, first mean-pooling and max-pooling operations are performed along the channel axis of the input feature map to generate two different spatial context descriptors and The generated descriptors are then concatenated along the channel axis to generate an effective spatial feature descriptor.

[0038] A 7x7 convolution is used to encode and map the region information that needs to be emphasized or suppressed in the space, and the spatial context information is more efficiently aggregated. The features after the convolution are operated by a Sigmoid function to generate a spatial weight map Finally, the generated spatial weight is broadcasted along the channel dimension to And then multiplied by the input feature map to obtain the feature map after injecting the weight.

[0039] Further preferably, in step four, the spatial weight calculation process can be represented as follows:

[0040]

[0041] where f 7×7 represents a 7x7 regular convolution,

[0042] The single-resolution weight is calculated using the following formula:

[0043] w s = S(X s )

[0044] where X s is the input single-resolution feature map, and the function S(·) performs the following operations to generate different weights in the channel and space, respectively.

[0045]

[0046] M c is the channel weight calculation process, and M s is the spatial weight calculation process.

[0047] Further preferably, in step five, in the cross-resolution weight calculation process, the s-th stage has s weight mappings W1, W2, … W s Each weight mapping corresponds to a corresponding resolution X1, X2, … X s , where X1 is the highest resolution, X s is the s-th highest resolution. First, use adaptive average pooling (AAP) to convert each parallel resolution {X1, X2, … X s-1} to the size of the smallest resolution X s :

[0048]

[0049] Then {X1', X'2, … X s '1} and X s are spliced together, and subsequent calculations are implemented at a small resolution, so the calculation complexity is very small. Subsequent operations directly use a single-resolution weighting method:

[0050]

[0051] A weight map W1', W2', … W s ' containing s branches is generated, and the weight at each position of each resolution depends on the channel and spatial features at the same position from the pooled multi-resolution feature map.

[0052] The embodiment of the present application has the following advantages due to the adoption of the above technical solutions: the present application proposes a lightweight high-resolution human pose estimation method based on channel and spatial weighting, and compared with other lightweight pose estimation networks, the pose estimation accuracy is higher and the model is lighter. In the core single-resolution and cross-resolution weight calculation process, the weight map is calculated independently and in parallel along the channel and the space, and the maximum pooling and average pooling are used to fully retain the original features. In the channel weight calculation, adaptive one-dimensional convolution is introduced to aggregate the information between adjacent channels, avoiding the adverse effects of channel dimension reduction. In the spatial weight calculation, convolution is used to increase the receptive field to aggregate wider spatial context information.

[0053] The above summary is merely intended to illustrate the present application and is not intended to limit in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present application will be apparent from a review of the drawings and the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0055] Figure 1 The step flowchart of the present application is shown in Figure 1.

[0056] Figure 2 The channel and spatial weighting module diagram of the present application is shown in Figure 2.

[0057] Figure 3 The single-resolution weighting process diagram of the present application is shown in Figure 3.

[0058] Figure 4 Channel weight calculation module diagram of the present application;

[0059] Figure 5 Spatial weight calculation module diagram of the present application;

[0060] Figure 6 Single resolution weight calculation process diagram of the present application;

[0061] Figure 7 Structure table of Lite CSW-HRNet of the present application. DETAILED DESCRIPTION

[0062] In the following, only certain exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present application. Therefore, the drawings and the description are considered to be exemplary in nature rather than limiting.

[0063] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0064] As Figures 1-7 shown, the embodiment of the present application provides a channel and spatial weight based lightweight high resolution human pose estimation method, comprising the following steps:

[0065] Step one: construct Lite CSW-HRNet network structure, calculate weights by using single resolution and channel and spatial features of all resolutions, and at the same time achieve the effect of exchanging information;

[0066] Step two: single resolution weight calculation, connect two weight calculation modules in parallel, then multiply the weight map by the input feature map to perform adaptive feature refinement;

[0067] Step three: channel weight calculation, use the inter-channel relationship of features to generate a channel weight map;

[0068] Step four: spatial weight calculation, first perform mean pooling and maximum pooling operations along the channel axis of the input feature map, splice the generated descriptors along the channel axis to generate an effective spatial feature descriptor;

[0069] Step five: cross resolution weight calculation, use adaptive average pooling to convert each parallel resolution to the size of the smallest resolution, and subsequent calculations are implemented on the small resolution, so the calculation complexity is very small.

[0070] In one embodiment, in step one, the network structure of Lite CSW-HRNet still follows Lite-HRNet, first stage1 takes 1 / 4 of the input image resolution as the input of the network, and then the formation process of stage is to add 1 / 2 of the lowest resolution of the previous stage as a new branch in parallel to the current stage, and fuse each resolution with each other, in Lite CSW-HRNet, the original conditional channel weighting is called channel spatial weighting; the weight calculation formula of the s-th resolution branch is:

[0071]

[0072] Where W s is a three-dimensional tensor with size C s ×W s ×H s , denotes element-wise multiplication;

[0073] The weight is calculated by using the channel spatial features of a single resolution and all resolutions, and at the same time, the function of exchanging information is achieved.

[0074] In one embodiment, in step two, given a feature map as input, a 1D channel weight map and a 2D spatial weight map

[0075] F c '=M c (F)

[0076] F s '=M s (F)

[0077] The two weight calculation modules are connected in parallel, then the weight map is multiplied by the input feature map to perform adaptive feature refinement, and F” is the final refined output:

[0078]

[0079] Where denotes element-wise multiplication; in the multiplication process, the weight value is broadcasted accordingly: the channel weight value is broadcasted along the spatial dimension, and vice versa, the calculation process is divided into two parallel sub-processes: channel weight calculation and spatial weight calculation, where F is the input feature map, F c ' is the generated channel weight, and F sFor the generated spatial weight, F" is the final refined output, the human skeleton key point is smaller compared to the global image target, and the parallel mode can better extract the shallow features of the human skeleton key point compared to the series mode of channel weight calculation and spatial weight calculation, which can learn the input feature map respectively, and the latter weight calculation module will not be disturbed by the former weight calculation module.

[0080] In one embodiment, in step three, the local cross-channel interaction strategy is realized by introducing one-dimensional convolution instead of full connection layer, and the size of one-dimensional convolution kernel is determined by applying adaptive channel dimension function, so as to train the network more efficiently, improve the accuracy, and reduce the parameter amount. In order to capture the local cross-channel interaction, only the interaction between each channel and its k adjacent channels is considered. Taking the average pooling branch as an example, the weight ω applied to the channel is generated by the calculation formula i ,

[0081]

[0082] wherein represents k adjacent channels of , represents the value of the feature map after pooling, represents the connection weight between the k adjacent channels and the next layer, at this time, the parameters in the average pooling branch have kxC, and C represents the channel number of the input feature map. In order to further reduce the complexity of the network and improve the efficiency, the parameters between all connected channels are shared, and ω is generated by the calculation formula i :

[0083]

[0084] Compared with the first ω i , becomes w j , after sharing the channel connection parameters, the second ω i is easily realized by fast one-dimensional convolution with kernel size k, at this time, the parameter amount is only k.

[0085] w=σ(C1D k (y))

[0086] In the above formula, C1D represents one-dimensional convolution, k represents the number of convolution kernels, and k and the channel dimension C are related, and there may be a certain mapping relationship between k and C k and C are in a nonlinear proportion, so it is a feasible choice to assume the mapping relationship between k and C by exponential function, that is

[0087]

[0088] where y, b are two hyper-parameters, generally set as 1 and 2, and since the channel dimension C is usually set as an integer power of 2, the above formula is replaced by

[0089]

[0090] where |t| odd represents the nearest odd number to t. Therefore, given the channel dimension C, the convolution kernel size k can be adaptively determined in the above manner, and the channel weight map is generated by utilizing the inter-channel relationship of features. First, the average pooling and maximum pooling operations are used to aggregate the spatial information of the feature map to generate two different channel descriptors and represent the average pooled features and the maximum pooled features, respectively. Then, a one-dimensional convolution with a kernel length of k is used to aggregate the information of k channels within the channel neighborhood. The two features after convolution are element-wise added and operated by the Sigmoid function to generate the channel weight map Subsequently, the generated channel weight is broadcasted along the two dimensions in space to and multiplied by the input feature map to obtain the feature map after injecting the channel weight. The channel weight calculation process can be represented as follows:

[0091]

[0092] where σ represents the Sigmoid function, represents a one-dimensional convolution operation with a kernel size of k.

[0093] In one embodiment, in step four, the spatial weight calculation is complementary to the channel weight calculation. First, the mean pooling and maximum pooling operations are performed along the channel axis of the input feature map to generate two different spatial context descriptors and and the descriptors generated respectively are spliced along the channel axis to generate an effective spatial feature descriptor;

[0094] A 7x7 convolution is used to encode and map the information of the regions that need to be emphasized or suppressed in space, and the spatial context information is more efficiently aggregated. The features after convolution are operated by the Sigmoid function to generate the spatial weight map Finally, the generated spatial weight is broadcasted along the channel dimension to and multiplied by the input feature map to obtain the feature map after injecting the weight. The spatial weight calculation process can be represented as follows:

[0095]

[0096] where f 7×7This represents a regular 7×7 convolution.

[0097] The single-resolution weights are calculated using the following formula:

[0098] w s =S(X) s )

[0099] Among them, X s Given a single-resolution feature map as input, the function S(·) performs the following operations to generate different weights in the channel and spatial dimensions, respectively.

[0100]

[0101] M c For the channel weight calculation process, M s This describes the spatial weight calculation process.

[0102] In one embodiment, in step five, during the cross-resolution weight calculation process, the s-th stage has s weight mappings W1, W2, ... W s Each weight mapping corresponds to a resolution X1, X2, ... X s Where X1 is the highest resolution, X s For the s-th high resolution, adaptive average pooling (AAP) is first used to pool the parallel resolutions {X1, X2, ... X...}. s-1 Convert to minimum resolution X s Size:

[0103]

[0104] Then {X1',X'2,…X s ' -1} and X s Since the data is concatenated and subsequent calculations are performed at a low resolution, its computational complexity is very low. Subsequent operations can directly use a single-resolution weighted method.

[0105]

[0106] Generate a weight map W1', W2', ... W with s branches. s The weights at each location for each resolution depend on the channel and spatial features at the same location from the pooled multi-resolution feature map.

[0107] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A lightweight high-resolution human pose estimation method based on channel spatial weighting, characterized in that, The method comprises the following steps: Step one: construct the Lite CSW-HRNet network structure, calculate the weight by using the channel space features of a single resolution and all resolutions, and achieve the effect of exchanging information at the same time; Step two: single resolution weight calculation, connect the two weight calculation modules in parallel, then multiply the weight map by the input feature map to perform adaptive feature refinement; Step three: channel weight calculation, use the inter-channel relationship of the features to generate a channel weight map; Step four: spatial weight calculation, first perform mean pooling and maximum pooling operations along the channel axis of the input feature map, concatenate the generated descriptors along the channel axis to generate an effective spatial feature descriptor; Step five: cross-resolution weight calculation, use adaptive average pooling to convert each parallel resolution to the size of the smallest resolution, and subsequent calculations are implemented on the small resolution; The channel weight and the spatial weight are calculated by channel and space independent parallel weight, using average pooling and maximum pooling, and introducing a one-dimensional convolution in the channel weight to replace the fully connected layer to calculate the weight coefficient.

2. The lightweight high-resolution human pose estimation method based on channel spatial weighting according to claim 1, characterized in that: In step one, the network structure of Lite CSW-HRNet still follows Lite-HRNet, first stage1 takes 1 / 4 of the input image resolution as the input of the network, and then the formation process of stage is to take 1 / 2 of the lowest resolution of the previous stage as a new branch and add it to the current stage in parallel, and fuse each resolution with each other.

3. The lightweight high-resolution human pose estimation method based on channel spatial weighting according to claim 1, characterized in that: In step one, in Lite CSW-HRNet, the original conditional channel weighting is called channel space weighting; the weight calculation formula of the s-th resolution branch is: ； wherein is a three-dimensional tensor of size , denotes the main element-wise multiplication; The weight is calculated by using the channel space features of a single resolution and all resolutions, and the effect of exchanging information is achieved at the same time.

4. The lightweight high-resolution human pose estimation method based on channel spatial weighting of claim 1, wherein: In step two, given a feature map As input, a 1D channel weight map and a 2D spatial weight map are computed respectively by the following formulas ； The two weight calculation modules are connected in parallel, and then the weight map is multiplied by the input feature map to perform adaptive feature refinement, is the final refined output: ； wherein denotes element-wise multiplication; during multiplication, the weight values are broadcast accordingly: channel weight values are broadcast along the spatial dimensions and vice versa.

5. The lightweight high-resolution human pose estimation method based on channel spatial weighting according to claim 4, characterized in that: In step two, the calculation formula is calculated respectively into two parallel sub-processes: channel weight calculation and spatial weight calculation, wherein is the input feature map, is the generated channel weight, is the generated spatial weight, is the final refined output.

6. The lightweight high-resolution human pose estimation method based on channel spatial weighting of claim 1, wherein: In step three, the local cross-channel interaction strategy is realized by introducing a one-dimensional convolution instead of the full connection layer, and an adaptive channel dimension function is applied to determine the size of the one-dimensional convolution kernel. In order to capture the local cross-channel interaction, only the interaction between each channel and its k adjacent channels is considered. Taking the average pooling branch as an example, the weights applied to the channels are generated by the calculation formula , ； wherein represents k adjacent channels, represents the value of the feature map after pooling, represents the connection weight of k adjacent channels and the next layer, at this time, the parameters in the average pooling branch share kxC, C represents the channel number of the input feature map; the parameters between all connection channels are shared, and the connection weight is generated by the calculation formula : ； Compared with the first , becomes , after the channel connection parameter sharing, the second is implemented by a fast one-dimensional convolution with a kernel size of k, at which time the parameter quantity is only k ； In the above formula, C1D represents one-dimensional convolution, k represents the number of convolution kernels, and k is related to the channel dimension C, and there is a certain mapping relationship between k and C , k is in a nonlinear proportion with C, so the mapping relationship between k and C is assumed by an exponential function, that is, ； Where y and b are two hyperparameters, generally set to 1 and 2, and since the channel dimension C is usually set to an integer power of 2, the above formula is replaced by ； wherein represents the odd number closest to ; therefore, in the case of a given channel dimension C, the convolution kernel size k can be adaptively determined in the above manner.

7. The lightweight high-resolution human pose estimation method based on channel spatial weighting according to claim 6, characterized in that: In step three, the channel weight map is generated by utilizing the inter-channel relationship of features. Firstly, average pooling and max pooling operations are used to aggregate the spatial information of feature maps, generating two different channel descriptors and , representing the average-pooled features and the max-pooled features, respectively; then a one-dimensional convolution with a kernel length of is used to aggregate the information of channels within the channel neighborhood; the two convolved features are element-wise added and operated by a Sigmoid function to generate the channel weight map ; subsequently, the generated channel weights are broadcasted along the two dimensions in space to , and then multiplied by the input feature map to obtain the feature map with injected channel weights; the channel weight calculation process can be represented as follows: ； wherein, denotes a Sigmoid function, denotes a one-dimensional convolution operation with a kernel size of k.

8. The lightweight high-resolution human pose estimation method based on channel spatial weighting of claim 1, wherein: In step four, the spatial weight calculation is complementary to the channel weight calculation. First, mean pooling and max pooling operations are performed along the channel axis of the input feature map to generate two different spatial context descriptors and , and the generated descriptors are spliced along the channel axis to generate an effective spatial feature descriptor; Use Convolution is used to encode and map the region information that needs to be emphasized or suppressed in space, more efficiently aggregate the spatial context information, and generate a spatial weight map through Sigmoid function operation on the convolutional features Finally, the generated spatial weight is broadcasted and expanded to , and then multiplied by the corresponding elements of the input feature map to obtain the feature map after injecting the weight.

9. The lightweight high-resolution human pose estimation method based on channel spatial weighting according to claim 8, characterized in that: In step four, the spatial weight calculation process can be represented as follows: ； wherein represents one regular convolution, The single resolution weight is calculated using the following formula: ； wherein, is the input single resolution feature map, the function performs the following operations to generate different weights respectively on channel and space; ； For the channel weight computation process, For the spatial weight computation process.

10. The lightweight high-resolution human pose estimation method based on channel spatial weighting of claim 1, wherein: In step five, in the cross-resolution weight calculation process, the s-th stage has s weight mappings Each weight mapping corresponds to a corresponding resolution Wherein is the highest resolution, is the s-th high resolution; first use adaptive average pooling (AAP) to convert each parallel resolution to the size of the minimum resolution : ； Then the with are spliced together, and subsequent calculations are implemented at a small resolution, and subsequent operations directly use a single-resolution weighting method: ； generating a weight map comprising a number of branches the weight at each location of each resolution depends on the channel and spatial features from the same location in the pooled multi-resolution feature map.

Images