Anchors-free object detection method based on multi-scale and double-center fusion

Abstract

The application discloses a kind of based on multi-scale and double center fusion's anchor frameless target detection method, comprising the following steps: constructing backbone network extraction image feature;Different scale feature is fused by neck network;Information enhancement is carried out to multi-scale feature map by multi-scale fusion module;Class, boundary box position is predicted by prediction network, and overall center degree is obtained by double center fusion module for the quality evaluation of boundary box, and finally the detection result is obtained.Compared with the traditional anchor frameless target detection method, the multi-scale fusion module can balance the feature enhancement between multi-scale feature maps, and the double center fusion module can filter low-quality boundary boxes more effectively by fusing the information of boundary box center degree and class center degree.Through the optimization of multi-scale fusion module and double center fusion module, the expression ability of the target detection method is improved, and the detection accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to an anchor-free target detection method based on multi-scale and double-center fusion. BACKGROUND

[0002] The research on target detection technology has always been one of the most challenging research topics in the field of computer vision. The purpose is to find all the targets in the image and determine the category and position of the target. The target detection technology is divided into anchor-free target detection method and anchor-free target detection method according to whether a large number of anchor frames must be preset to generate a prediction frame. However, in the anchor frame target detection method, the size, proportion and number of anchor frames need to be designed very carefully, and the number of anchor frames is very large, which affects the generalization ability of the method and brings complicated calculation. Therefore, the image recognition using anchor-free target detection method is studied.

[0003] FCOS is a kind of anchor-free pixel-by-pixel full convolution target detection method, but we found at least the following problems in the process of studying the present application.

[0004] In FCOS, the center degree is the main index for evaluating the quality of the boundary frame. The information of the center degree is stored in the center branch. The center degree has two placement methods in FCOS, one is added in parallel with the classification branch, called category center degree, and the other is added in parallel with the regression branch, called boundary frame center degree. However, in fact, the learning of classification task and regression task actually has certain contradiction, the classification task pays more attention to the place where the semantic information is rich, while the features around the boundary may be good at boundary frame regression. Moreover, there are many target centers in the picture which are not in the center position of the boundary frame. If only the category center degree is considered, the correlation between the center degree and the position relationship is ignored. If only the boundary frame center degree is considered, the semantic information is ignored, and for the target prediction which is not in the center of the target frame, the center degree from the center branch has little inhibitory effect on the low-quality boundary frame in theory.

[0005] FCOS uses a feature pyramid network (FPN) in the neck network for the prediction of targets of different scales. However, FPN has two limitations: (1) information attenuation in the fusion process; and (2) aliasing effect in cross-scale fusion. In view of the above two limitations, there are many networks to optimize FPN at present, but these optimizations only emphasize the fusion of adjacent feature maps, but ignore the balance of non-adjacent feature map fusion. However, the balanced feature pyramid (BFP) emphasizes that the non-adjacent feature layers can balance the fusion of feature maps of each layer, but the BFP uses a self-attention mechanism to refine the feature maps, which brings a very large parameter quantity and high spatial complexity. SUMMARY

[0006] The purpose of the present application is to solve the above-mentioned defects in the prior art, and to provide an anchor-free target detection method based on multi-scale and double-center fusion.

[0007] The purpose of the present application can be achieved by adopting the following technical solutions:

[0008] An anchor-free target detection method based on multi-scale and double-center fusion, the detection method comprising the following steps:

[0009] S1, constructing a backbone network, inputting an image F to be detected into the backbone network for feature extraction, the purpose being to obtain feature maps of different scales, wherein C represents the number of channels of the feature map, W and H are the height and width of the feature map, and R represents the real number domain; the output is feature maps F in ∈R C×W×H The input is input into the backbone network for feature extraction, the purpose being to obtain feature maps of different scales, wherein C represents the number of channels of the feature map, W and H are the height and width of the feature map, and R represents the real number domain; the output is feature maps F P1 ∈ The feature maps The feature maps Wherein C1, C2, C3 are the number of channels of the feature map after dimension reduction by the backbone network, W1, W2, W3 are the width of the feature map after dimension reduction by the backbone network, H1, H2, H3 are the height of the feature map after dimension reduction by the backbone network; the feature maps of different scales obtained from the backbone network The feature maps The feature maps For the subsequent fusion of multi-scale feature maps;

[0010] S2, inputting the feature maps F obtained in step S1 into a neck network for different scale feature fusion, the purpose being to fuse the feature maps F The feature maps The feature maps into the neck network for different scale feature fusion, the purpose being to fuse the feature maps F The feature maps The feature maps semantic information and deep feature information, and the outputs are feature maps feature maps feature maps feature maps feature maps for subsequent large target detection, wherein C0 is the channel number of the feature map after dimension reduction, W4 and W5 are the width of the feature map after dimension reduction, and H4 and H5 are the height of the feature map after dimension reduction;

[0011] S3, taking the feature map P1, the feature map P2, the feature map P3, the feature map P4 and the feature map P5 obtained in step S2 as inputs of a multi-scale fusion module, performing a merging operation of the multi-scale fusion module, processing by a first refinement mechanism and a second refinement mechanism, restoring and performing a pixel-by-pixel addition operation with the corresponding input feature maps, and obtaining feature maps, respectively feature maps feature maps feature maps feature maps Finally, a feature map set is obtained The multi-scale fusion module enhances the features of different scale levels by balancing the semantic information of the same dimension, and the fused information contains balanced information of each resolution;

[0012] S4, constructing a prediction network, taking each feature map in the feature map set P * obtained in step S3 as an input of the prediction network, obtaining a final prediction result, predicting the class confidence by a prediction class module, predicting the position of the bounding box by a prediction bounding box module, and obtaining the bounding box center degree and the class center degree, obtaining the overall center degree by a double center fusion module from the bounding box center degree and the class center degree, obtaining the overall confidence by using the overall center degree and the class confidence, and removing the detection boxes with low overall confidence by non-maximum suppression (NMS, from Neubeck A, Efficient non-maximum suppression, published in 18th International Conference on Pattern Recognition, pages 850-855) according to the position of the predicted bounding box.

[0013] Further, the backbone network has the following specific structure:

[0014] The input is sequentially connected to the output as: convolutional layer conv1_0, BN layer conv1_0_bn, Relu layer conv1_0_relu, maximum pooling layer max_pooling1, res_1 module, res_2 module, res_3 module, and res_4 module.

[0015] The maximum pooling layer max_pooling1 is used for information compression of the feature map to reduce the calculation complexity; the res_1 module, the res_2 module, the res_3 module, and the res_4 module are all composed of ResBlock; the res_1 module is composed of three ResBlock; the res_2 module is composed of four ResBlock; the res_3 module is composed of six ResBlock; and the res_4 module is composed of three ResBlock.

[0016] Further, the ResBlock has the following specific structure:

[0017] The input is sequentially connected to the output as: convolutional layer conv2_1, BN layer conv2_1_bn, Relu layer conv2_1_relu, convolutional layer conv2_2, BN layer conv2_2_bn, Relu layer conv2_2_relu, convolutional layer conv2_3, and BN layer conv2_3_bn.

[0018] Further, the step S1 has the following process:

[0019] S11, the feature map F in is input through the convolutional layer conv1_0, the BN layer conv1_0_bn, the Relu layer conv1_0_relu, the maximum pooling layer max_pooling1, the res_1 module, and the res_2 module to obtain the feature map F

[0020] S12, the feature map F is input into the res_3 module to obtain the feature map F

[0021] S13, the feature map F is input into the res_4 module to obtain the feature map F

[0022] Further, the step S2 has the following process:

[0023] S21, the feature map F is input into the convolutional layer conv3_1 to obtain the feature map F The feature map F The feature map is input into a convolutional layer conv3_2 to obtain a feature map The feature map is input into a convolutional layer conv3_3 to obtain a feature map

[0024] S22, the feature map is added element by element with the feature map obtained after passing through an upsampling layer un_sampling2 to obtain a feature map The feature map is input into a convolutional layer conv4_2 to output a feature map P2, and the feature map P2 is thus obtained and the feature map fused feature information;

[0025] S23, the feature map is added element by element with the feature map obtained after passing through an upsampling layer un_sampling1, and the structure is input into a convolutional layer conv4_1 to output a feature map P1, and the feature map P1 is thus obtained the feature map and the feature map fused feature information;

[0026] S24, the feature map is input into a convolutional layer conv4_3 to output a feature map P3;

[0027] S25, the feature map P3 is input into a convolutional layer conv4_4 to output a feature map P4;

[0028] S26, the feature map P4 is input into a convolutional layer conv4_5 to output a feature map P5;

[0029] The feature map P4 and the feature map P5 not only contain deeper feature information than the feature map P3, but also cover a larger receptive field than the feature map P3, which is used for subsequent large target prediction;

[0030] Further, the step S3 is as follows:

[0031] S31, in order to fuse feature maps of different scales while retaining their semantic information, the feature map P1 is input into a max-pooling layer max_pooling2 to obtain a feature map The feature map P2 is input into a max-pooling layer max_pooling3 to obtain a feature map The feature map P4 is input to an up-sampling layer un_sampling3 to obtain a feature map with the same dimension as the feature map P3 The feature map P5 is input to an up-sampling layer un_sampling4 to obtain a feature map with the same dimension as the feature map P3

[0032] S32, the feature map P3 is input to a down-sampling layer down_sampling3 to obtain a feature map with the same dimension as the feature map P4 The feature map P3 is input to a down-sampling layer down_sampling3 to obtain a feature map with the same dimension as the feature map P4 The feature map P3 is input to a down-sampling layer down_sampling3 to obtain a feature map with the same dimension as the feature map P4 The feature map P3 is input to a down-sampling layer down_sampling3 to obtain a feature map with the same dimension as the feature map P4 A merging operation is performed to obtain a feature map T The merging operation is an element-by-element addition of the five feature maps, and after the addition, each element is divided by 5 to obtain a balanced semantic feature map T.

[0033] S33, in order to further enhance the resolution of the balanced semantic feature map T, the feature map T is input to a first refinement mechanism to output a feature map E Through the first refinement mechanism, each element in the feature map T is connected with the elements in the same row and the same column, and finally the obtained feature map T' contains the dependency relationship between each element and the elements in the same row and the same column.

[0034] S34, in order to obtain the remote dependency relationship of all elements in the feature map T, the feature map T' is input to a second refinement mechanism to output a feature map E Since the feature map T' contains the dependency relationship between each element and the elements in the same row and the same column, the elements in the feature map T ′ are connected with the elements in the same row and the same column, the remote dependency relationship of all elements in the feature map T can be indirectly and sparsely obtained, thereby enhancing the resolution of the feature map T.

[0035] S35, in order to fuse the refined feature map E with the multi-scale feature map, the feature map E must be scaled to the dimension of each layer of feature map; the feature map E is respectively input to an up-sampling layer un_sampling5 and an up-sampling layer un_sampling6 to obtain a feature map with the same dimension as the feature map P1 and the feature map P2 The feature map E is respectively input to a maximum pooling layer max_pooling4 and a maximum pooling layer max_pooling5 to obtain a feature map with the same dimension as the feature map P4 and the feature map P5 The feature map E is respectively input to a maximum pooling layer max_pooling4 and a maximum pooling layer max_pooling5 to obtain a feature map with the same dimension as the feature map P4 and the feature map P5 The feature map E is respectively input to a maximum pooling layer max_pooling4 and a maximum pooling layer max_pooling5 to obtain a feature map with the same dimension as the feature map P4 and the feature map P5

[0036] S36, in order to make each layer of feature map contain the information of the feature map E, the feature map P1 and the feature map E1 are added element by element to output a feature map Add feature maps P2 and E2 element by element to output the feature map. Add feature map P3 and feature map E element by element to output the feature map. Add feature maps P4 and E4 element by element to output the feature map. Add feature maps P5 and E5 element by element to output the feature map.

[0037] S37. Feature map Feature map Feature map Feature map Feature map Constructing a set of feature maps

[0038] Furthermore, the first and second refinement mechanisms share the same structure and computational method: extracting information from the same row and column to enhance the discriminative power of the feature map, thereby sparsely obtaining the relationship between elements and their corresponding rows and columns. Assuming the input feature map... Input either the first or second refinement mechanism, and output the feature map. The specific process is as follows:

[0039] To reduce computational cost, the feature map M is dimensionality-reduced by using convolutional layers conv5_1, conv5_2, and conv5_3 to obtain the feature maps respectively. Feature map and feature map Where C′0 is less than C0;

[0040] The feature maps Q and K are associated based on their element positions to output the feature map. For a certain location point (w) in feature map Q u ,h u ), forming a set Where w u h is the x-coordinate of the location point. u The ordinate of this location point is given, and the feature map K extracts the location point (w). u ,h u The eigenvectors in the same row and column form a set. It is a set The i-th u Bit element, where 1≤i u ≤H3+W3-1, the specific association operation is shown in the following formula:

[0041]

[0042] According to formula (1), an association operation is performed on each location point to obtain the same-row and same-column attention feature map D containing all location points, that is...

[0043] Normalize the feature map D using the softmax function to obtain the feature map A∈

[0044] In order to retain information from the input feature map in the feature map, the position points (w) of feature map V are then... u ,h u Extract location points (w) u ,h u The eigenvectors in the same row and column form a set. and the location point (w) of feature map A u ,h u The set of ) and the location points of feature map M (w u ,h u A set consisting of ) Perform aggregation operations to obtain feature maps.

[0045]

[0046] in, It is a set The i-th element in It is a set The i-th element in;

[0047] For each location point, formula (2) is used to calculate the context attention feature map Y containing all location points, i.e. Output feature map Y.

[0048] Furthermore, the prediction network structure includes a prediction category module, a prediction bounding box module, and a dual-center fusion module, wherein,

[0049] The category prediction module is connected sequentially from input to output as follows: convolutional layer conv6_1, convolutional layer conv6_2, convolutional layer conv6_3, convolutional layer conv6_4, and convolutional layer conv6_5; the category prediction module is used to predict the confidence level of the category.

[0050] The bounding box prediction module is connected sequentially from input to output as follows: convolutional layer conv7_1, convolutional layer conv7_2, convolutional layer conv7_3, convolutional layer conv7_4, and convolutional layer conv7_5; the bounding box prediction module is used for the parameters of the bounding boxes;

[0051] The bicenter fusion module is composed of a convolutional layer conv8_1 and a convolutional layer conv8_2, and is used to predict the overall center degree of each element.

[0052] Further, the step S4 is as follows:

[0053] S41, for the feature map set P * The feature map in the middle The feature map is the feature map set P * The feature map in the middle of the lth, 1≤l≤5, is obtained through a convolutional layer conv6_1, a convolutional layer conv6_2, a convolutional layer conv6_3, and a convolutional layer conv6_4 in the prediction category module.

[0054] S42, the feature map Q l The feature map L used for predicting the category is obtained through a convolutional layer conv6_5. l ∈R W×H×N Where N is the number of predicted categories, and the category confidence of each position point can be obtained from the feature map L l .

[0055] S43, the feature map is obtained through a convolutional layer conv7_1, a convolutional layer conv7_2, a convolutional layer conv7_3, and a convolutional layer conv7_4 in the prediction bounding box module.

[0056] S44, the feature map G l The feature map used for predicting the bounding box is obtained through a convolutional layer conv7_5. 4 distance parameters are predicted at each position of the feature map G l , which are the distance from the left side distance , the distance from the upper side distance , the distance from the right side distance , and the distance from the lower side distance For a position (b l , b x ) of the feature map B y , where b x is the horizontal coordinate of the position point and b y is the vertical coordinate of the position point, the formula is:

[0057]

[0058] Where , respectively, at the position point (b x , b yThe coordinates of the left, right, top, and bottom boundaries of the detection box are formed with ) as the center.

[0059] S45, Transfer feature map Q l The feature map Ψ∈R of class centrality is obtained through the convolutional layer conv8_1 in the dual-center fusion module. W×H×1 , feature map G l The feature map Θ∈R of the bounding box centrality is obtained through the convolutional layer conv8_2 in the dual-center fusion module. W×H×1 ;

[0060] Considering that centrality is related to bounding box information and semantic information, it is necessary to perform a center fusion calculation on the class centrality and bounding box centrality for a certain position (w) of feature map Ψ and feature map Θ. d ,h d ), where w d h is the x-coordinate of the location point. d Using the ordinate of this location point, perform center fusion calculation to obtain the position (w) d ,h d Overall centrality

[0061]

[0062] in, It is the feature map Ψ position (w) d ,h d The value of ) It is the feature map Θ position (w) d ,h d The values ​​of ) are α and β respectively. The balance parameters;

[0063] S46. Calculate the overall centrality obtained in step S45. With feature map L l At position (w) d ,h d ) corresponding category confidence Calculations are performed to obtain the final (w) d ,h d The overall confidence level of the bounding box centered at )

[0064]

[0065] Then based on the overall confidence level Then, the NMS method is used to filter out bounding boxes with low overall confidence.

[0066] S47. For feature map Q l and feature map Gl The step S45 and the step S46 are performed for each position point of the detection result, and a final detection result is obtained.

[0067] The present application has the following advantages and effects relative to the prior art:

[0068] The present application first fuses a multi-scale fusion module and a double-center fusion module into an anchor-free target detection method. First, in order to realize multi-scale feature enhancement, a multi-scale fusion module is used to realize balanced multi-scale feature enhancement, and a feature map is refined in a manner of extracting the same row and the same column, and twice refinement mechanism is performed between feature pixels in the refinement to improve the fusion capability. Second, in view of the lack of feature information of the center degree, a double-center fusion module is used for optimization, the double-center fusion module combines the boundary box center degree and the category center degree to form the overall center degree, and then calculates the overall confidence according to the overall center degree and the category confidence, and uses the overall confidence to evaluate the quality of the boundary box. BRIEF DESCRIPTION OF DRAWINGS

[0069] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0070] Figure 1 is a structure diagram of the anchor-free target detection method based on multi-scale and double-center fusion disclosed in the present application;

[0071] Figure 2 is a structure diagram of the backbone network and the neck network in the anchor-free target detection method based on multi-scale and double-center fusion in the present application;

[0072] Figure 3 is a column chart of the accuracy rate comparison between the present method and other target detection methods under the premise of embodiment 1 using the HyperKvasir dataset;

[0073] Figure 4 is a column chart of the accuracy rate comparison between the present method and other target detection methods under the premise of embodiment 2 using the Kvasir-Instrument dataset. DETAILED DESCRIPTION

[0074] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0075] Embodiment 1

[0076] In this embodiment, a real data set from a hospital is disclosed, named HyperKvasir data set, and each picture in the data set has a polyp lesion. The data set has 1000 pictures.

[0077] The method comprises the following steps:

[0078] Step S1, processing the data set: first, data segmentation is performed on the data set, and the data set is segmented into a training set of 800 samples and a verification set of 200 samples.

[0079] Step S2, experimental environment, the experimental environment of this embodiment only needs a general hardware configuration and a graphics processing unit (GPU) that can improve the calculation speed to perform accelerated operation. The model building, training and training result testing are all completed under the Pytorch deep learning framework and the mmdetection tool, and the Compute Unified Device Architecture (CUDA) is used to enable the GPU to solve complex calculation problems. The specific running environment configuration required by the experiment of this embodiment is shown in Table 1.

[0080] Table 1. Experimental environment configuration table of this embodiment

[0081]

[0082] Step S3, constructing the backbone network, the structure diagram of the backbone network is as shown in Figure 2 The specific structure of the backbone network is as follows:

[0083] Connected in order from input to output: convolution layer conv1_0, BN layer conv1_0_bn, Relu layer conv1_0_relu, maximum pooling layer max_pooling1, res_1 module, res_2 module, res_3 module, res_4 module; wherein the res_1 module, the res_2 module, the res_3 module and the res_4 module are all composed of ResBlock; the res_1 module is composed of 3 ResBlock; the res_2 module is composed of 4 ResBlock; the res_3 module is composed of 6 ResBlock; the res_4 module is composed of 3 ResBlock, and the specific structure of the ResBlock is as follows:

[0084] The input is sequentially connected to the output as: the convolution layer conv2_1, the BN layer conv2_1_bn, the Relu layer conv2_1_relu, the convolution layer conv2_2, the BN layer conv2_2_bn, the Relu layer conv2_2_relu, the convolution layer conv2_3, and the BN layer conv2_3_bn.

[0085] The image F to be detected is input into the backbone network for feature extraction, wherein C represents the number of feature map channels, W and H are the height and width of the feature map, and R represents the real field; the outputs are feature maps F1, F2 and F3 respectively. in ∈R C×W×H Input into the backbone network for feature extraction, wherein C represents the number of feature map channels, W and H are the height and width of the feature map, and R represents the real field; the outputs are feature maps F1, F2 and F3 respectively. Feature map Feature map Wherein C1, C2, C3 are the number of channels of the feature map after dimension reduction by the backbone network, W1, W2, W3 are the width of the feature map after dimension reduction by the backbone network, H1, H2, H3 are the height of the feature map after dimension reduction by the backbone network, and the specific process is as follows:

[0086] S31, the feature map F is input into the convolution layer conv1_0, the BN layer conv1_0_bn, the Relu layer conv1_0_relu, the maximum pooling layer max_pooling1, the res_1 module and the res_2 module to obtain the feature map F in

[0087] S32, the feature map F is input into the res_3 module to obtain the feature map F

[0088] S33, the feature map F is input into the res_4 module to obtain the feature map F

[0089] Step S4, constructing the neck network, the structure diagram of the neck network is as shown in Figure 2 Feature map Feature map Input into the neck network for different scale feature fusion, and the outputs are feature maps F0, F1, F2, F3 and F4 respectively. Feature map Feature map Feature map Feature map Wherein C0 is the number of channels of the feature map after dimension reduction, W4 and W5 are the width of the feature map after dimension reduction, and H4 and H5 are the height of the feature map after dimension reduction, and the specific process is as follows:

[0090] ​​​​S41, input the feature map into the convolutional layer conv3_1 to obtain a feature map input the feature map into the convolutional layer conv3_2 to obtain a feature map input the feature map into the convolutional layer conv3_3 to obtain a feature map

[0091] S42, input the feature map add the feature map element by element, to obtain a feature map input the feature map into the convolutional layer conv4_2 to output a feature map P2;

[0092] S43, input the feature map add the feature map element by element, and input the structure into the convolutional layer conv4_1 to output a feature map P1;

[0093] S44, input the feature map into the convolutional layer conv4_3 to output a feature map P3;

[0094] S45, input the feature map P3 into the convolutional layer conv4_4 to output a feature map P4;

[0095] S46, input the feature map P4 into the convolutional layer conv4_5 to output a feature map P5.

[0096] Step S5, construct a multi-scale fusion module, the structure diagram of the multi-scale fusion module is as shown in Figure 1 , input the feature map P1, the feature map P2, the feature map P3, the feature map P4, and the feature map P5 obtained in step S4 into the multi-scale fusion module, process through the merging operation, the first refinement mechanism and the second refinement mechanism of the multi-scale fusion module, restore and add pixel by pixel with the corresponding input feature map, and the obtained feature map is respectively a feature map a feature map a feature map a feature map a feature map Finally, a feature map set is obtained The specific process is as follows:

[0097] S51, input the feature map P1 into a max pooling layer max_pooling2 to obtain a feature map with the same dimension as the feature map P3 input the feature map P2 into a max pooling layer max_pooling3 to obtain a feature map with the same dimension as the feature map P3 input the feature map P4 into an up-sampling layer un_sampling3 to obtain a feature map with the same dimension as the feature map P3 input the feature map P5 into an up-sampling layer un_sampling4 to obtain a feature map with the same dimension as the feature map P3

[0098] S52, input the feature map P1 and the feature map E1 into a merge layer to obtain a feature map the feature map P1 and the feature map E1 are added element by element to obtain a feature map the feature map P3 and the feature map E are added element by element to obtain a feature map the feature map P4 and the feature map E4 are added element by element to obtain a feature map perform a merge operation to obtain a feature map the merge operation is to add the five feature maps element by element, and then divide each element by 5 to obtain a new feature map T;

[0099] S53, input the feature map T into a first refinement mechanism to output a feature map

[0100] S54, input the feature map T' into a second refinement mechanism to output a feature map

[0101] S55, the feature map E is respectively input into an up-sampling layer un_sampling5 and an up-sampling layer un_sampling6 to obtain a feature map with the same dimension as the feature map P1 and the feature map P2 the feature map E is respectively input into a max pooling layer max_pooling4 and a max pooling layer max_pooling5 to obtain a feature map with the same dimension as the feature map P4 and the feature map P5 the feature map E is respectively input into a max pooling layer max_pooling4 and a max pooling layer max_pooling5 to obtain a feature map with the same dimension as the feature map P4 and the feature map P5 the feature map E is respectively input into a max pooling layer max_pooling4 and a max pooling layer max_pooling5 to obtain a feature map with the same dimension as the feature map P4 and the feature map P5

[0102] S56, add the feature map P1 and the feature map E1 element by element to output a feature map add the feature map P2 and the feature map E2 element by element to output a feature map add the feature map P3 and the feature map E element by element to output a feature map add the feature map P4 and the feature map E4 element by element to output a feature map add the feature map P5 and the feature map E5 element by element to output a feature map

[0103] S57, input the feature map feature map feature map feature map feature map constructing a feature map set

[0104] wherein the structure and calculation method of the first refinement mechanism and the second refinement mechanism are the same, assuming that the input feature map inputting the first refinement mechanism or the second refinement mechanism, and outputting a feature map The specific process is as follows:

[0105] The feature map M is respectively obtained through the convolution layer conv5_1, the convolution layer conv5_2 and the convolution layer conv5_3. feature map and the feature map wherein C'0 is less than C0;

[0106] The feature map Q and the feature map K are associated according to the element positions, and a feature map wherein for a position point (w u ,h u ) of the feature map Q, a set is formed, wherein w u is the horizontal coordinate of the position point, h u is the vertical coordinate of the position point, and the feature vector of the position point (w u ,h u ) extracted from the feature map K is in the same row and the same column, and a set is the i u th element of the set , wherein 1≤i u ≤H3+W3-1, and the specific association operation is shown in the following formula:

[0107]

[0108] According to the association operation of formula (1) on each position point, a same-row-and-same-column attention feature map D containing all position points is obtained, that is,

[0109] The feature map D is normalized through a softmax function to obtain a feature map A∈

[0110] The position point (w u ,h u ) of the feature map V is extracted, the position point (w u ,hu The eigenvectors in the same row and column form a set. And with the location point (w) of feature map A u ,h u The set of ) and the location points of feature map M (w u ,h u A set consisting of ) Perform aggregation operations to obtain feature maps.

[0111]

[0112] in, It is a set The i-th element in It is a set The i-th element in;

[0113] For each location point, formula (2) is used to calculate the context attention feature map Y containing all location points, i.e. Output feature map Y.

[0114] Step S6: Construct the prediction network. The structure diagram of the prediction network is shown below. Figure 1 As shown, the prediction network structure includes a prediction category module, a prediction bounding box module, and a two-center fusion module, wherein...

[0115] The prediction category module is connected sequentially from input to output as follows: convolutional layer conv6_1, convolutional layer conv6_2, convolutional layer conv6_3, convolutional layer conv6_4, and convolutional layer conv6_5;

[0116] The bounding box prediction module is connected sequentially from input to output as follows: convolutional layer conv7_1, convolutional layer conv7_2, convolutional layer conv7_3, convolutional layer conv7_4, and convolutional layer conv7_5;

[0117] The dual-center fusion module consists of convolutional layer conv8_1 and convolutional layer conv8_2.

[0118] The feature map set P obtained in step S5 *Each feature map in the prediction network is used as input to obtain the final prediction result. The class prediction module predicts the class confidence, and the bounding box prediction module predicts the bounding box position, obtaining the bounding box centrality and class centrality. The bounding box centrality and class centrality are then used by the dual-center fusion module to obtain the overall centrality. Finally, the overall centrality and class confidence are used to obtain the overall confidence. The predicted bounding box position is then used to remove detection boxes with low overall confidence using non-maximum suppression (NMS) based on the overall confidence. The specific process is as follows:

[0119] S61. For the feature map set P * Middle feature map Feature map It is a set of feature maps P * The l-th feature map in the prediction class, where 1 ≤ l ≤ 5, is processed by convolutional layers conv6_1, conv6_2, conv6_3, and conv6_4 in the prediction class module to obtain the feature map.

[0120] S62, Feature Map Q l The feature map L for predicting the class is obtained through the convolutional layer conv6_5. l ∈R W×H×N Where N is the number of predicted categories, from feature map L l The category confidence score for each location point can be obtained from the data.

[0121] S63, Feature Map The feature map is obtained by passing through convolutional layers conv7_1, conv7_2, conv7_3, and conv7_4 in the bounding box prediction module.

[0122] S64, Feature Map G l The feature map B for predicting the bounding box is obtained through the convolutional layer conv7_5. l ∈R W×H×4 In feature map G l Predict four distance parameters at each location: distance to the left of the location, distance to the left of the location, and distance to the left of the location. upper distance Right distance and lower side distance For feature map B l a certain position (b) x ,b y ), where b x Let b be the x-coordinate of the point. y Let be the ordinate of the point, then the formula is:

[0123]

[0124] in At location point (b) x ,b y The coordinates of the left, right, top, and bottom boundaries of the detection box are formed with ) as the center.

[0125] S65, Transfer feature map Q l The feature map Ψ∈R of class centrality is obtained through the convolutional layer conv8_1 in the dual-center fusion module. W×H×1 , feature map G l The feature map Θ∈R of the bounding box centrality is obtained through the convolutional layer conv8_2 in the dual-center fusion module. W×H×1 For a certain position (w) of feature map Ψ and feature map Θ d ,h d ), where w d h is the x-coordinate of the location point. d Using the ordinate of this location point, perform center fusion calculation to obtain the position (w) d ,h d overall centrality

[0126]

[0127] in, It is the feature map Ψ position (w) d ,h d The value of ) It is the feature map Θ position (w) d ,h d The values ​​of ) are α and β respectively. The balance parameters are set as follows: In this embodiment, α = 0.7 and β = 0.3.

[0128] S66. Calculate the overall centrality obtained in step S65. With feature map L l At position (w) d ,h d ) corresponding category confidence Calculations are performed to obtain the final (w) d ,h d The overall confidence level of the bounding box centered at )

[0129]

[0130] Then based on the overall confidence level The NMS method is used to filter bounding boxes with low overall confidence.

[0131] S67, performing step S65, step S66 on each location point of the feature map Q l and the feature map G l Step S65, step S66 are performed on each location point of the feature map Q

[0132] Step S7, input image training set for training model, in the process of training, the model in the loss value of the validation set tends to be stable state, it is indicated that the model has converged. The detailed parameters are as follows: the optimizer uses the random gradient descent, the initial value is 0.001 learning rate and 0.9 momentum, the batch size is set to 16. The number of training epoch is 30, the learning rate is reduced by 0.1 rate after the 10th iteration and the 20th iteration. The threshold value of filtering bounding box is 0.5. We use the pre-trained weight on ImageNet dataset to initialize the backbone network. Update the parameters of all stages of the backbone network during training.

[0133] Step S8, verify the model, input the validation set into the model, get the detection result, wherein the batch size is set to 1, the threshold value of filtering bounding box is set to 0.3, and the final accuracy is calculated.

[0134] The comparison results of the experiment of embodiment 1 are shown in Figure 3 The embodiment 1 is compared with other target detection methods on 200 HyperKvasir validation sets, and the method obtains an accuracy of 71%, which is 6.6% higher than that of the Faster R-CNN method, 7.3% higher than that of the SSD method, 22% higher than that of the CenterNet method, 9.4% higher than that of the FSAF method, 3.1% higher than that of the ATSS method, and 5.3% higher than that of the traditional FCOS method, which further verifies the effectiveness of the present application.

[0135] Embodiment 2

[0136] In this embodiment, the data set comes from a publicly available real data set, named Kvasir-Instrument data set, and each picture in the data set has an instrument picture. The data set has 590 pictures.

[0137] The method comprises the following steps:

[0138] Step S1, processing data set: first, data set is divided into training set 472 pictures, validation set 118 pictures.

[0139] Step S2, experimental environment, the experimental environment of the embodiment only needs general hardware configuration and graphics processing unit (GPU) that can improve the calculation speed to perform accelerated operation. Among them, the model building, training and test of training results are completed under the Pytorch deep learning framework and mmdetection tool, using compute unified device architecture (CUDA) to enable GPU to solve complex calculation problems. Specifically, the hardware and software configuration used in the embodiment is as follows: the GPU version is GeForce RTX2080Ti; the CPU version is Intel(R) Xeon(R) Silver 4216 CPU@2.10GHz; the operating system version is CentOS 8.3.2011; the Python version is 3.6.13; and the CUDA version is 11.2.

[0140] Step S3, constructing the backbone network, the structure diagram of the backbone network is as shown in Figure 2 The specific structure of the backbone network is as follows:

[0141] Connected in order from input to output are: convolution layer conv1_0, BN layer conv1_0_bn, Relu layer conv1_0_relu, maximum pooling layer max_pooling1, res_1 module, res_2 module res_3 module, res_4 module; wherein, the res_1 module, the res_2 module, the res_3 module and the res_4 module are all composed of ResBlock; the res_1 module is composed of 3 ResBlock; the res_2 module is composed of 4 ResBlock; the res_3 module is composed of 6 ResBlock; the res_4 module is composed of 3 ResBlock, and the specific structure of the ResBlock is as follows:

[0142] Connected in order from input to output are: convolution layer conv2_1, BN layer conv2_1_bn, Relu layer conv2_1_relu, convolution layer conv2_2, BN layer conv2_2_bn, Relu layer conv2_2_relu, convolution layer conv2_3, BN layer conv2_3_bn.

[0143] The to-be-detected image F in ∈R C×W×H is input into the backbone network for feature extraction, wherein C represents the number of feature map channels, W and H are the height and width of the feature map, and R represents the real number field; the outputs are feature maps Feature map Feature map wherein C1, C2, C3 are the channel numbers of the feature maps after dimension reduction by the backbone network, W1, W2, W3 are the widths of the feature maps after dimension reduction by the backbone network, H1, H2, H3 are the heights of the feature maps after dimension reduction by the backbone network.

[0144] Step S4, a neck network is constructed, and a structural diagram of the neck network is as shown in Figure 2 The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. wherein C0 is the channel number of the feature map after dimension reduction, W4, W5 are the widths of the feature maps after dimension reduction, and H4, H5 are the heights of the feature maps after dimension reduction.

[0145] Step S5, a multi-scale fusion module is constructed, and a structural diagram of the multi-scale fusion module is as shown in Figure 1 The feature maps P1, P2, P3, P4 and P5 obtained in step S4 are input into the multi-scale fusion module as inputs, and through the merging operation of the multi-scale fusion module, the processing of the first and second refinement mechanisms, restoration and pixel-by-pixel addition operation with the corresponding input feature maps, the obtained feature maps are respectively feature maps P1”, P2”, P3”, P4” and P5”. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. The feature maps P1, P2, P3, P4 and P5 obtained in step S3 are input into the neck network for different scale feature fusion, and the outputs are respectively feature maps P1’, P2’, P3’, P4’ and P5’. Finally, the feature map set

[0146] Step S6, a prediction network is constructed, and a structural diagram of the prediction network is as shown in Figure 1 The prediction network structure includes a prediction category module, a prediction bounding box module and a double center fusion module, wherein,

[0147] The prediction category module is sequentially connected from input to output as: a convolutional layer conv6_1, a convolutional layer conv6_2, a convolutional layer conv6_3, a convolutional layer conv6_4 and a convolutional layer conv6_5.

[0148] The prediction bounding box module is sequentially connected from input to output as: a convolutional layer conv7_1, a convolutional layer conv7_2, a convolutional layer conv7_3, a convolutional layer conv7_4 and a convolutional layer conv7_5.

[0149] The double-center fusion module is composed of the convolutional layer conv8_1 and the convolutional layer conv8_2.

[0150] Each feature map in the feature map set P obtained in step S5 is taken as an input of the prediction network, and a final prediction result is obtained, the class confidence is predicted by the prediction class module, the bounding box position is predicted by the prediction bounding box module, and the bounding box center degree and the class center degree are obtained, the overall center degree is obtained by the double-center fusion module from the bounding box center degree and the class center degree, the overall confidence is obtained by using the overall center degree and the class confidence, and the predicted bounding box position is removed by the non-maximum suppression (NMS) according to the overall confidence, and the specific process is as follows:

[0151] S61, for the feature map in the feature map set P * The feature map is the lth feature map in the feature map set P * , 1≤l≤5, and the feature map

[0152] S62, the feature map Q l is obtained by the convolutional layer conv6_5, and the feature map L l ∈R W×H×N is obtained, where N is the number of predicted classes, and the class confidence of each position point can be obtained from the feature map L l .

[0153] S63, the feature map is taken as an input of the prediction bounding box module, and the feature map

[0154] S64, the feature map G l is obtained by the convolutional layer conv7_5, and the feature map B l ∈R W×H×4 is obtained, and four distance parameters are predicted at each position of the feature map G l , which are left distance , upper distance , right distance , and lower distance For a position (b l of the feature map B x , the left distance is calculated as follows:​x ,b y ), where b x Let b be the x-coordinate of the location. y Let be the ordinate of the point, then the formula is:

[0155]

[0156] in At location point (b) x ,b y The coordinates of the left, right, top, and bottom boundaries of the detection box are formed with ) as the center.

[0157] S65, Transfer feature map Q l The feature map Ψ∈R of class centrality is obtained through the convolutional layer conv8_1 in the dual-center fusion module. W×H×1 , feature map G l The feature map Θ∈R of the bounding box centrality is obtained through the convolutional layer conv8_2 in the dual-center fusion module. W×H×1 For a certain position (w) of feature map Ψ and feature map Θ d ,h d ), where w d h is the x-coordinate of the location point. d Using the ordinate of this location point, perform center fusion calculation to obtain the position (w) d ,h d overall centrality

[0158]

[0159] in, It is the feature map Ψ position (w) d ,h d The value of ) It is the feature map Θ position (w) d ,h d The values ​​of ) are α and β respectively. The balance parameters are set as follows: In this embodiment, α = 0.5 and β = 0.5.

[0160] S66. Calculate the overall centrality obtained in step S65. With feature map L l At position (w) d ,h d ) corresponding category confidence Calculations are performed to obtain the final (w) d ,h d The overall confidence level of the bounding box centered at )

[0161]

[0162] According to the overall confidence The NMS method is used to filter the bounding box with low overall confidence.

[0163] S67, performing steps S65 and S66 on each position point of the feature map Q l and the feature map G l to obtain the final detection result.

[0164] Step S7, input the image training set to train the model. During the training process, when the loss value of the model on the validation set tends to be stable, it means that the model has converged. The detailed parameters are as follows: the optimizer uses the stochastic gradient descent, the initial value of the learning rate is 0.0001 and the momentum is 0.8, and the batch size is set to 16. The number of training epochs is 30, and the learning rate is reduced by 0.1 after the 10th iteration and the 20th iteration. The threshold for filtering bounding boxes is 0.6. We use the pre-trained weights on the ImageNet dataset to initialize the backbone network. During the training, the parameters of all stages of the backbone network are updated.

[0165] Step S8, verify the model, input the validation set into the model to obtain the detection result, wherein the batch size is set to 1 and the threshold for filtering bounding boxes is set to 0.3, and the final accuracy is calculated.

[0166] The comparison results of the experiments of embodiment 2 are shown in Figure 4 The embodiment 2 is compared with other target detection methods on the 118 Kvasir-Instrument validation set, and the method obtains an accuracy of 75.6%, which is 6.5% higher than the Faster R-CNN method, 10.7% higher than the SSD method, 15.8% higher than the CenterNet method, 6% higher than the FSAF method, 2.4% higher than the ATSS method, 5.6% higher than the traditional FCOS method, which proves the effectiveness of the method.

[0167] In summary, the above embodiment discloses a target detection method based on anchor-free target detection based on multi-scale and double-center fusion, which uses two modules of multi-scale fusion and double-center fusion to optimize FCOS. The multi-scale module realizes balanced multi-scale feature enhancement, and uses the first refinement mechanism and the second refinement mechanism to refine the relevant information of the element set with the same row and column between the feature elements to improve the fusion ability. The double-center fusion module contains the semantic information and regression information of the target. We use the double-center fusion module to form a total center degree, and use the total center degree and the class confidence to calculate the overall confidence, and evaluate the quality of the bounding box according to the overall confidence.

[0168] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications, etc. made without departing from the spirit and principles of the present application should be equivalent replacement manners and should be included in the protection scope of the present application.

Claims

1. A method for anchor-free object detection based on multi-scale and dual-center fusion, characterized in that, The detection method comprises the following steps: S1, a backbone network is constructed, and an image to be detected F in ∈R C×W×H is input into the backbone network for feature extraction, wherein C represents the number of image channels, W and H are the height and width of the image, and R represents the real number field; the outputs are feature maps feature map feature map wherein C1, C2 and C3 are the number of channels of the feature maps after dimension reduction by the backbone network, W1, W2 and W3 are the width of the feature maps after dimension reduction by the backbone network, and H1, H2 and H3 are the height of the feature maps after dimension reduction by the backbone network. S2, obtaining a feature map from step S1 feature map feature map inputting the neck network to fuse different scale features, and outputting feature maps respectively feature map feature map feature map feature map wherein C0 is the channel number of the feature map after dimension reduction, W4 and W5 are the width of the feature map after dimension reduction, and H4 and H5 are the height of the feature map after dimension reduction. S3, input the feature maps P1, P2, P3, P4 and P5 obtained in step S2 to a multi-scale fusion module, perform a merging operation on the multi-scale fusion module, process by a first refinement mechanism and a second refinement mechanism, restore and perform a pixel-by-pixel addition operation with the corresponding input feature maps, and obtain feature maps P6, P7, P8, P9 and P10, respectively. feature map feature map feature map feature map Finally, a feature map set S4. Construct a prediction network and use the feature map set P obtained in step S3. * Each feature map in the prediction network is used as input to obtain the final prediction result. The prediction class module predicts the class confidence, and the prediction bounding box module predicts the bounding box position, obtaining the bounding box centrality and class centrality. These are then combined using a two-center fusion module to obtain the overall centrality. Finally, the overall centrality and class confidence are used to obtain the overall confidence. The predicted bounding box position is then determined using Non-Maximum Suppression (NMS) based on the overall confidence to remove detection boxes with low overall confidence. The prediction network includes a prediction class module, a prediction bounding box module, and a two-center fusion module. The prediction category module is sequentially connected from input to output as: a convolutional layer conv6_1, a convolutional layer conv6_2, a convolutional layer conv6_3, a convolutional layer conv6_4 and a convolutional layer conv6_5; The prediction bounding box module is sequentially connected from input to output as: a convolutional layer conv7_1, a convolutional layer conv7_2, a convolutional layer conv7_3, a convolutional layer conv7_4 and a convolutional layer conv7_5; The double-center fusion module is composed of a convolutional layer conv8_1 and a convolutional layer conv8_2; The step S4 process is as follows: S41, for the feature map set P * the feature map in the middle the feature map is the feature map set P * the feature map in the middle, 1≤l≤5, through the convolutional layer conv6_1, the convolutional layer conv6_2, the convolutional layer conv6_3, and the convolutional layer conv6_4 in the prediction category module to obtain the feature map S42, feature map Q l The feature map L for predicting the category is obtained through the convolution layer conv6_5 l ∈R W×H×N where N is the number of predicted categories, and the category confidence of each position point can be obtained from the feature map Ll. S43, the feature map The feature map is obtained through the convolution layer conv7_1, the convolution layer conv7_2, the convolution layer conv7_3 and the convolution layer conv7_4 in the predicted boundary box module S44, feature map G l By the convolution layer conv7_5, a feature map B for predicting the bounding box is obtained l ∈R W×H×4 At each position of the feature map G l 4 distance parameters are predicted, respectively, the distance from the left side of the position the distance from the top side of the position the distance from the right side of the position and the distance from the bottom side of the position For a position (b x , b y ) of the feature map B l , where b x is the horizontal coordinate of the position point and b y is the vertical coordinate of the position point, the formula is: wherein respectively, the left boundary, the right boundary, the upper boundary and the lower boundary of the detection frame formed with the position points (b x ,b y ) as the center. S45, the feature map Q l is obtained by the convolution layer conv8_1 in the dual center fusion module W×H×1 , the feature map G l is obtained by the convolution layer conv8_2 in the dual center fusion module W×H×1 , the feature map Θ d is obtained by the convolution layer conv8_2 in the dual center fusion module d , the feature map Θ d is obtained by the convolution layer conv8_2 in the dual center fusion module d , the feature map Θ d is obtained by the convolution layer conv8_2 in the dual center fusion module d , the feature map Θ in, It is the feature map Ψ position (w) d ,h d The value of ) It is the feature map Θ position (w) d ,h d The values ​​of ) are α and β respectively. The balance parameters; S46, the overall center degree calculated in step S45 with the feature map L l At the position (w d ,h d ), the corresponding category confidence is calculated to obtain the final overall confidence of the bounding box formed with (w d ,h d ) as the center Further according to the overall confidence Further, the NMS method is used to filter the bounding boxes with low overall confidence; S47, performing step S45 and step S46 on each position point of the feature map Q l and the feature map G l S47, performing step S45 and step S46 on each position point of the feature map Q 2. The multi-scale and dual-center fusion based anchor-free object detection method according to claim 1, characterized in that, The backbone network has the following specific structure: The input is sequentially connected to the output as: a convolutional layer conv1_0, a BN layer conv1_0_bn, a Relu layer conv1_0_relu, a maximum pooling layer max_pooling1, a res_1 module, a res_2 module, a res_3 module and a res_4 module. The res_1 module, the res_2 module, the res_3 module and the res_4 module are all composed of ResBlock; the res_1 module is composed of three ResBlock; the res_2 module is composed of four ResBlock; the res_3 module is composed of six ResBlock; and the res_4 module is composed of three ResBlock.

3. The multi-scale and dual-center fusion based anchor-free object detection method according to claim 2, characterized in that, The ResBlock has the following specific structure: The input is sequentially connected to the output as: a convolutional layer conv2_1, a BN layer conv2_1_bn, a Relu layer conv2_1_relu, a convolutional layer conv2_2, a BN layer conv2_2_bn, a Relu layer conv2_2_relu, a convolutional layer conv2_3 and a BN layer conv2_3_bn.

4. The multi-scale and dual-center fusion based anchor-free object detection method according to claim 3, characterized in that, The step S1 process is as follows: S11, the feature map F in As input, the feature map F is input through the convolution layer conv1_0, the BN layer conv1_0_bn, the Relu layer conv1_0_relu, the max pooling layer max_pooling1, the res_1 module, and the res_2 module to obtain a feature map F S12, the feature map input to the res_3 module to obtain the feature map S13, the feature map input to the res_4 module to obtain the feature map 5. The multi-scale and dual-center fusion based anchor-free object detection method according to claim 1, wherein, The step S2 process is as follows: S21, the feature map is input into the convolutional layer conv3_1 to obtain a feature map is input into the convolutional layer conv3_2 to obtain a feature map is input into the convolutional layer conv3_3 to obtain a feature map ​​ S22, the feature map by upsampling layer un_sampling2 and the feature map element-wise addition, to obtain the feature map the feature map the structure input to the convolution layer conv4_2, output the feature map P2; S23, the feature map by upsampling layer un_sampling1 and the feature map element-wise addition is performed, and the structure is input into the convolutional layer conv4_1, and the feature map P1 is output; S24, the feature map is input into the convolutional layer conv4_3, and the feature map P3 is output; S25, input the feature map P3 into a convolutional layer conv4_4, and output a feature map P4; S26, input the feature map P4 into a convolutional layer conv4_5, and output a feature map P5.

6. The multi-scale and dual-center fusion based anchor-free object detection method according to claim 1, characterized in that, The step S3 process is as follows: S31, input the feature map P1 into a max pooling layer max_pooling2 to obtain a feature map with the same dimension as the feature map P3 input the feature map P2 into a max pooling layer max_pooling3 to obtain a feature map with the same dimension as the feature map P3 input the feature map P4 into an up-sampling layer un_sampling3 to obtain a feature map with the same dimension as the feature map P3 input the feature map P5 into an up-sampling layer un_sampling4 to obtain a feature map with the same dimension as the feature map P3 S32, the feature map the feature map the feature map P3, the feature map the feature map performing a merging operation to obtain a feature map The merging operation is to add the five feature maps element by element, and then divide each element by 5 to obtain a new feature map T. S33, inputting the feature map T into a first refinement mechanism to output a feature map S34, the feature map T ′ input to the second refinement mechanism outputs a feature map S35, the feature map E respectively obtains the feature map with the same dimension as the feature map P1, the feature map P2 through the up-sampling layer un_sampling5, the up-sampling layer un_sampling6 respectively feature map feature map feature map S36, add the feature map P1 and the feature map E1 element by element to output a feature map add the feature map P2 and the feature map E2 element by element to output a feature map add the feature map P3 and the feature map E element by element to output a feature map add the feature map P4 and the feature map E4 element by element to output a feature map add the feature map P5 and the feature map E5 element by element to output a feature map S37, constructing a feature map feature map feature map feature map feature map constructing a feature map set 7. The multi-scale and dual-center fusion based anchor-free object detection method according to claim 1, characterized in that, The structure and calculation method of the first refinement mechanism and the second refinement mechanism are the same, assuming that the input feature map The input of the first refinement mechanism or the second refinement mechanism is the output feature map The specific process is as follows: The feature map M is respectively obtained through the convolution layer conv5_1, the convolution layer conv5_2 and the convolution layer conv5_3 feature map and feature map wherein C0 ′ is less than C0; The feature maps Q and K are associated based on their element positions to output the feature map. For a certain location point (w) in feature map Q u ,h u ), forming a set Where w u h is the x-coordinate of the location point. u The ordinate of this location point is given, and the feature map K extracts the location point (w). u ,h u The eigenvectors in the same row and column form a set. It is a set The i-th u Bit element, where 1≤i u ≤H3+W3-1, the specific association operation is shown in the following formula: According to the formula (1), the correlation operation is performed on each position point to obtain the same row and column attention feature map D containing all position points, that is The feature map D is normalized by a softmax function to obtain a feature map The position points (w u ,h u ) of the feature map V are extracted, the feature vectors of the position points (w u ,h u ) in the same row and the same column are constituted into a set , and the set of the position points (w u ,h u ) of the feature map A and the set of the position points (w u ,h u ) of the feature map M are aggregated to obtain a feature map wherein is the i-th element of the set is the i-th element of the set is the i-th element of the set is the i-th element of the set The formula (2) is calculated for each position point to obtain the context attention feature map Y containing all position points, that is The output feature map Y.

Images