A yolov3 road garbage detection method based on decoupling head

Abstract

The application discloses a Yolov3 road garbage detection method based on a decoupling head, and belongs to the technical field of computer vision, the detection method comprises the steps of establishing an improved Yolov3 network, the improved Yolov3 network comprises a backbone network, a Neck structure and a detection head, the Neck structure comprises a channel attention mechanism CA; the backbone network is optimized through the obtained training sample set, and then a road garbage recognition network with the best recognition effect is obtained, and the network is used for realizing detection and recognition of road garbage. Compared with the traditional detection method, the recognition method can better realize feature separation and fusion, and improve the recognition ability of the network. The accuracy of the method in the test sample set is higher than that of other methods, and the method can better adapt to complex road environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a Yolov3 road garbage detection method based on decoupled heads. BACKGROUND

[0002] With the development of the times, computer vision technology has been deeply integrated into various industries, and the use of deep learning and image processing methods to realize the recognition of road garbage has become a research hotspot in the field of computer vision in environmental engineering applications in recent years. Road garbage located in open areas of cities has the characteristics of small target, multiple types, and different shapes, and has relatively complex features. The traditional target detection method is easily disturbed by complex features, which can cause false detection and missed detection. Therefore, directly applying the traditional target detection method to the field of road garbage recognition will have certain limitations.

[0003] At present, the traditional road garbage recognition method mainly relies on manual operation, and manual operation has low classification efficiency. Especially when dealing with a large amount of garbage, the harsh classification environment and heavy task can pose a threat to the health of the operator, and it is easy to cause recognition errors due to difficult tasks. Mixed garbage not only easily pollutes the environment, but also may cause waste of recyclable resources. Another method that can be used is to establish a picture database of various garbage, and use image comparison technology and scale-invariant feature transformation matching algorithms to recognize various garbage objects, but this method is only suitable for garbage recognition occasions with a fixed background of a conveyor belt in a garbage treatment plant, and cannot be applied to road garbage recognition occasions with complex backgrounds. In addition, it is difficult to guarantee the recognition accuracy in a multi-target task, and cannot meet the actual needs of road garbage recognition.

[0004] In summary, the existing road garbage recognition technology has the following problems:

[0005] 1. The background of road garbage is complex, and the traditional target detection method is difficult to handle such complex feature tasks;

[0006] 2. The proportion of road garbage in the recognition area is small, and multiple different types and shapes of garbage are distributed in the same recognition area;

[0007] 3. Due to the complexity of the road conditions, road garbage may be partially obscured, which can mask the original features of the road garbage, which has a certain adverse effect on feature extraction. SUMMARY

[0008] The purpose of the present application is to solve the problems existing in the prior art mentioned above. Specifically, the CSPDarkNet53 network adopting the channel attention mechanism is used as the backbone network, the decoupling head is used as the detection head, the training sample set is used to optimize the backbone network, the test sample set is used to select the best optimized backbone network as the road garbage recognition network to recognize the road garbage. Compared with the traditional method, the method of the present application can better detect small sample targets and adapt to complex background environments due to the addition of the channel attention mechanism, and the decoupling detection head is used to improve the feature extraction and fusion ability of the road garbage with various categories and different shapes, and improve the recognition rate of the road garbage.

[0009] In order to achieve the above-mentioned target, the present application provides a Yolov3 road garbage detection method based on a decoupling head. The collected road garbage image is improved by the Yolov3 network structure, and the optimal detection effect of the road garbage recognition network is obtained through training, and the garbage recognition network is used to complete the recognition of the road garbage, which specifically includes the following steps:

[0010] Step 1, collection and processing of road garbage images

[0011] Collect D-class road garbage images, wherein D represents the number of categories of road garbage images;

[0012] Select M road garbage images in each category of the D-class road garbage images to obtain MxD road garbage images, then respectively adopt Z image processing methods to complete data enhancement on the MxD road garbage images to obtain ZxMxD road garbage images, and the ZxMxD road garbage images are combined into a training sample data set;

[0013] Select N road garbage images other than the M images in each category of the D-class road garbage images to obtain NxD road garbage images, and the NxD road garbage images are combined into a test sample data set, wherein N≠M;

[0014] Step 2, establish an improved Yolov3 network based on a decoupling detection head and a channel attention mechanism, wherein the improved Yolov3 network includes a backbone network, a Neck structure and a detection head;

[0015] Step 2.1, using CSPDarkNet53 network as the backbone network, and defining the value of the actual network layer number divided by the labeled network layer number 256 as the depth coefficient ζ, the structure of the backbone network comprises: a standard convolution layer α1 connected in series by a convolution layer with a convolution kernel size of 6×6, a batch normalization layer, and a SiLU activation function, the standard convolution layer α1 has 32 input channels; a standard convolution layer α2 connected in series by a convolution layer with a convolution kernel size of 3×3, a batch normalization layer, and a SiLU activation function, the standard convolution layer α2 has 64 input channels; a standard convolution layer α3 connected in series by a convolution layer with a convolution kernel size of 3×3, a batch normalization layer, and a SiLU activation function, the standard convolution layer α3 has 128 input channels; a standard convolution layer α4 connected in series by a convolution layer with a convolution kernel size of 3×3, a batch normalization layer, and a SiLU activation function, the standard convolution layer α4 has 256 input channels; a standard convolution layer α5 connected in series by a convolution layer with a convolution kernel size of 6×6, a batch normalization layer, and a SiLU activation function, the standard convolution layer α5 has 512 input channels; a C3 module layer β1 comprising 3 standard convolution layers α2 and 128ζ bottleneck modules; a C3 module layer β2 comprising 3 standard convolution layers α3 and 256ζ bottleneck modules; a C3 module layer β3 comprising 3 standard convolution layers α4 and 512ζ bottleneck modules; a C3 module layer β4 comprising 3 standard convolution layers α5 and 1024ζ bottleneck modules; a SPPF module layer γ1;

[0016] The input of the backbone network is the standard convolution layer α1, and the output is the SPPF module layer γ1. Specifically, the standard convolution layer α1, the standard convolution layer α2, the C3 module layer β1, the standard convolution layer α3, the C3 module layer β2, the standard convolution layer α4, the C3 module layer β3, the standard convolution layer α5, the C3 module layer β4, and the SPPF module layer γ1 are connected in series.

[0017] Step 2.2, using FPN+PAN network as the Neck structure, the Neck structure comprises: a convolution layer χ1 with a convolution kernel size of 1×1 and a channel number of 512, a convolution layer χ2 with a convolution kernel size of 1×1 and a channel number of 256, a convolution layer χ3 with a convolution kernel size of 3×3 and a channel number of 128, a convolution layer χ4 with a convolution kernel size of 3×3 and a channel number of 256; a down-sampling layer δ with a channel number of 256; four Concat module layers are respectively denoted as Concat module layer C1, Concat module layer C2, Concat module layer C3, and Concat module layer C4; two C3 module layers with a channel number of 512 are respectively denoted as C3 module layer D1 and C3 module layer D2, and two C3 module layers with a channel number of 256 are respectively denoted as D3 and fourth C3 module layer D4; channel attention mechanism CA;

[0018] The input of the neck structure is three, denoted as inputoutput 11 , inputoutput 12 and inputoutput 13 , wherein the inputoutput 11 is connected to the output of the C3 module layer β2 of the backbone network, the inputoutput 12 is connected to the output of the C3 module layer β3 of the backbone network, and the inputoutput 13 is connected to the output of the SPPF module layer γ1 of the backbone network; the output of the neck structure is three, denoted as outputoutput 21 , outputoutput 22 and outputoutput 23 , wherein the outputoutput 21 is the output of the C3 module layer D2, the outputoutput 22 is the output of the C3 module layer D3, and the outputoutput 23 is the output of the channel attention mechanism CA.

[0019] Step 2.3, using the decoupled detection head as the detection head, the structure of the detection head includes: a convolution layer Z1 with a convolution kernel size of 1×1 and a channel number of 256, a convolution layer Z2 with a convolution kernel size of 3×3 and a channel number of 256, a convolution layer Z3 with a convolution kernel size of 3×3 and a channel number of 512, a convolution layer Z4 with a convolution kernel size of 1×1 and a channel number of D, a convolution layer Z5 with a convolution kernel size of 1×1 and a channel number of 4, and a convolution layer Z6 with a convolution kernel size of 1×1 and a channel number of 1;

[0020] The input of the decoupled head is the convolution layer Z1, which is connected to the three outputs outputoutput 21 , outputoutput 22 and outputoutput 23 of the neck structure respectively; the output forms the following three paths: the first path is composed of the convolution layer Z1, the convolution layer Z2, the convolution layer Z3 and the convolution layer Z4 connected in sequence; the second path is composed of the convolution layer Z1, the convolution layer Z2, the convolution layer Z3 and the convolution layer Z5 connected in sequence; and the third path is composed of the convolution layer Z1, the convolution layer Z2, the convolution layer Z3 and the convolution layer Z6 connected in sequence;

[0021] Step 3, training the improved Yolov3 network established in step 2 to obtain a network with optimal detection effect, and taking the network with optimal detection effect as the road garbage recognition network, the specific steps are as follows:

[0022] Step 3.1, uniformly adjusting the pixels of the road garbage images in the training sample set to S×S;

[0023] Step 3.2: Randomly select B images of road litter from the training sample set and form a series Γ, r = (y1, y2, ..., y3). s ..., y B ), where y s Let y be any image of road litter in the series Γ. s , s = 1, 2, ..., B, calculate the image y s The actual class probability tensor Y s Actual category coordinate tensor W s And the actual category IoU tensor X s The actual class probability tensor Y s The dimensions are H×W×C, and the actual class coordinate tensor is W. s The size is H×W×4, and the actual class IoU tensor X s The dimensions are H×W×1; where H represents the height of each tensor, W represents the width of each tensor, and C represents the depth of each tensor.

[0024] Initialize image y s Predicted category probability tensor O s Predicting the category coordinate tensor P s And the predicted category IoU tensor Q s Their values ​​are as follows:

[0025] Define the prediction class tensor O s Predicting the category coordinate tensor P s Predicting category IoU tensor Q s The coordinates are composed of the horizontal coordinate n, the vertical coordinate m, and the depth coordinate γ, and are denoted as (n, m, γ).

[0026] Arbitrarily select the x-coordinate n, y-coordinate m, and depth coordinate γ (where n = 1, 2, ... H, m = 1, 2, ... W, γ = 1, 2, ... C) and let... And O s The predicted probability values ​​for all other coordinates in the tensor are equal to 0, and the predicted category coordinate tensor P... s Arbitrarily select the x-coordinate n, y-coordinate m, and depth coordinate γ (where n = 1, 2, ... H, m = 1, 2, ... W, γ = 1, 2, ... 4) and let... And P s The predicted probability values ​​for all other coordinates are equal to 0, and the predicted class IoU tensor Q... s Arbitrarily select the x-coordinate n, y-coordinate m, and depth coordinate γ (where n = 1, 2, ... H, m = 1, 2, ... W, γ = 1) and let... And Q s The predicted probability values ​​for all other coordinates are equal to 0;

[0027] Step 3.3, input the B selected road garbage images in step 3.2 into the backbone network to update the prediction class probability tensor O of each road garbage image s , the prediction class coordinate tensor P s , and the prediction class IoU tensor Q s , s = s = 1, 2,..., B.

[0028] Step 3.4, optimize the backbone network according to the updated prediction tensors and actual tensors:

[0029] Divide the height of the image y s into H segments and the width into W segments, i.e., divide the image y s into HxW grids.

[0030] Predict each grid on the image y s , and compare the obtained prediction information with the true information to obtain a loss function loss, and obtain the minimum loss function loss by gradient descent method to complete the optimization of the backbone network.

[0031] Step 3.5, repeat steps 3.2 to 3.4 until all road garbage images in the training sample set are selected, wherein if the number of road garbage images left in the sample training set is less than B in the last round of selection, randomly select road garbage images from the already selected road garbage images for supplementation.

[0032] The backbone network optimized through steps 3.2-3.5 is denoted as the backbone network T h of the hth generation, where h is the serial number of the generation.

[0033] Step 3.6, calculate the mean average precision V h of the road garbage images in the test sample set using the backbone network T h of the hth generation, as follows:

[0034] Step 3.6.1, define any one of the D types of road garbage as the ith type of garbage, i = 1, 2,..., D.

[0035] Define the prediction box as the rectangular box marked on the grid, wherein the prediction class probability tensor O s determines the garbage type detected by the rectangular box, the prediction class coordinate tensor P s determines the center coordinates of the rectangular box, and the prediction class IoU tensor Q s determines the confidence of the rectangular box; define the actual box as the rectangular box manually labeled on the road garbage image; define the overlap I as the value of the area intersection of the prediction box and the actual box divided by the area union.

[0036] Step 3.6.2, randomly take n unequal decimals to form an overlapping threshold sequence K, K = {K i1 , K i2 ,... K ij ,..K in}, wherein K ij is the jth overlapping threshold corresponding to the ith type of garbage, j = 1, 2,..., n;

[0037] Define TP as the number of predicted boxes in the ith type of garbage whose overlapping degree I is greater than or equal to the jth overlapping threshold K ij , FP as the number of predicted boxes in the ith type of garbage whose overlapping degree I is less than the jth overlapping threshold K ij , and FN as the number of actual boxes for which no corresponding predicted box is given, calculate the recall rate R ij and the precision P ij of the ith type of garbage in the jth overlapping threshold K ij , whose calculation formulas are as follows:

[0038]

[0039]

[0040] Step 3.6.3, calculate the recall rate and the precision of all overlapping thresholds in the overlapping threshold sequence K in the ith type of garbage according to the method of step 3.6.2, obtain n recall rates R ij and n precisions P ij , and draw a curve in the plane coordinate system with the recall rate R ij as the horizontal coordinate and the precision P ij as the vertical coordinate, and mark it as the P i -R i curve;

[0041] Take the P i -R i curve, the horizontal coordinate, the vertical coordinate, and the line connecting the end point of the P i -R i curve and the horizontal coordinate as the contour line, calculate the area in the contour line and mark it as the AP value F i of the ith type of garbage;

[0042] Step 3.6.4, calculate the AP value of each type of road garbage in the D type of road garbage according to the method of step 3.6.2-step 3.6.3, obtain D AP values F i , and calculate the main network T h of the hth generation according to the D AP values F i , and the average precision V of the test sample set of road garbage imagesh ,

[0043]

[0044] Step 3.7, assuming the number of repetitions is G, G times repeating step 3.4 and step 3.6, obtaining a network set T and a mean average precision set V, T = {T1, T2,..., T h ,..., T G}, V = {V1, V2,..., V h ,..., V G};

[0045] Let V o be the highest mean average precision, V o = max{V1, V2,..., V h ,..., V G ), V o The corresponding backbone network T o is the network with the best recognition effect, and is recorded as the road garbage recognition network;

[0046] Step 4, using the road garbage recognition network to recognize road garbage.

[0047] Preferably, the C3 module layer β1, the C3 module layer β2, the C3 module layer β3, and the C3 module layer β4 are each composed of a three-layer structure connected in series along the input-output direction of the backbone network, specifically as follows:

[0048] The 128ζ, 256ζ, 512ζ, and 1024ζ are collectively referred to as n×ζ, and n is equal to 128 or equal to 256 or equal to 512 or equal to 1024. The standard convolution layer α2, the standard convolution layer α3, the standard convolution layer α4, and the standard convolution layer α5 are collectively referred to as the standard convolution layer α.

[0049] The first layer structure of the above four C3 module layers is composed of parallel passage Z1 and passage Z2, wherein the passage Z1 is composed of a standard convolution layer α and n×ζ bottleneck modules connected in series, and the passage Z2 is composed of a standard convolution layer α. The second layer structure is a Concat module layer, the input is the passage Z1 and the passage Z2, and the output is connected in series to the third layer structure; the third layer structure is a standard convolution layer α.

[0050] Preferably, the Neck structure is composed of 8 layers connected in series along the input-output direction of the backbone network:

[0051] The first layer structure is respectively connected to the input output 11 , the input output 12For input, including and parallel passage b1 and passage b2, wherein, passage b1 is composed of input output 11 , Concat module layer C1, convolution layer χ1, down-sampling layer δ in turn, passage b2 is connected by input output 12 and Concat module layer C1; the second layer structure is composed of Concat module layer C1; the third layer structure is respectively taken as the output of Concat module layer C1, input output 13 As input, including and parallel passage b3 and passage b4, wherein, passage b3 is composed of the output of Concat module layer C1, Concat module layer C2, C3 module layer D1, convolution layer χ2, down-sampling layer δ in turn, passage b4 is connected by input output 13 And Concat module layer C2; the fourth layer structure is composed of Concat module layer C2; the fifth layer structure is respectively taken as the output of Concat module layer C2, the output of convolution layer χ2 as input, including and parallel passage b5 and passage b6, wherein, passage b5 is composed of the output of Concat module layer C2, Concat module layer C3, C3 module layer D2, convolution layer χ3 in turn, passage b6 is connected by the output of convolution layer χ2 and Concat module layer C3; the sixth layer structure is Concat module layer C3; the seventh layer structure is respectively taken as the output of Concat module layer D3, the output of convolution layer χ1 as input, including two parallel passages b7 and b8, passage b7 is composed of the output along Concat module layer, C3 module layer D2, convolution layer χ4 in turn, passage b8 is connected by the output of convolution layer χ1 and Concat module layer C4; the eighth layer structure is composed of Concat module layer C4, C3 module layer D4, channel attention mechanism CA in turn.

[0052] Preferably, the implementation steps of the channel attention mechanism CA are as follows:

[0053] The channel attention mechanism CA includes average pooling layer η X , average pooling layer η Y , Concat module layer L, convolution layer λ, batch normalization layer A and Sigmoid nonlinear activation layer J;

[0054] The channel attention mechanism CA is composed of 3 layer structures in turn along the input-output direction of the backbone network: the first layer structure is composed of two parallel passages D1 and D2, passage D1 is composed of average pooling layer η X , passage D2 is composed of average pooling layer η YThe two paths access a Concat module layer L, the second layer structure is composed of the Concat module layer L and a batch normalization layer A connected in series along the direction of the two parallel paths, and the third layer structure branches from the output of the batch normalization layer A into two parallel paths D3 and D4, both of which are composed of a convolutional layer λ and a Sigmoid nonlinear activation layer J connected in series.

[0055] The input of the channel attention mechanism CA is the output of the fourth C3 module layer D4, and the outputs of the paths D3 and D4 are multiplied by the input of the channel attention mechanism CA to obtain the output of the channel attention mechanism CA.

[0056] Preferably, the specific steps of identifying road litter by using the road litter identification network are as follows:

[0057] Step 4.1, input an original road litter image to be identified, and perform the following image processing: adjust the pixels of the original road litter image to be identified to E×E;

[0058] The original road litter image to be identified after the foregoing channel processing is denoted as image Y d ;

[0059] Step 4.2, divide the image Y d into grids and denote any one grid as grid K d v , where v is the serial number of the grid, v = 1, 2,..., Λ, and Λ is the number of grids;

[0060] Step 4.3, input the image Y d into the road litter identification network to obtain the predicted class probability tensor, the predicted class coordinate tensor and the predicted class IoU tensor of each prediction box in the grid K d v , define any one prediction box as prediction box R d vu , u = 1, 2,..., U, and U is the number of prediction boxes in the grid K d v , then obtain the predicted probability value O d vu , the predicted probability value P d vu , and the predicted probability value Q d vu of the predicted class probability tensor corresponding to the prediction box R d vu , and the predicted probability value Q d vu of the predicted class IoU tensor, to obtain the prediction box Rd vu The confidence of the prediction box is recorded as confidence L d vu ;

[0061] The confidence L d v is compared with a given confidence threshold L 0 , and the following judgment is made:

[0062] If L d vu ≥ L 0 , the prediction box is retained;

[0063] If L d vu < L 0 , the prediction box is discarded;

[0064] Step 4.4, repeat step 4.3 until all the grids of the image Y d are selected, and then mark the prediction boxes on the image Y d according to the results of the prediction boxes in each grid of the image to be recognized, and obtain the recognized road garbage image.

[0065] Compared with the prior art, the beneficial effects of the present application are as follows:

[0066] 1. The channel attention mechanism in the present application adopts an effective data compression method, which can selectively extract the required features in a complex recognition background.

[0067] 2. The present application adopts a positioning and classification feature separation recognition method, which separates the processing process of the geographical position positioning feature of the road garbage and the road garbage category recognition feature, improves the utilization ability of various complex features, and has good effect on small targets and multi-form targets.

[0068] 3. The CSPDarkNet53 network based on the channel attention mechanism proposed in the present application has strong recognition ability when recognizing multiple targets and occluded targets, and is more suitable for the needs of actual life. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 is the flowchart of the road garbage recognition method of the present application;

[0070] Figure 2 is a schematic block diagram of the road garbage recognition method of the present application;

[0071] Figure 3 is a schematic diagram of the implementation of the head in the embodiment of the present application;

[0072] Figure 4is the mAP comparison chart of the road recognition method and other methods in the embodiment of the application. Specific implementation method

[0073] The technical solutions of the application will be described clearly and completely in the following with reference to the drawings and specific embodiments.

[0074] Figure 1 is a flowchart of the road garbage recognition method of the application, Figure 2 is a schematic block diagram of the road garbage recognition method of the application, which comprises Figure 1 and Figure 2 It can be seen that the application provides a Yolov3 road garbage detection method based on decoupled head, which collects road garbage images, improves the Yolov3 network structure, trains the road garbage recognition network with the optimal detection effect, and uses the garbage recognition network to complete the recognition of road garbage, and specifically comprises the following steps:

[0075] Step 1, collection and processing of road garbage images

[0076] Collecting D-class road garbage images, wherein D represents the number of categories of road garbage images;

[0077] Selecting M road garbage images in each of the D-class road garbage images to obtain MxD road garbage images, then respectively adopting Z image processing methods to complete data enhancement on the MxD road garbage images to obtain ZxMxD road garbage images, and grouping the ZxMxD road garbage images into a training sample data set;

[0078] Selecting N road garbage images other than the M images in each of the D-class road garbage images to obtain NxD road garbage images, and grouping the NxD road garbage images into a test sample data set, wherein N≠M.

[0079] In this embodiment, D=14, M=70, Z=8, and N=30.

[0080] In this embodiment, z=8, and specifically includes the following 8 image processing methods: random cropping, random translation, changing brightness, adding Gaussian random noise, random rotation, random flipping, random occlusion processing, and mosaic data enhancement.

[0081] Step 2, establishing an improved Yolov3 network based on decoupled detection head and channel attention mechanism, wherein the improved Yolov3 network comprises a backbone network, a Neck structure, and a detection head.

[0082] Step 2.1, using CSPDarkNet53 network as the backbone network, and defining the value of the actual network layer number divided by the labeled network layer number 256 as the depth coefficient ζ, the structure of the backbone network comprises: a standard convolutional layer α1 connected in series in turn by a convolutional layer with a convolution kernel size of 6x6, a batch normalization layer, and a SiLU activation function, the standard convolutional layer α1 has 32 input channels; a standard convolutional layer α2 connected in series in turn by a convolutional layer with a convolution kernel size of 3x3, a batch normalization layer, and a SiLU activation function, the standard convolutional layer α2 has 64 input channels; a standard convolutional layer α3 connected in series in turn by a convolutional layer with a convolution kernel size of 3x3, a batch normalization layer, and a SiLU activation function, the standard convolutional layer α3 has 128 input channels; a standard convolutional layer α4 connected in series in turn by a convolutional layer with a convolution kernel size of 3x3, a batch normalization layer, and a SiLU activation function, the standard convolutional layer α4 has 256 input channels; a standard convolutional layer α5 connected in series in turn by a convolutional layer with a convolution kernel size of 6x6, a batch normalization layer, and a SiLU activation function, the standard convolutional layer α5 has 512 input channels; a C3 module layer β1 comprising 3 standard convolutional layers α2 and 128ζ bottleneck modules; a C3 module layer β2 comprising 3 standard convolutional layers α3 and 256ζ bottleneck modules; a C3 module layer β3 comprising 3 standard convolutional layers α4 and 512ζ bottleneck modules; a C3 module layer β4 comprising 3 standard convolutional layers α5 and 1024ζ bottleneck modules; a SPPF module layer γ1.

[0083] The input of the backbone network is the standard convolutional layer α1, and the output is the SPPF module layer γ1. Specifically, the standard convolutional layer α1, the standard convolutional layer α2, the C3 module layer β1, the standard convolutional layer α3, the C3 module layer β2, the standard convolutional layer α4, the C3 module layer β3, the standard convolutional layer α5, the C3 module layer β4, and the SPPF module layer γ1 are connected in series.

[0084] In this embodiment, the C3 module layer β1, the C3 module layer β2, the C3 module layer β3, and the C3 module layer β4 are all composed of a three-layer structure connected in series along the input-output direction of the backbone network, specifically as follows:

[0085] The 128ζ, 256ζ, 512ζ, and 1024ζ are collectively referred to as n x ζ, and n is equal to 128 or equal to 256 or equal to 512 or equal to 1024, and the standard convolutional layer α2, the standard convolutional layer α3, the standard convolutional layer α4, and the standard convolutional layer α5 are collectively referred to as a standard convolutional layer α.

[0086] The first layer structure of the four C3 module layers is composed of parallel passage Z1 and passage Z2, wherein passage Z1 is composed of a standard convolution layer α and n×ζ bottleneck modules connected in series, and passage Z2 is composed of a standard convolution layer α, the second layer structure is a Concat module layer, the input is passage Z1 and passage Z2, and the output is connected in series to the third layer structure; the third layer structure is a standard convolution layer α.

[0087] In step 2.2, the FPN+PAN network is used as the Neck structure, and the Neck structure comprises: a convolution layer χ1 with a convolution kernel size of 1×1 and a channel number of 512, a convolution layer χ2 with a convolution kernel size of 1×1 and a channel number of 256, a convolution layer χ3 with a convolution kernel size of 3×3 and a channel number of 128, a convolution layer χ4 with a convolution kernel size of 3×3 and a channel number of 256, a down-sampling layer δ with a channel number of 256, four Concat module layers respectively denoted as Concat module layer C1, Concat module layer C2, Concat module layer C3 and Concat module layer C4, two C3 module layers with a channel number of 512 respectively denoted as C3 module layer D1 and C3 module layer D2, two C3 module layers with a channel number of 256 respectively denoted as D3 and fourth C3 module layer D4, and a channel attention mechanism CA.

[0088] The input of the Neck structure is three, respectively denoted as input output 11 , input output 12 and input output 13 , wherein the input output 11 is the output of the C3 module layer β2 of the backbone network, the input output 12 is the output of the C3 module layer β3 of the backbone network, and the input output 13 is the output of the SPPF module layer γ1 of the backbone network; the output of the Neck structure is three, respectively denoted as output output 21 , output output 22 and output output 23 , wherein the output output 21 is the output of the C3 module layer D2, the output output 22 is the output of the C3 module layer D3, and the output output 23 is the output of the channel attention mechanism CA.

[0089] In the embodiment, the Neck structure is composed of eight layer structures connected in series along the input-output direction of the backbone network.

[0090] The first layer structure is respectively input output 11 , input output12 For input, including parallel path b1 and path b2, wherein path b1 is composed of input output 11 , Concat module layer C1, convolution layer χ1, down-sampling layer δ in turn, path b2 is composed of input output 12 and Concat module layer C1 connection; the second layer structure is composed of Concat module layer C1; the third layer structure is respectively with Concat module layer C1 output, input output 13 As input, including parallel path b3 and path b4, wherein path b3 is composed of Concat module layer C1 output, Concat module layer C2, C3 module layer D1, convolution layer χ2, down-sampling layer δ in turn, path b4 is composed of input output 13 And Concat module layer C2 connection; the fourth layer structure is composed of Concat module layer C2; the fifth layer structure is respectively with Concat module layer C2 output, convolution layer χ2 output as input, including parallel path b5 and path b6, wherein path b5 is composed of Concat module layer C2 output, Concat module layer C3, C3 module layer D2, convolution layer χ3 in turn, path b6 is composed of convolution layer χ2 output and Concat module layer C3 direct connection; the sixth layer structure is Concat module layer C3; the seventh layer structure is respectively with Concat module layer C3 output, convolution layer χ1 output as input, including two parallel path b7 and path b8, path b7 is composed of along Concat module layer output, C3 module layer D3, convolution layer χ4 in turn, path b8 is composed of convolution layer χ1 output and Concat module layer C4 direct connection; the eighth layer structure is composed of Concat module layer C4, C3 module layer D4, channel attention mechanism CA in turn.

[0091] In the embodiment, the implementation steps of the channel attention mechanism CA are as follows:

[0092] The channel attention mechanism CA includes average pooling layer η X , average pooling layer η Y , Concat module layer L, convolution layer λ, batch normalization layer A and Sigmoid nonlinear activation layer J;

[0093] The channel attention mechanism CA is composed of three layer structures in turn along the input-output direction of the backbone network: the first layer structure is composed of two parallel paths D1 and path D2, path D1 is composed of average pooling layer η X , path D2 is composed of average pooling layer η YThe two paths access a Concat module layer L, the second layer structure is composed of the Concat module layer and a batch normalization layer A connected in series along the direction of the two parallel paths, and the output direction of the backbone network, the third layer structure is branched from the output of the batch normalization layer A into two parallel paths D3 and D4, and the paths D3 and D4 are both composed of a convolution layer λ and a Sigmoid nonlinear activation layer J connected in series.

[0094] The input of the channel attention mechanism CA is the output of the fourth C3 module layer D4, the outputs of the paths D3 and D4 are multiplied with the input of the channel attention mechanism CA respectively, and the output of the channel attention mechanism CA is obtained.

[0095] Step 2.3, a decoupling detection head is used as a detection head, and the structure of the detection head comprises a convolution layer Z1 with a convolution kernel size of 1*1 and a channel number of 256, a convolution layer Z2 with a convolution kernel size of 3*3 and a channel number of 256, a convolution layer Z3 with a convolution kernel size of 3*3 and a channel number of 512, a convolution layer Z4 with a convolution kernel size of 1*1 and a channel number of D, a convolution layer Z5 with a convolution kernel size of 1*1 and a channel number of 4, and a convolution layer Z6 with a convolution kernel size of 1*1 and a channel number of 1.

[0096] The input of the decoupling head is the convolution layer Z1, the convolution layer Z1 is connected with the three outputs output 21 , output 22 , and output 23 of the Neck structure respectively; and the output forms the following three paths: the first path is composed of the convolution layer Z1, the convolution layer Z2, the convolution layer Z3, and the convolution layer Z4 connected in series; the second path is composed of the convolution layer Z1, the convolution layer Z2, the convolution layer Z3, and the convolution layer Z5 connected in series; and the third path is composed of the convolution layer Z1, the convolution layer Z2, the convolution layer Z3, and the convolution layer Z6 connected in series.

[0097] Figure 3 It is a step diagram for implementing the detection head in the embodiment of the application.

[0098] Step 3, the improved Yolov3 network established in step 2 is trained to obtain a network with optimal detection effect, and the network with optimal detection effect is taken as a road garbage recognition network, and the specific steps are as follows:

[0099] Step 3.1, the pixels of the road garbage images in the training sample set are uniformly adjusted to S*S.

[0100] Step 3.2, B road garbage images in the training sample set are randomly selected and combined into a series Γ, Γ=(y1, y2,..., y s ..., y B ), wherein ys For any one road litter image in the series Γ, denoted as image y s , s = 1, 2, …, B, calculate the actual class probability tensor Y s , the actual class coordinate tensor W s and the actual class IoU tensor X s of the image y s , wherein the size of the actual class probability tensor Y s is H x W x C, the size of the actual class coordinate tensor W s is H x W x 4, and the size of the actual class IoU tensor X s is H x W x 1; wherein H represents the height of each tensor, W represents the width of each tensor, and C represents the depth of each tensor.

[0101] Initialize the predicted class probability tensor O s , the predicted class coordinate tensor P s and the predicted class IoU tensor Q s of the image y s , which take values as follows:

[0102] Define the coordinates of the predicted class tensor O s , the predicted class coordinate tensor P s and the predicted class IoU tensor Q s , which are composed of the horizontal coordinate n, the vertical coordinate m and the depth coordinate γ, and denoted as (n, m, γ);

[0103] Randomly select the horizontal coordinate n, the vertical coordinate m and the depth coordinate γ (wherein n = 1, 2, …, H, m = 1, 2, …, W, and γ = 1, 2, …, C) and let and the predicted probability value of other coordinates in O s is equal to 0, for the predicted class coordinate tensor P s , randomly select the horizontal coordinate n, the vertical coordinate m and the depth coordinate γ (wherein n = 1, 2, …, H, m = 1, 2, …, W, and γ = 1, 2, …, 4) and let and the predicted probability value of other coordinates in P s is equal to 0, for the predicted class IoU tensor Q s , randomly select the horizontal coordinate n, the vertical coordinate m and the depth coordinate γ (wherein n = 1, 2, …, H, m = 1, 2, …, W, and γ = 1) and let and the predicted probability value of other coordinates in Q s is equal to 0.

[0104] Step 3.3, input the B road litter images selected in step 3.2 into the backbone network to update the predicted class probability tensor O s , the predicted class coordinate tensor Ps and prediction class IoU tensor Q s s = s = 1, 2, …, B.

[0105] Step 3.4, optimize the backbone network according to the updated individual prediction tensor and the actual tensor:

[0106] Divide the height of the image y s into H segments and the width into W segments, that is, divide the image y s into HxW grids.

[0107] Predict each grid on the image y s , and compare the obtained prediction information with the true information to obtain a loss function loss, and obtain a minimized loss function min-loss by gradient descent method to complete the optimization of the backbone network.

[0108] In this embodiment, the expression of the loss function loss is as follows:

[0109] loss = box_gain x bbox_loss + cls_gain x cls_loss + obj_gain x obj_loss

[0110] Wherein, bbox_loss is the rectangle box loss, cls_loss is the classification loss, obj_loss is the confidence loss, box_gain is the rectangle box loss weight, cls_gain is the classification loss weight, and obj_gain is the confidence loss.

[0111] The default box_gain = 0.05, cls_gain = 0.5, and obj_gain = 1.0. The expressions of the rectangle box loss bbox_loss, the classification loss cls_loss, and the confidence loss obj_loss are as follows:

[0112]

[0113]

[0114]

[0115] Wherein ||·||2 represents the Euclidean norm.

[0116] By gradient descent method, the minimized loss function min-loss is obtained, and the optimization of the backbone network is completed.

[0117] Step 3.5, repeat step 3.2 to step 3.4 until all road garbage images in the training sample set are selected, wherein if the number of road garbage images left in the sample training set is less than B in the last round of selection, randomly select road garbage images from the road garbage images that have been selected to supplement;

[0118] The optimized backbone network completed through step 3.2-step 3.5 is denoted as the backbone network T of the hth generation h , wherein h is the serial number of the generation.

[0119] Step 3.6, calculate the backbone network T of the hth generation using the test sample set h The average precision V of the road garbage images in the test sample set h , specifically as follows:

[0120] Step 3.6.1, define any one of the D types of road garbage as the ith type of garbage, i = 1, 2, …, D;

[0121] Define the predicted box as the rectangular box marked on the grid, wherein the predicted class probability tensor O s Determine the garbage class detected by the rectangular box, the predicted class coordinate tensor P s Determine the center coordinates of the rectangular box, the predicted class IoU tensor Q s Determine the confidence of the rectangular box; define the actual box as the rectangular box manually labeled on the road garbage image; define the overlap I as the value of the area intersection of the predicted box and the actual box divided by the area union;

[0122] Step 3.6.2, randomly take n unequal decimals between 0 and 1 to form an overlap threshold sequence K, K = {K i1 , K i2 , … K ij ,.. K in}, wherein K ij is the jth overlap threshold value corresponding to the ith type of garbage, j = 1, 2, …, n;

[0123] Define TP as the number of predicted boxes with overlap I ≥ the jth overlap threshold value K ij in the ith type of garbage, FP as the number of predicted boxes with overlap I < the jth overlap threshold value K ij in the ith type of garbage, FN as the number of actual boxes without corresponding predicted boxes, calculate the recall R ij and the precision P ij of the ith type of garbage in the jth overlap threshold value K ij , whose calculation formula is as follows:

[0124]

[0125]

[0126] Step 3.6.3: Following the method in Step 3.6.2, calculate the recall and precision of all overlap thresholds in the overlap threshold sequence K for the i-th type of garbage, obtaining n recall rates R. ij And accuracy P ij Following the order from 1 to n, with recall rate R ij x-axis, precision P ij Using P as the ordinate, plot a curve in a plane coordinate system and denote it as P. i -R i curve;

[0127] With P i -R i Curve, x-axis, y-axis, P i -R i The line connecting the endpoint of the curve to the x-axis is the contour line. The area within the contour line is calculated and recorded as the AP value F of the i-th type of waste. i ;

[0128] Step 3.6.4: Calculate the AP value for each type of road waste in category D using the methods described in steps 3.6.2-3.6.3, resulting in D AP values ​​F. i And based on D AP values ​​F i Calculate the backbone network T of generation h. h The mean accuracy V of road litter images in the test sample set h ,

[0129]

[0130] Step 3.7, let the number of repetitions be G. Repeating steps 3.4-3.6 G times yields a network set T and a set of mean and average precision V, T = {T1, T2, ..., T...} h , ..., T G}, V = {V1, V2, ..., V} h , ..., V G};

[0131] Remember V o For the highest mean averaging accuracy, V o =max{V1, V2, ..., V h , ..., V G}, V o The corresponding backbone network T o This is the network with the best recognition effect, and it is denoted as the road litter recognition network;

[0132] In this embodiment, B = 16, S = 640, and G = 51.

[0133] Step 4: Use the road waste identification network to identify road waste.

[0134] In this embodiment, the specific steps for identifying road waste using a road waste identification network are as follows:

[0135] Step 4.1: Input a raw image of road litter to be identified, and perform the following image processing: Adjust the pixel size of this raw image of road litter to be identified to E×E:

[0136] The original road debris image to be identified, processed as described above, is denoted as image Y. d ;

[0137] Step 4.2, in image Y d Divide the grid into equal parts and denote any one of the grids as grid K. d v , where v is the grid number, v = 1, 2, ..., Λ, and Λ is the number of grids;

[0138] Step 4.3, transfer image Y d The data is fed into the road waste identification network to obtain grid K. d v For each predicted bounding box, there are predicted class probability tensors, predicted class coordinate tensors, and predicted class IoU tensors. Any one of these predicted bounding boxes is defined as the predicted bounding box R. d vu u = 1, 2, ..., U, where U is the grid K d v The number of predicted boxes in the predicted box is then used to obtain the predicted box R. d vu The predicted probability value O of the corresponding predicted class probability tensor d vu The predicted probability value P of the predicted class coordinate tensor d vu and the predicted probability value Q of the predicted category IoU tensor d vu The probability value Q is predicted by predicting the class IoU tensor. d vu Obtain the prediction box R d vu The confidence level is denoted as confidence level L. d vu ;

[0139] Confidence level L d v With a given confidence threshold L 0 Compare the results and make the following judgments:

[0140] If L d vu ≥ L 0 , the prediction box is reserved;

[0141] If Ld vu < L 0 , the prediction box is discarded;

[0142] Step 4.4, repeat step 4.3 until the image Y d All grids are selected, and then the prediction box is marked on the image Y d according to the result of the prediction box in each grid of the image to be identified, to obtain the identified road garbage image.

[0143] In this embodiment, U≥3.

[0144] So far, the detection of the road garbage to be identified is completed.

[0145] In the above detection process, the specific settings of the bottleneck module, the SiLU activation function, the SPPF module, the Concat module layer and the Sigmoid nonlinear activation layer J described in step 2 are as follows.

[0146] The specific structure of the bottleneck module is: the input image is divided into two paths, one path is connected in series by a convolution layer with a convolution kernel size of 1x1, a convolution layer with a convolution kernel size of 3x3, and the other path retains the original input image, then the two input images are directly added to obtain a new image output.

[0147] The expression of the SiLU activation function is:

[0148]

[0149] Where e is the exponential function, ω is the input of the SiLU activation function, and Y(ω) is the output of the SiLU activation function.

[0150] The SPPF module is composed of four layers in series along the input-output direction of the backbone network: the first layer is a convolution layer with a convolution kernel size of 1*1 and a channel number of 512; the second layer is four parallel paths output from the convolution layer in the first layer, respectively denoted as path K1, path K2, path K3 and path K4, path K1 is composed of three pooling layers connected in series along the input-output direction of the backbone network, path K2 is composed of two pooling layers connected in series along the input-output direction of the backbone network, path K3 is composed of one pooling layer connected in series along the input-output direction of the backbone network, and path K4 is directly output from the first layer, wherein the pooling layers are all maximum pooling down-sampling layers with a convolution kernel size of 5*5; the third layer is a Concat module layer, which inputs the four paths of the second layer and outputs a serial connection to the fourth layer, and the fourth layer is a convolution layer with a convolution kernel size of 1*1 and a channel number of 512.

[0151] The Concat module layer structure is as follows: the two input channel tensors are connected in front and back, and a connected tensor copy is returned.

[0152] The Sigmoid nonlinear activation layer J contains a Sigmoid function, and the expression of the Sigmoid function is as follows:

[0153]

[0154] Wherein, ω1 is the input of the Sigmoid function, and Y1(ω1) is the output of the Sigmoid function.

[0155] Figure 4 The average precision V h of the road garbage detection method of the present application and other recognition methods is compared, and other recognition methods include Yolov3 algorithm without using any improvement, Yolov3 algorithm using ECA channel attention mechanism, and Yolov3 algorithm using ASFF detection head. From Figure 4 It can be seen that the average precision V h of the method of the present application is better than that of other recognition algorithms, which shows the superiority of the recognition method proposed in the present application in the field of road garbage recognition.

[0156] In summary, the method of the present application improves the recognition accuracy by adding an attention mechanism and a new detection head to the Yolov3 network, greatly improves the recognition ability of road garbage, and can well meet the application requirements of complex road environment, and better serve the environmental protection cause.

Claims

1. A Yolov3 road garbage detection method based on decoupled head, characterized in that, The collected road garbage image is processed through the improved Yolov3 network structure, and the optimal detection effect of the road garbage recognition network is obtained through training, and the garbage recognition network is used to complete the recognition of the road garbage, which specifically includes the following steps: Step 1, collection and processing of road garbage image Collecting road-like trash images, where D denotes the number of categories of road-like trash images; In selecting Z road garbage images in each class of the road garbage image, obtaining Z road garbage images, and then respectively adopting Z road garbage images, respectively adopting a kind of image processing mode to complete data enhancement, obtaining Z×M×D road garbage images, and the Z×M×D road garbage images are composed into a training sample data set; In reselecting outside of the obtained road junk images, the a test sample data set, wherein ; Step 2, establish an improved Yolov3 network based on decoupling detection head and channel attention mechanism, the improved Yolov3 network includes a backbone network, a Neck structure and a detection head; Step 2.1, using the CSPDarkNet53 network as the backbone network, and defining the value of the actual network layer number divided by the labeled network layer number 256 as the depth coefficient ζ, the structure of the backbone network comprises: a standard convolutional layer sequentially connected by a convolutional layer with a convolution kernel size of , a batch normalization layer, and a SiLU activation function , the standard convolutional layer has 32 input channels; a standard convolutional layer sequentially connected by a convolutional layer with a convolution kernel size of , a batch normalization layer, and a SiLU activation function , the standard convolutional layer has 64 input channels; a standard convolutional layer sequentially connected by a convolutional layer with a convolution kernel size of , a batch normalization layer, and a SiLU activation function , the standard convolutional layer has 128 input channels; a standard convolutional layer sequentially connected by a convolutional layer with a convolution kernel size of , a batch normalization layer, and a SiLU activation function , the standard convolutional layer has 256 input channels; a standard convolutional layer sequentially connected by a convolutional layer with a convolution kernel size of , a batch normalization layer, and a SiLU activation function , the standard convolutional layer has 512 input channels; a C3 module layer containing 3 standard convolutional layers , 128ζ bottleneck modules ; a C3 module layer containing 3 standard convolutional layers , 256ζ bottleneck modules ; a C3 module layer containing 3 standard convolutional layers , 512ζ bottleneck modules ; a C3 module layer containing 3 standard convolutional layers , 1024ζ bottleneck modules ; a SPPF module layer ; The input of the backbone network is a standard convolutional layer , and the output is an SPPF module layer , specifically, a standard convolutional layer , a standard convolutional layer , a C3 module layer , a standard convolutional layer , a C3 module layer , a standard convolutional layer , a C3 module layer , a standard convolutional layer , a C3 module layer , and an SPPF module layer are sequentially connected in series. Step 2.2, using FPN+PAN network as the Neck structure, the Neck structure comprising: a convolutional layer with a convolution kernel size of a channel number of 512 , a convolutional layer with a convolution kernel size of a channel number of 256 , a convolutional layer with a convolution kernel size of a channel number of 128 , a convolutional layer with a convolution kernel size of a channel number of 256 , a down-sampling layer with a channel number of 256 , four Concat module layers respectively denoted as Concat module layer , Concat module layer , Concat module layer and Concat module layer , two C3 module layers with a channel number of 512 respectively denoted as C3 module layer and C3 module layer , two C3 module layers with a channel number of 256 respectively denoted as and fourth C3 module layer , a channel attention mechanism CA; The Neck structure has three inputs, denoted as inputs. ,enter and input , where, input C3 module layer connected to the backbone network The output, input C3 module layer connected to the backbone network The output, input SPPF module layer connected to the backbone network The output of the Neck structure is three, denoted as outputs. Output and output , where the output For C3 module layer The output, the output For C3 module layer The output, the output This is the output of the channel attention mechanism CA; Step 2.3, using the decoupled detection head as the detection head, the structure of the detection head includes: the convolution kernel size is The number of channels of the convolution layer is 256 The number of channels of the convolution layer is 256 The number of channels of the convolution layer is 256 The number of channels of the convolution layer is 256 The number of channels of the convolution layer is 512 The number of channels of the convolution layer is 512 The number of channels of the convolution layer is The number of channels of the convolution layer is The number of channels of the convolution layer is The number of channels of the convolution layer is 4 The number of channels of the convolution layer is 4 The number of channels of the convolution layer is 1 The number of channels of the convolution layer is 1 The input of the decoupling head is a convolutional layer , a convolutional layer , a convolutional layer , a convolutional layer , a convolutional layer , a convolutional layer , a convolutional layer , a convolutional layer , and a convolutional layer are connected in turn to form the following three paths: the first path is composed of a convolutional layer , a convolutional layer , a convolutional layer , and a convolutional layer ; the second path is composed of a convolutional layer , a convolutional layer , a convolutional layer , and a convolutional layer ; and the third path is composed of a convolutional layer Step 3, train the improved Yolov3 network established in step 2 to obtain a network with optimal detection effect, and use the network with optimal detection effect as the road garbage recognition network, the specific steps are as follows: Step 3.1, uniformly adjust the pixels of the road garbage images in the training sample set to ; Step 3.2, randomly select a training sample set in Zhang Road Garbage Image, and form a series of Г, Г= Where, Any one of the road garbage images in Г, denoted as image , s = 1, 2,..., B, calculate the actual class probability tensor of image , actual class coordinate tensor and actual class IoU tensor , where the size of the actual class probability tensor is , the size of the actual class coordinate tensor is , and the size of the actual class IoU tensor is ; wherein represents the height of each tensor, represents the width of each tensor, represents the depth of each tensor;​ Initialize the image the predicted class probability tensor , the predicted class coordinate tensor and the predicted class IoU tensor whose initialization procedure is: a predicted class tensor , a predicted class coordinate tensor , a predicted class IoU tensor The coordinates are composed of a horizontal coordinate , a vertical coordinate , and a depth coordinate , and are denoted as ; arbitrarily select the horizontal coordinate , the vertical coordinate , the depth coordinate (wherein , , ) and let , and the prediction probability values of other coordinates in arbitrarily select the horizontal coordinate , the vertical coordinate , the depth coordinate (wherein , , ) and let , and the prediction probability values of other coordinates in arbitrarily select the horizontal coordinate , the vertical coordinate , the depth coordinate (wherein , , ) and let , and the prediction probability values of other coordinates in Step 3.3, the selected prediction class probability tensor of each road garbage image is updated by inputting the road garbage image into the backbone network , the prediction class coordinate tensor , and the prediction class IoU tensor of each road garbage image are updated by inputting the road garbage image into the backbone network , s = s = 1, 2, …, B; Step 3.4, optimize the backbone network according to the updated each prediction tensor and the actual tensor: dividing the image into H segments in height and W segments in width, i.e. dividing the image into HxW grids; Prediction is made on each grid on the image , and the obtained prediction information is compared with the real information to obtain a loss function , and the loss function is minimized by gradient descent method , and the optimization of the backbone network is completed; Step 3.5, repeat steps 3.2 to 3.4 until all road garbage images in the training sample set are selected, wherein if the number of road garbage images left in the sample training set is less than B in the last round of selection, randomly select road garbage images from the road garbage images that have been selected to supplement; The optimized backbone network completed by step 3.2-step 3.5 is denoted as the backbone network of the hth generation wherein h is the serial number of the generation; Step 3.

6. Calculate the first Generation backbone network Mean average precision on road trash images in the test sample set , as follows: Step 3.6.1, Definition Any one of the road litter is the ith road litter, i = 1, 2,..., D; define a prediction bounding box as a rectangular box marked on the grid, wherein the prediction class probability tensor determine the garbage class detected by the rectangular box, the prediction class coordinate tensor determine the center coordinates of the rectangular box, the prediction class IoU tensor determine the confidence of the rectangular box; define an actual bounding box as a rectangular box manually marked on a road garbage image; define the overlap I as the value of the area intersection of the prediction bounding box and the actual bounding box divided by the area union; Step 3.6.2, randomly taking unequal decimals to constitute an overlap threshold sequence , K = {K i1 , K i2 ,... K ij ,... K in}, wherein K ij is the jth overlap threshold corresponding to the ith type of garbage, j = 1, 2,..., n; Define TP as the overlap degree I ≥ j-th overlap degree threshold K in the i-th type of waste. ij The number of predicted bounding boxes, FP, where the overlap I in the i-th type of waste is less than the j-th overlap threshold K. ij The number of predicted bounding boxes, FN is the number of actual bounding boxes for which no corresponding predicted bounding box is given, and the overlap threshold K of the i-th type of garbage is calculated at the j-th time. ij Recall rate R in ij And accuracy P ij The calculation formulas are as follows: Step 3.6.

3. Calculate the overlap threshold sequence according to the method of step 3.6.2 The recall rate and precision of all overlap threshold values in the ith type of spam, obtaining n recall rates R ij and precisions P ij , in order from 1 to n, with the recall rate R ij as the horizontal coordinate and the precision P ij as the vertical coordinate, a curve is drawn in the plane coordinate system, and is denoted as P i -R i curve; P i -R i curve, abscissa, ordinate, P i -R i The line connecting the end point of the curve and the abscissa is the contour line. The area in the contour line is calculated and recorded as the AP value F of the ith type of garbage i ; Step 3.6.4, calculate the AP value of each type of road refuse in the D-type road refuse according to the method of step 3.6.2-step 3.6.3, and obtain D AP values F i , and calculate the first i generation backbone network The average precision of the mean value of the road refuse image in the test sample set ,​ Step 3.

7. Set the number of repetitions as G, and repeat steps 3.4-3.6 G times to obtain a set of networks and a set of mean average precisions , , ; the highest average mean accuracy, , the corresponding backbone network is the network with the best recognition effect, and is denoted as the road garbage recognition network; Step 4, use the road garbage recognition network to recognize the road garbage. 2.The Yolov3 road garbage detection method based on decoupled head according to claim 1, wherein, The C3 module layer The C3 module layer The C3 module layer The C3 module layer Each of the C3 module layers is composed of three layers in series along the input-output direction of the backbone network, specifically as follows: Let 128ζ, 256ζ, 512ζ, 1024ζ be collectively denoted by n ζ, n or equal to 128 or equal to 256 or equal to 512 or equal to 1024, the standard convolutional layer , the standard convolutional layer , the standard convolutional layer and the standard convolutional layer is collectively denoted as the standard convolutional layer α; The first layer structure of the four C3 module layers is composed of parallel passage z1 and passage z2, wherein the passage z1 is composed of one standard convolution layer a and n The ζ bottleneck modules are sequentially connected in series, the passage z2 is composed of one standard convolution layer a, the second layer structure is a Concat module layer, the input is the passage z1 and the passage z2, and the output is serially connected to the third layer structure; and the third layer structure is one standard convolution layer a. 3.The Yolov3 road garbage detection method based on decoupled head according to claim 1, characterized in that, The Neck structure is composed of 8 layers of structures connected in series along the input-output direction of the backbone network: The first layer structure takes the input , the input as input, and includes two parallel paths b1 and b2, wherein the path b1 is formed by the input , a Concat module layer , a convolution layer , and a down-sampling layer connected in sequence, and the path b2 is formed by the input and the Concat module layer ; the second layer structure is formed by the Concat module layer ; the third layer structure takes the output of the Concat module layer , the input as input, and includes two parallel paths b3 and b4, wherein the path b3 is formed by the output of the Concat module layer , a Concat module layer , a C3 module layer , a convolution layer , and a down-sampling layer connected in sequence, and the path b4 is formed by the input and the Concat module layer ; the fourth layer structure is formed by the Concat module layer ; the fifth layer structure takes the output of the Concat module layer , the output of the convolution layer as input, and includes two parallel paths b5 and b6, wherein the path b5 is formed by the output of the Concat module layer , a Concat module layer , a C3 module layer , and a convolution layer connected in sequence, and the path b6 is formed by the output of the convolution layer and the Concat module layer ; the sixth layer structure is the Concat module layer ; the seventh layer structure takes the output of the Concat module layer , the output of the convolution layer as input, and includes two parallel paths b7 and b8, wherein the path b7 is formed by the output of the Concat module layer, a C3 module layer , and a convolution layer connected in sequence, and the path b8 is formed by the output of the convolution layer and the Concat module layer ; the eighth layer structure is formed by the Concat module layer , a C3 module layer , and a channel attention mechanism CA connected in sequence.

4. The Yolov3 road garbage detection method based on the decoupling head according to claim 3, characterized in that, The implementation steps of the channel attention mechanism CA are as follows: The channel attention mechanism CA comprises an average pooling layer , an average pooling layer , a Concat module layer L, a convolution layer , a batch normalization layer A and a Sigmoid nonlinear activation layer J; The channel attention mechanism CA is composed of three layers of structures in series along the input-output direction of the backbone network: the first layer of structure is composed of two parallel channels d1 and d2, the channel d1 is composed of an average pooling layer , and the channel d2 is composed of an average pooling layer , and the two channels are connected to a Concat module layer L; the second layer of structure is composed of the Concat module layer L and a batch normalization layer A in series along the direction of the two parallel channels and the output direction of the backbone network; and the third layer of structure is branched from the output of the batch normalization layer A into two parallel channels d3 and d4, and each of the channels d3 and d4 is composed of a convolution layer and a Sigmoid nonlinear activation layer J in series. The input of the channel attention mechanism CA is the output of the fourth C3 module layer The outputs of the channel d3 and the channel d4 are multiplied with the input of the attention mechanism CA respectively to obtain the output of the attention mechanism CA. 5.The Yolov3 road garbage detection method based on decoupled head according to claim 1, wherein, The specific steps of using the road garbage recognition network to recognize the road garbage are as follows: Step 4.1, input an original road garbage image to be identified, and make the following image processing: adjust the pixels of the original road garbage image to be identified to ; The image Y to be identified of the preceding processing of the channel is recorded as an original road garbage image d ; Step 4.2, in the image Y d subdivide the grid and let any one of them be grid K d v where v is the serial number of the grid, v = 1, 2,..., Λ, Λ is the number of grids; Step 4.3, input the image Y d into the road garbage recognition network to obtain a grid K d v a prediction class probability tensor, a prediction class coordinate tensor and a prediction class IoU tensor of each prediction box in the grid K d vu , u = 1, 2, …, U, U is the number of prediction boxes in the grid K d v , then the prediction class probability tensor corresponding to the prediction box R d vu is obtained, and the prediction probability value O d vu , the prediction probability value P d vu of the prediction class coordinate tensor and the prediction probability value Q d vu of the prediction class IoU tensor are obtained d vu , and the confidence of the prediction box R d vu is obtained, and is recorded as the confidence L d vu ; The confidence L d v is compared with a given confidence threshold L 0 and the following decisions are made: If L d vu ≥ L 0 , the prediction frame is retained; If L d vu < L 0 , the prediction frame is discarded; Step 4.4, repeat step 4.3 until the image Y d All the grids are selected, and then according to the result of the prediction frame judgment in each grid of the image to be identified road garbage, the prediction frame is marked in the image Y d The identified road garbage image is obtained.

Images