Improved yolov5 target detection system and method

Abstract

The application discloses an improved YOLOv5 target detection system and method, and mainly solves the problem that the existing detection algorithm is difficult to balance between network parameter quantity and detection precision. The system comprises an input module, a main unit, a feature fusion unit and an output module which are sequentially connected. External data first enters the input module, is transmitted to the main unit after being processed, the main unit comprises a dilated convolution module and a three-layer continuous dilated convolution module, and the two modules are formed after introducing the dilated convolution in the YOLOv5 network and are used for extracting feature maps of different scales and transmitting the feature maps to the feature fusion unit. The feature fusion unit adopts a structure of a feature pyramid (FPN) plus a path aggregation network (PAN) to fuse and process the feature maps of different scales obtained in the main unit. Finally, the output module is used for target prediction on the fused features, and a detection result is output. The application can effectively improve the detection precision without increasing the parameter quantity and the calculation amount of the detection model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision recognition, further relates to target detection, and in particular to an improved YOLOv5 target detection system and method. BACKGROUND

[0002] Target detection in the field of machine vision refers to automatically detecting and recognizing the position and category of a target in an image or video. Target detection technology has been widely applied in various fields, such as autonomous driving, video surveillance, face recognition, etc. The method based on deep learning is the mainstream method in the current target detection field, which automatically learns the features and classifier of the target by using deep neural networks. Among them, the most popular method is the region proposal-based method, such as Faster R-CNN, SSD and YOLO, etc. These methods first use a deep neural network to generate region proposals of the target, then use a classifier to classify and position regress each proposal, and finally get the position and category of the target. The core of the deep learning-based target detection and recognition method relies on the feature extraction ability of convolution and deep convolution. Enhancing the feature extraction ability of convolution without increasing the inference calculation amount can comprehensively improve the network performance, and has positive theoretical research significance and use value for the detection accuracy of target detection and recognition network with convolution as the core of feature extraction.

[0003] At present, in the field of target detection, the method based on model quantization and compression is developing rapidly, which has attracted widespread attention of researchers. For example, Wei X et al. proposed a random drop activation quantization method in the literature "Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization". However, there are still two problems in the process of model quantization and compression: 1) the precision loss caused by parameter quantization will inevitably affect the detection effect of the model, and once the quantization method is set unreasonably, the detection performance will be poor; 2) the current model compression is mostly not lossless, resulting in differences in detection effect between the compressed and uncompressed models, and the actual effect is not ideal. SUMMARY

[0004] The present application aims at the deficiencies of the prior art, and proposes an improved YOLOv5 target detection system and method, which introduces expanded convolution EC (Expanded Convolution) and feature similarity loss FSL (Feature Similarity Loss) function, improves the detection accuracy of the network while maintaining the existing inference calculation amount and parameter amount of the network. The improved YOLOv5s target detection system Yolov5s-EC-FSL based on expanded convolution and feature similarity loss function can effectively solve the technical problem that the network parameter amount and network accuracy are difficult to balance in the prior art.

[0005] The idea of realizing the present application is as follows: first, in view of the problems of weak feature extraction capability of traditional convolution modules and single performance enhancement mode, expanded convolution is proposed, which enhances the feature extraction capability of the convolution module by introducing virtual parameters in the convolution module. After the model training is completed, the virtual parameters introduced in the expanded convolution are losslessly compressed, so as to improve the detection accuracy of the model without increasing the model parameter amount and calculation amount; second, in view of the problem that there are a large number of redundant feature maps in the network output channel, a feature similarity loss function is introduced, which suppresses the generation of redundant feature maps by backward propagation of the feature similarity loss between feature maps, so that the network parameters are more effectively utilized, thereby realizing the improvement of the detection accuracy of the model.

[0006] The specific scheme for realizing the above-mentioned object of the present application is as follows:

[0007] An improved YOLOv5 target detection system, comprising an input module, a backbone unit, a feature fusion unit and an output module which are sequentially cascaded; external data first enters the input module, is transmitted to the backbone unit after processing, and different scale feature maps are extracted from the backbone unit and transmitted to the feature fusion unit for feature fusion, and finally the detection result is output through the output module;

[0008] The input module is used for formatting processing of image data collected from the outside, and transmitting the processed data to the backbone unit;

[0009] The backbone unit is used for feature extraction, pooling operation of different levels of data processed by the input module, and obtaining different scale feature maps; the unit comprises an expanded convolution EConv module and a three-layer continuous expanded convolution EC3 module, wherein the EConv module is formed by introducing expanded convolution based on the Conv module of the YOLOv5 network; and the EC3 module is formed by introducing expanded convolution in the C3 module of the YOLOv5 network;

[0010] The feature fusion unit adopts a structure of a feature pyramid network (FPN) plus a path aggregation network (PAN) to perform fusion processing on feature maps of different scales obtained in the backbone unit.

[0011] The output module is configured to perform target prediction on the fused feature maps generated by the feature fusion unit to generate a target matrix of a fixed size, i.e., a result of target detection.

[0012] A method for target detection by using an improved YOLOv5 target detection system, comprising the following steps:

[0013] (1) The Pascal Object Recognition Challenge (Pascal VOC) dataset is divided to obtain a VOC07+12 dataset, and the dataset is divided into a training set and a validation set, and the data labels are converted into the same label format as in the YOLOv5 network;

[0014] (2) The training set data is sent to the input module of the system for system model training, the data is preprocessed by the input module to obtain preprocessed data, and the preprocessed data is sent to the backbone unit;

[0015] (3) The backbone unit introduces dilated convolution in the system through the EConv module and the EC3 module, uses more than twice the number of parameters of traditional convolution to extract features from the preprocessed data, and calculates the feature similarity loss of the output channels of all EConv modules in the backbone unit;

[0016] (4) The feature fusion unit first performs top-down semantic information transmission on three feature maps of different scales output by the backbone unit, then performs bottom-up positioning information transmission in the path aggregation network, completes feature fusion of different scales, and finally generates three groups of fused feature maps of different scales;

[0017] (5) The output module predicts three types of targets, i.e., large, medium, and small, from the three groups of fused feature maps of different scales generated by the feature fusion unit, and generates a fixed-size prediction target matrix;

[0018] (6) The confidence loss, position loss, and classification loss are calculated according to the error between the prediction target matrix and the target matrix generated from the real label, the confidence loss, position loss, classification loss, and feature similarity loss obtained in step (3) are added with weights to obtain a total loss, and then the total loss is back propagated to update the model parameters;

[0019] (7) using the updated model to infer the validation set data, respectively calculating the confidence loss, position loss, classification loss and feature similarity loss, and adding the total loss by weight, judging whether the total loss is convergent, if so, obtaining the trained model, and executing step (8);Otherwise, return to step (2) and continue training;

[0020] (8) lossless compression is carried out on the expanded convolution in the trained model to generate a final target detection model;

[0021] (9) input the to-be-tested data into the final target detection model for detection to obtain a detection result.

[0022] Compared with the prior art, the present application has the following advantages:

[0023] Firstly, the present application proposes an expanded convolution based on the traditional convolution, which is an enhanced convolution module, which can be switched with the traditional convolution at any time, and the parameter amount of the expanded convolution can be several times of that of the traditional convolution, so that under the same conditions, the network training effect using the expanded convolution is better, and the detection precision is higher;

[0024] Secondly, since the present application uses a linear method to expand the input channel, more low-cost feature maps are generated, and the information fusion between the channels is more sufficient;

[0025] Thirdly, the expanded convolution proposed by the present application is lossless compressed after training, so that the network has both the advantages of traditional convolution calculation speed and high detection precision of expanded convolution;

[0026] Fourthly, the present application proposes a feature similarity loss function for the redundant feature problem between the output channels of the convolution module, and through the calculation and back propagation of the feature similarity loss between the feature maps, the similarity between the feature maps is suppressed, so that the information extracted by each channel is more comprehensive, thereby effectively improving the detection effect of the network. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The structure schematic diagram of the Yolov5s-EC-FSL target detection system constructed by the present application;

[0028] Figure 2 The flowchart of the method of the present application;

[0029] Figure 3 The application example schematic diagram of the double expanded convolution;

[0030] Figure 4 The output channel expansion rule schematic diagram of the double expanded convolution;

[0031] Figure 5 A schematic diagram of the screening results of the feature similarity loss function in the application;

[0032] Figure 6 A comparison chart of the visualization effects of the first layer output feature maps of the traditional YOLOv5s network and the Yolov5s-EC-FSL network of the application; wherein (a) is the visualization effect of the first layer output feature map of the traditional YOLOv5s network after training, and (b) is the visualization effect of the first layer output feature map of the Yolov5s-EC-FSL network of the application after training;

[0033] Figure 7 A comparison chart of the detection effects of the Yolov5s-EC-FSL network before and after compression; wherein (a) is the detection effect of the Yolov5s-EC-FSL network before compression, and (b) is the detection effect of the Yolov5s-EC-FSL network after compression;

[0034] Figure 8 A comparison chart of the experimental effects of using the method of the application and the existing method for detection; wherein (a) is the experimental effect of using the existing method for detection, and (b) is the experimental effect of using the method of the application for detection. DETAILED DESCRIPTION

[0035] The application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] Embodiment one: refer to Figure 1 The Yolov5s-EC-FSL target detection system, i.e. the improved YOLOv5 target detection system, constructed by the application specifically comprises: an input module, a backbone unit, a feature fusion unit and an output module in turn; external data first enters the input module, is transmitted to the backbone unit after processing, and different scale feature maps are extracted from the backbone unit and transmitted to the feature fusion unit for feature fusion, and finally the detection result is output through the output module.

[0037] The input module is used for performing format processing on the image data collected from the outside, and transmitting the processed data to the backbone unit; specifically, the pictures in the external input system are subjected to data enhancement, adaptive scaling and normalization processing to obtain normalized data of uniform size, and then output to the backbone unit.

[0038] The backbone unit is used for feature extraction and pooling operation of different levels on the data processed by the input module to obtain feature maps of different scales; the unit includes an expanded convolution EConv module and a three-layer continuous expanded convolution EC3 module, wherein the EConv module includes a linear transformation layer, a convolution layer, a normalization layer and a nonlinear activation function layer, and is formed after introducing expanded convolution into the Conv module of the YOLOv5 network; the EC3 module is a residual module including three EConv modules, and is formed by introducing expanded convolution into the C3 module of the YOLOv5 network. The expanded convolution introduced in the backbone network is an enhanced convolution module based on traditional point-by-point convolution, the input and output of the expanded convolution are completely the same as those of the traditional convolution, the input channel is expanded through linear transformation of the input data, and the parameter amount of the expanded convolution is more than twice that of the traditional convolution, wherein the input channel CH and the expanded channel ECH are defined as follows:

[0039] CH=[CH0,CH1,...,CH y ,...,CH n-1 ],

[0040] ECH=[ECH0,ECH1,...,ECH x ,...,ECH m-1 ],

[0041] wherein n is the number of channels of the input data, and m is the number of expanded channels after linear expansion of the input channel;

[0042] The relationship between the input channel CH and the expanded channel ECH is as follows:

[0043]

[0044] wherein CH y represents the yth channel in CH, ECH x represents the xth channel in ECH, λ (x,y) represents the weight coefficient of CH y to ECH x ; the depth of the convolution kernel of the expanded convolution is increased from n to m, and the corresponding parameter amount is also increased by m / n times.

[0045] The feature fusion unit adopts a structure of a feature pyramid FPN plus a path aggregation network PAN, and is used for fusion processing of the feature maps of different scales obtained in the backbone unit; the feature pyramid FPN is used for transmitting semantic features in a deep layer to a shallow layer, the path aggregation network PAN is used for transmitting positioning information in a shallow layer to a deep layer, and the structure of the feature pyramid FPN plus the path aggregation network PAN is used for enhancing the multi-scale positioning capability.

[0046] The output module is configured to perform target prediction on the fused feature map generated by the feature fusion unit to generate a target matrix of a fixed size, i.e., the result of target detection.

[0047] The improved YOLOv5s network, i.e., the YOLOv5s-EC-FSL network, mainly includes an input module Input, a backbone network Backbone, a feature fusion network Neck, and an output end Head.Firstly, the input module is configured to uniformly input an image and scale the image to 640x640x3. Figure 1 The EConv module is formed by introducing a dilated convolution into a Conv module of YOLOv5, and the EC3 module is also formed by introducing a dilated convolution into a C3 module of YOLOv5.

[0048] Embodiment Two: Refer to Figure 2 The method for implementing target detection by using the improved YOLOv5 target detection system includes the following steps:

[0049] Step 1: divide the Pascal Object Recognition Challenge (Pascal VOC) dataset, obtain the VOC07+12 dataset, divide the training set and the validation set, and convert the data labels into the same label format as in the YOLOv5 network.

[0050]

[0051] wherein (x1, y1) and (x2, y2) are the coordinates of the upper left corner and the lower right corner of the target frame, W and H are the width and height of the image, respectively, x and y represent the horizontal and vertical coordinates of the normalized target center point, and w and h represent the normalized target width and height, respectively. centre centre

[0052] ​Step 2. The training set data is input into the input module of the system for system model training. The input module pre-processes the data to obtain pre-processed data and sends it to the backbone unit. The pre-processing operation includes using Mosaic data enhancement technology to increase the number of samples, using adaptive anchor frame to adapt to different data sets, using adaptive picture scaling to reduce the amount of calculation, and adding a Focus module to speed up the inference speed.

[0053] Step 3. The backbone unit introduces dilated convolution in the system through the EConv module and the EC3 module, uses more than twice the number of parameters than traditional convolution to extract features from the pre-processed data, and calculates the feature similarity loss of all output channels of the EConv module in the backbone unit, that is, the feature similarity loss between the output feature maps of the dilated convolution. This calculation is applied between all output feature maps in a convolution module. First, construct the output channel ECHO:

[0054] ECHO = [ECHO0, ECHO1,..., ECHO z ,...,ECHO o-1 ],

[0055] where o represents the number of output channels of ECHO, ECHO z represents the zth output channel of ECHO;

[0056] Then, according to the following steps, calculate the feature similarity loss loss sim :

[0057] (3.1) Convert each output channel in ECHO from a two-dimensional feature map to a one-dimensional vector. Take ECHO M(1) z as an example, where M(1) represents a one-dimensional vector, and ECHO M(1) z represents a one-dimensional vector converted from ECHO z , and its expression is as follows:

[0058] ECHO M(1) z = [x (z,0) ,x (z,1) ,...,x (z,r) ,...,x (z,p-1) ],

[0059] where p represents the number of elements in ECHO M(1) z , that is, the length of the one-dimensional vector, and x (z,r) represents the rth element in ECHO M(1) z .

[0060] (3.2) Calculate the feature similarity loss between each pair of channels in the o output channels of ECHO, to ECHO M(1) zi and ECHO M(1) zj For example, where ECHO M(1) zi and ECHO M(1) zj represent two different channels in ECHO, ECHO zi and ECHO zj After conversion, the corresponding one-dimensional vectors are calculated, and the feature similarity loss A M(1) zi and ECHO M(1) zj between ECHO (zi,zj) :

[0061]

[0062] where x (zi,l) and x (zj,l) are the lth elements in ECHO M(1) zi and ECHO M(1) zj

[0063] (3.3) Set the feature similarity screening threshold to margin, then ECHO M(1) zi and ECHO M(1) zj between ECHO (zi,zj) :

[0064]

[0065] (3.4) Obtain the loss loss all :

[0066] loss all = λ sim loss sim + λ1loss rect + λ2loss obj + λ3loss clc ,

[0067] where loss sim represents the sum of the average values of the feature similarity losses between the respective output feature maps of all dilated convolution modules, loss rect , loss clc , and loss​obj respectively represent the rectangular bounding box loss, the classification loss and the confidence loss, and λ sim represents the loss sim , and λ1, λ2 and λ3 represent the weight of the loss rect , the loss obj and the loss clc , respectively.

[0068] Step 4. The feature fusion unit first performs top-down semantic information transmission on the three different scale feature maps outputted in the backbone unit, then transmits positioning information in a bottom-up manner in the path aggregation network, completes feature fusion of different layers, and finally generates three groups of fused feature maps of different scales;

[0069] Step 5. The output module generates a fixed-size predicted target matrix by predicting the three types of large, medium and small targets respectively from the three groups of fused feature maps of different scales generated by the feature fusion unit;

[0070] Step 6. Calculate the confidence loss, position loss and classification loss according to the error between the predicted target matrix and the target matrix generated by the real label, and add the confidence loss, position loss, classification loss and feature similarity loss obtained in step 3 according to the weight to obtain the total loss, then perform back propagation on the total loss to update the model parameters;

[0071] Step 7. Use the model with updated parameters to infer the validation set data, calculate the confidence loss, position loss, classification loss and feature similarity loss, and add them according to the weight to obtain the total loss, and judge whether the total loss is convergent or not. If yes, the trained model is obtained, and step 8 is executed; otherwise, return to step 2 for further training;

[0072] Step 8. Lossless compression is performed on the dilated convolution in the trained model to generate the final target detection model. Referring to Figure 4 the compression rules corresponding to the dilated convolution are constructed:

[0073] After training, the convolution kernel parameters of the dilated convolution are inversely compressed according to the expansion rule of the input channel, and finally the parameter amount of the dilated convolution is compressed to the same size as the traditional convolution. The compression calculation formula of the dilated convolution is as follows:

[0074]

[0075] where EK represents the convolution kernel of the dilated convolution, EK x represents the parameter of depth x in the convolution kernel of the dilated convolution; the depth of the convolution kernel of the dilated convolution is increased from n to m, and the corresponding parameter amount is increased by m / n times;

[0076]

[0077] wherein K represents the dilated convolution compressed convolution kernel, K y represents the parameters in K with depth y;

[0078] The expression of K is obtained:

[0079]

[0080] Step 9. Input the to-be-tested data into the final target detection model for detection to obtain a detection result.

[0081] Embodiment Three: The target detection method proposed in this embodiment has the same overall implementation steps as those in Embodiment Two. Now, the data preparation process of obtaining the data set and dividing it is described in further detail:

[0082] In this embodiment, the Pascal VOC data set is set according to different division methods, and the VOC07+12 data set is set. The VOC07+12 data set is formed by integrating the VOC2007 data set and the VOC2012 data set to form a larger data set, and the specific settings are as follows:

[0083]

[0084] The VOC07+12 data set includes a total of 21,020 pictures, which is more than twice the VOC2007 data set, of which 16,011 pictures are used as a training set and 5,009 pictures are used as a verification set:

[0085] Each picture in the Pascal VOC data set corresponds to a label file in xml format. The file sequentially covers the picture name corresponding to the label, the picture size, the category of the target, the shooting angle, the truncation state, the recognition difficulty and the position of the target box, and the like. The position information of the target box contains the coordinates of the upper left corner (x min ,y min ) and the lower right corner (x max ,y max ) of the target box. In order to meet the format requirements of the YOLOv5 network used in this paper for data labels during training, it is necessary to convert the VOC label format into the TXT label of YOLOv5. Each line of data in the TXT label file represents a target in the picture corresponding to the label, and each target information is represented by a non-negative integer and four floating-point numbers between 0 and 1. The integer represents the category of the target, and the four floating-point numbers represent the normalized target center coordinates and the width and height of the drawing frame.

[0086] Embodiment Four: The target detection method proposed in this embodiment has the same overall implementation steps as those in Embodiment Two. Now, the data preparation process of obtaining the data set and dividing it is described in further detail: Figure 5The screening of the feature similarity loss is described in further detail as follows:

[0087] The calculation and back propagation of the similarity loss restricts the feature similarity between channels, so that the expansion contains less redundant information under the same total amount of extracted information. The calculation of the similarity loss is applied between all output feature maps in an expansion convolution module. The construction of the output data ECHO is as follows:

[0088] ECHO = [ECHO0, ECHO1,..., ECHO z ,...,ECHO o-1 ]

[0089] where o represents the number of output channels of ECHO. The feature similarity loss loss sim of this network is constructed as follows:

[0090] Each output channel in ECHO is converted from a two-dimensional feature map to a one-dimensional vector. Take ECHO M(1) z as an example, where M(1) represents a one-dimensional vector, and ECHO M(1) z represents a one-dimensional vector converted from ECHO z , and its expression is as follows:

[0091] ECHO M(1) z = [x (z,0) ,x (z,1) ,...,x (z,r) ,...,x (z,p-1) ]

[0092] where p represents the number of elements in ECHO M(1) z , i.e., the length of the one-dimensional vector, and x (z,r) represents the rth element in ECHO M(1) z .

[0093] In this embodiment, the feature map similarity is measured by the absolute value of the spatial cosine value between two vectors. Take ECHO M(1) zi and ECHO M(1) zj as examples, where ECHO M(1) zi and ECHO M(1) zj represent two different channels ECHO zi and ECHO zjThe corresponding one-dimensional vector after conversion, calculate ECHO M(1) zi And ECHO M(1) zj The feature similarity loss A between (zi,zj) :

[0094]

[0095] Where x (zi,l) And x (zj,l) The first element in ECHO M(1) zi And ECHO M(1) zj ;

[0096] In this embodiment, in order to avoid the loss of small feature similarity affecting the convergence direction of model parameters, a feature similarity loss threshold is set to screen the feature similarity loss, Figure 5 In this embodiment, in order to avoid the loss of small feature similarity affecting the convergence direction of model parameters, a feature similarity loss threshold is set to screen the feature similarity loss,

[0097] The feature similarity loss loss (i,j) Between the ith and jth output feature maps:

[0098]

[0099] Where margin represents the feature similarity screening threshold.

[0100] The construction method of the loss loss all In the whole network:

[0101] loss all =λ sim loss sim +λ1loss rect +λ2loss obj +λ3loss clc

[0102] Where loss sim Represents the sum of the average values of the feature similarity losses between the respective output feature maps of all dilated convolution modules in the network, loss rect , loss clc And loss obj Respectively represent the rectangle frame loss, classification loss and confidence loss, λ sim Is the loss weight of loss sim , λ1, λ2 and λ3 represent the loss weights of the corresponding losses respectively.

[0103] The effect of the present application is further illustrated by experiments.

[0104] 1. Experimental conditions:

[0105] The experimental software and hardware environment of the present application:

[0106]

[0107] 2. Experimental content:

[0108] In order to fully test the real effect of the dilated convolution and the feature similarity loss function, YOLOv5s is used as the basic network in this experiment, and double dilated convolution and feature similarity loss function are introduced on this basis. The experiment is verified in VOC07+12 dataset, and the map0.5 and FPS of the two networks are compared.

[0109] 3. Analysis of experimental results:

[0110] The following are the objective results of the experiment:

[0111]

[0112] It can be seen that after introducing the dilated convolution and the feature similarity loss function, the mAP of the Yolov5s-EC-FSL model in the VOC07+12 dataset reaches 80.8%, which is 1.6% higher than the basic model. From the comparison of the experimental data of Yolov5s and Yolov5s-EC-FSL, it can be seen that after introducing the feature similarity loss function, the parameter quantity, calculation quantity and detection speed of the model are consistent with the basic model.

[0113] The following is the comparison of the feature similarity distribution of the first layer output feature map of the two networks when the feature similarity threshold is 0.80:

[0114]

[0115]

[0116] In the VOC07+12 dataset, after introducing the feature similarity loss function, the number of redundant feature maps of the network is reduced to 1 pair. It can be seen that the feature similarity loss function has a very obvious effect on the feature similarity between feature maps, and can effectively reduce the occurrence of redundant feature maps.

[0117] Figure 5Figures 1(a) and 1(b) are visualized results of the first layer output feature maps of the Yolov5s and Yolov5s-EC-FSL after training, respectively. In the output feature maps of the Yolov5s model, there are many pictures with high subjective similarity. In the output feature maps of the Yolov5s-EC-FSL model, the number of feature maps with high similarity is obviously reduced. Such an effect is consistent with the statistical results of the feature similarity loss, and achieves the design goal of the feature similarity loss function. After the feature similarity loss function is added, the similarity between the feature maps is significantly reduced, the difference between the feature maps is increased, and the expected effect of suppressing redundant feature maps is achieved.

[0118] Figure 6 Figures 1(a) and 1(b) are visualized results of the first layer output feature maps of the Yolov5s and Yolov5s-EC-FSL after training, respectively. In the output feature maps of the Yolov5s model, there are many pictures with high subjective similarity. In the output feature maps of the Yolov5s-EC-FSL model, the number of feature maps with high similarity is obviously reduced. Such an effect is consistent with the statistical results of the feature similarity loss, and achieves the design goal of the feature similarity loss function. After the feature similarity loss function is added, the similarity between the feature maps is significantly reduced, the difference between the feature maps is increased, and the expected effect of suppressing redundant feature maps is achieved.

[0119] Figure 7 Figures 2(a) and 2(b) are detection effect diagrams of the Yolov5s-EC-FSL model before and after compression, respectively. From the detection results of the model before and after compression, it can be seen that the target categories, confidence size, frame position, etc. are completely the same. The subjective comparison result is completely consistent with the objective analysis result, fully showing that the compression of the dilated convolution is lossless.

[0120] Figure 8In the middle, (a) is the actual detection effect diagram of the Yolov5s network, (b) is the actual detection effect diagram of the Yolov5s-EC-FSL network, it can be seen that the introduction of the dilated convolution and the feature similarity loss function has certain improvement on the detection effect of small targets such as bottles and potted plants. In the detection of the first picture, the yolov5s model does not detect the potted plant target, and the yolov5s-EC-FSL model detects two potted plant targets, which has greater effect improvement than the yolov5s model. In the detection of the second picture, the yolov5s model has the problem of frame overlapping, and in the detection of the third picture, the yolov5s model misses a bottle target, which are all correctly detected in the Yolov5s-EC-FSL network.

[0121] The above experimental results prove the correctness and effectiveness of the method of the present application.

[0122] The part not described in detail in the present application belongs to the common knowledge of those skilled in the art.

[0123] The above only describes the preferred embodiments of the present application and is not used to limit the present application, and it is obvious that those skilled in the art can make various modifications and changes in form and details without departing from the principles and structure of the present application.

[0124] of the present application, and these modifications and changes based on the idea of the present application are still within the protection scope of the claims of the present application.

Claims

1. An improved YOLOv5 target detection system, characterized in that: It includes an input module, a backbone unit, a feature fusion unit, and an output module that are cascaded in sequence. External data first enters the input module, is processed, and then transmitted to the backbone unit. The backbone unit extracts feature maps of different scales from the data and transmits them to the feature fusion unit for feature fusion. Finally, the detection results are output through the output module. The input module is used to format the image data acquired from the outside and transmit the processed data to the backbone unit; The backbone unit is used to perform feature extraction and pooling operations at different levels on the data processed by the input module to obtain feature maps at different scales. This unit includes a dilated convolution EConv module and a three-layer consecutive dilated convolution EC3 module. The EConv module is formed by introducing dilated convolution on top of the Conv module of the YOLOv5 network; the EC3 module is formed by introducing dilated convolution into the C3 module of the YOLOv5 network. The EConv module in the backbone unit contains a linear transformation layer, a convolutional layer, a normalization layer, and a non-linear activation function layer; the EC3 module is a residual module containing three EConv modules. The dilated convolution introduced into the backbone network is an enhanced convolutional module based on traditional pointwise convolution. The input and output of the dilated convolution are exactly the same as those of traditional convolution. It expands the input channels through a linear transformation of the input data. The number of parameters in the dilated convolution is more than twice that of traditional convolution, and its input channels... With extended channels They are defined as follows: ， ， in, The number of channels for the input data. The number of extended channels after linear expansion of the input channels; The input channel With extended channels The relationship between them is as follows: ， in, represent The y-th channel in represent The xth channel in express right The weight coefficients; the kernel depth of dilated convolution is determined by... Increase to The number of parameters also increases accordingly. times; The feature fusion unit adopts a structure of feature pyramid (FPN) plus path aggregation network (PAN) to fuse feature maps of different scales obtained from the backbone unit. The feature pyramid (FPN) transmits deep semantic features to shallow layers, while the path aggregation network (PAN) transmits shallow positioning information to deep layers. The structure of feature pyramid (FPN) plus path aggregation network (PAN) is used to enhance multi-scale positioning capabilities. The output module is used to predict targets from the fused feature map generated by the feature fusion unit, and generate a target matrix of fixed size, which is the result of target detection.

2. The system according to claim 1, characterized in that: The input module is used to format image data acquired from the outside. Specifically, it performs data enhancement, adaptive scaling, and normalization on the images in the external input system to obtain normalized data of uniform size and outputs it to the backbone unit.

3. A method for target detection using the system according to claim 1, characterized in that, Includes the following steps: (1) Divide the Pascal VOC dataset of the Pascal Object Recognition Challenge to obtain the VOC07+12 dataset, divide it into training set and validation set, and convert the data labels into the same label format as in the YOLOv5 network; (2) The training set data is sent to the system input module for system model training. The input module performs preprocessing operations on the data to obtain preprocessed data, which is then sent to the backbone unit. (3) The backbone unit introduces dilated convolution in the system through the EConv module and the EC3 module, uses more than twice the number of parameters as traditional convolution to extract features from the preprocessed data, and performs feature similarity loss calculation on the output channels of all EConv modules in the backbone unit. (4) The feature fusion unit first performs top-down semantic information transmission on the three different scale feature maps output from the backbone unit, and then transmits the localization information from bottom to top in the path aggregation network to complete the feature fusion of different layers, and finally generates three sets of fused feature maps of different scales. (5) The output module predicts three types of targets—large, medium, and small—from the three sets of fused feature maps of different scales generated by the feature fusion unit, and generates a fixed-size prediction target matrix. (6) Calculate the confidence loss, location loss and classification loss based on the error between the predicted target matrix and the target matrix generated from the real labels, and add the confidence loss, location loss, classification loss and the feature similarity loss obtained in step (3) according to weights to obtain the total loss. Then backpropagate the total loss to update the model parameters. (7) Use the model with updated parameters to perform inference on the validation set data, calculate the confidence loss, location loss, classification loss and feature similarity loss respectively, and add them together according to their weights to obtain the total loss. Determine whether the decrease in the total loss tends to converge. If yes, obtain the trained model and execute step (8); otherwise, return to step (2) to continue training. (8) Perform lossless compression on the dilated convolutions in the trained model to generate the final object detection model; (9) Input the data to be tested into the final target detection model for detection and obtain the detection results.

4. The method according to claim 3, characterized in that: In step (1), the data tags are converted to the same tag format as in the YOLOv5 network. Specifically, the VOC data tags are converted to TXT tags, as follows: ， in, and These are the coordinates of the top-left and bottom-right corners of the target bounding box, respectively. and These are the width and height of the image, respectively. and y and y represent the normalized x and y coordinates of the target center point, respectively, and w and h represent the normalized target width and height, respectively.

5. The method according to claim 3, characterized in that: In step (2), the input module performs preprocessing operations on the data, including using Mosaic data augmentation technology to increase the number of samples, using adaptive anchor boxes to adapt to different datasets, using adaptive image scaling to reduce the amount of computation, and adding the Focus module to speed up inference.

6. The method according to claim 3, characterized in that: In step (3), feature similarity loss is calculated for the output channels of all EConv modules in the backbone unit, that is, feature similarity loss is calculated between the output feature maps of the dilated convolution. This calculation applies to the pairwise relationships between all output feature maps in a dilated convolution module. First, the output data of the dilated convolution module is constructed. : ， in, represent The number of output channels, represent The z-th output channel; Then, calculate the feature similarity loss using the following steps. : (3.1) will Each output channel in the process undergoes a transformation from a two-dimensional feature map to a one-dimensional vector. For example, where M(1) represents a one-dimensional vector, Representative by The resulting one-dimensional vector has the following expression: ， Where p represents The number of elements in the vector, i.e., the length of the one-dimensional vector. represent The r-th element; (3.2) Calculation In the o output channels, the feature similarity loss between each pair of channels is... and For example, among which and Represent Two different channels and The corresponding one-dimensional vector after transformation is calculated. and Feature similarity loss between : ， in and respectively and The l-th element; (3.3) Set the feature similarity filtering threshold to margin, then and Feature similarity loss between : ； (3.4) Obtain the loss of the entire system : ， in, This represents the sum of the mean feature similarity losses among the output feature maps of all dilated convolutional modules. , and These represent the bounding box loss, classification loss, and confidence loss, respectively. express The loss weight, , and They represent , and The weight of the loss.

7. The method according to claim 3, characterized in that: In step (8), lossless compression is performed on the dilated convolutions in the trained model. Specifically, after training, the kernel parameters of the dilated convolutions are compressed in reverse according to the expansion rules of their input channels, ultimately compressing the number of parameters of the dilated convolutions to the same size as those of traditional convolutions. The implementation process is as follows: ； in, The kernel representing dilated convolution. The parameter representing the depth x in the kernel of the dilated convolution; the depth of the kernel of the dilated convolution is determined by... Increase to The number of parameters also increases accordingly. times; ； Where K represents the convolution kernel after dilated convolution compression. The parameter representing depth y in K; The expression for obtaining K is: 。

Images