Improved target detection method, device, and training method based on YOLOv5

Abstract

The present application relates to an improved target detection method, device, and training method based on YOLOv5. The target detection method includes: inputting the weighted backbone features output by each key feature extraction layer into the key feature fusion node specified by the feature fusion module, re-fusing the weighted backbone features with the original FPN fusion features to obtain target fusion features; adding a sub-detection network to weight and re-fuse the high-resolution features obtained by the shallowest key feature extraction layer to obtain high-detail features, and inputting the high-detail features into a new small detection head and feature fusion module; outputting the prediction box of each image to be detected by each detection head to obtain the target detection result of the image to be detected; introducing the SIOU loss function that considers multiple losses such as angle loss during the network training process; and adopting this method can improve the accuracy of small target detection.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an improved target detection method, device, and training method based on YOLOv5. Background Art

[0002] With the popularization of drone technology, drones have been widely used in environmental surveys, road traffic flow monitoring, safety inspections and other fields. When using drones for environmental surveys, road traffic flow monitoring and safety inspections, it is necessary to perform target detection on the images captured by drones.

[0003] In related technologies, traditional target detection algorithms are often used to detect objects in images captured by drones. However, since objects in drone images are relatively small, using these algorithms can easily lead to errors in detecting and identifying small objects. This means that traditional target detection algorithms have low accuracy in detecting objects in drone aerial photography scenarios. Therefore, improving the accuracy of small target detection has become a pressing technical issue for those skilled in the art. Summary of the Invention

[0004] Based on this, it is necessary to provide a target detection method, device and training method based on YOLOv5 that can improve the accuracy of target detection in response to the above technical problems.

[0005] In a first aspect, the present application provides a target detection method. The method comprises:

[0006] Obtain the image to be detected;

[0007] Inputting the image to be detected into an object detection network improved based on YOLOv5, wherein the object detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module;

[0008] Extract features from the image to be detected through multiple key feature extraction layers of the feature extraction module, and sequentially obtain target backbone features of multiple scales, wherein the target backbone features refer to key image features obtained sequentially in the feature extraction module according to the connection order. As the network depth increases, the corresponding target backbone features carry less detailed information about the image to be detected and more semantic information about the image to be detected;

[0009] Performing weighted processing on the target backbone features of the multiple scales respectively to obtain weighted backbone features of the multiple scales;

[0010] Inputting the weighted backbone features of the multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module respectively, and fusing them again with the original fusion features in the feature fusion module to obtain the target fusion features;

[0011] The target fusion feature is input into the detection head, and the output of the detection head is obtained to obtain the prediction box corresponding to the image to be detected, and the target detection result of the image to be detected is determined according to the prediction box.

[0012] In a second aspect, the present application also provides a method for training a target detection network. The method comprises:

[0013] Obtaining a training image set and a ground truth box corresponding to an object to be detected in each training image in the training image set;

[0014] Inputting the training image into an original detection network improved based on YOLOv5, wherein the original detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module;

[0015] Performing feature extraction on the image to be detected through multiple key feature extraction layers of the feature extraction module to sequentially obtain target backbone features of multiple scales, wherein the target backbone features refer to key image features obtained sequentially in the feature extraction module according to the connection order. As the network depth increases, the corresponding target backbone features carry less detailed information about the training image and more semantic information about the training image;

[0016] Performing weighted processing on the target backbone features of the multiple scales respectively to obtain weighted backbone features of the multiple scales;

[0017] Inputting the weighted backbone features of the multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module respectively, and fusing them again with the original fusion features in the feature fusion module to obtain the target fusion features;

[0018] Inputting the target fusion feature into the detection head, obtaining the output of the detection head, and obtaining the prediction box corresponding to the training image;

[0019] According to the difference between the predicted box corresponding to the training image and the real box, the original detection network is adjusted until the original detection network converges to obtain the object detection network.

[0020] In a third aspect, the present application further provides a target detection device. The device comprises:

[0021] An image acquisition module, used to acquire an image to be detected;

[0022] An image input module is used to input the image to be detected into a target detection network improved based on YOLOv5, wherein the target detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module;

[0023] An extraction module is configured to extract features from the image to be detected through multiple key feature extraction layers of the feature extraction module, and sequentially obtain target backbone features of multiple scales, wherein the target backbone features refer to key image features obtained sequentially in the feature extraction module according to the connection order, and as the network depth increases, the corresponding target backbone features carry less detailed information about the image to be detected and more semantic information about the image to be detected;

[0024] A weighting module, configured to perform weighted processing on the target backbone features of the multiple scales to obtain weighted backbone features of the multiple scales;

[0025] A fusion module is used to input the weighted backbone features of the multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module, and fuse them with the original fusion features in the feature fusion module to obtain the target fusion features;

[0026] The prediction module is used to input the target fusion feature into the detection head, obtain the output of the detection head, obtain the prediction box corresponding to the image to be detected, and determine the target detection result of the image to be detected based on the prediction box.

[0027] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the target detection method of the first embodiment or the steps of the target detection network training method of the second embodiment.

[0028] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the target detection method of the first embodiment, or the steps of the target detection network training method of the second embodiment.

[0029] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the target detection method of the first embodiment, or implements the steps of the target detection network training method of the second embodiment.

[0030] The above-mentioned improved target detection method, device, and training method based on YOLOv5, through weighted processing of target backbone features of multiple scales in the original YOLOv5 feature extraction module, input them into the key nodes of the specified depth in the original YOLOv5 feature fusion module, and fuse them with the original fusion features again to obtain target fusion features, so that the target backbone features carrying more detail information and the fusion features carrying richer semantic information are fused again, so that the feature fusion module pays more attention to the shallow target backbone features with rich details, so as to pay more attention to the detail information of the image to be detected, thereby facilitating the identification of small targets in the image to be detected, avoiding the target detection network from missing small targets in the image to be detected, and improving the accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 FIG2 is a diagram illustrating an application environment of a target detection method and a training method for a target detection network in one embodiment;

[0032] Figure 2 1 is a schematic diagram of a first flow chart of a target detection method in one embodiment;

[0033] Figure 3 1 is a schematic diagram of a first structure of an object detection network in one embodiment;

[0034] Figure 4 2 is a schematic diagram of a second structure of a target detection network in one embodiment;

[0035] Figure 5 1 is a schematic diagram of a second flow chart of a target detection method according to an embodiment;

[0036] Figure 6 1 is a schematic diagram of a third flow chart of a target detection method in one embodiment;

[0037] Figure 7 1 is a schematic diagram of a fourth flow chart of a target detection method in one embodiment;

[0038] Figure 8 1 is a flow chart of a method for training an object detection network in one embodiment;

[0039] Figure 9 1 is a flow chart of the steps for adjusting the target detection network in one embodiment;

[0040] Figure 10 A schematic diagram of the angle difference between the real frame and the predicted frame in one embodiment;

[0041] Figure 11 Schematic diagram of the distance difference between the real box and the predicted box in one embodiment;

[0042] Figure 12 Schematic diagram of scale information between a predicted box and a true box in one embodiment;

[0043] Figure 13 A schematic diagram of the overlapping area between the predicted box and the true box in one embodiment;

[0044] Figure 14 1 is a fifth flow chart of a target detection method according to an embodiment;

[0045] Figure 15 is a structural block diagram of a target detection device in one embodiment;

[0046] Figure 16 is a structural block diagram of a training device for an object detection network in one embodiment;

[0047] Figure 17 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0049] The target detection method and target detection network training method provided in the embodiments of the present application can be applied to the field of small target detection, such as the field of drone aerial photography, and can be applied to Figure 1In the application environment shown. The target detection method can be executed by the terminal 102 or the server 104, and the target detection network training method can be executed by the server 104. After the server 104 trains the target detection network, it can store the target detection network on a data storage system or in the local memory of the terminal. The terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process, such as images to be detected, original images, training image sets, and target detection networks. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. Terminal 102 obtains an image to be detected and inputs the image to be detected into a target detection network improved based on YOLOv5, wherein the target detection network includes a feature extraction module and a feature fusion module, and the output of the feature extraction module is used as the input of the feature fusion module; the feature extraction module extracts features of the image to be detected and obtains target backbone features of multiple scales in sequence, wherein the target backbone features refer to key image features obtained in sequence in the feature extraction module according to the connection order, and as the network depth increases, the corresponding target backbone features carry less detail information about the image to be detected and carry more semantic information about the image to be detected; the target backbone features of multiple scales are weighted to obtain weighted backbone features of multiple scales; the weighted backbone features of multiple scales are input into the key fusion nodes of the corresponding levels in the feature fusion module, and are fused again with the original fusion features in the feature fusion module to obtain target fusion features, the target fusion features are input into the detection head, and the output of the detection head is obtained to obtain the prediction box corresponding to the image to be detected, and the target detection result of the image to be detected is determined based on the prediction box.

[0050] The terminal 102 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0051] In one embodiment, Figure 2 As shown, a target detection method is provided, which is applied to Figure 1 Taking the terminal 102 in FIG. 1 as an example, the method includes the following steps:

[0052] Step 202: Acquire an image to be detected.

[0053] The image to be detected may refer to an image for target detection. The image to be detected may be a high-angle, high-resolution image captured by a drone in the field of drone aerial photography, such as a traffic flow image, agricultural monitoring image, or security monitoring image. It may also be a vehicle image captured by a vehicle camera installed on the road. The image to be detected may be captured in real time or stored in the terminal 102 or server 104.

[0054] For example, the image to be detected stored in the server 104 may be acquired through a network.

[0055] In step 204, the image to be detected is input into the target detection network improved based on YOLOv5, wherein the target detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module.

[0056] The target detection network can refer to a network used to detect targets in an image to be detected. The target detection network can be improved by the original YOLOv5 network. The structure of the target detection network is as follows: Figure 3 The target detection network includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module.

[0057] The feature extraction module includes multiple ConvBNSILU modules, C3 modules, and SPFF modules. The upsampling module is used to upsample the input features. The ConvBNSILU module consists of a CONV convolution operation, a BN operation (Batch Normalization), and a SILU activation function, performing convolution, batch normalization, and activation on the input features. The structure and function of the C3 module are the same as those of the CSP in the original YOLO series, and are used to improve the learning ability of CNNs, reduce computation time, and reduce memory usage.

[0058] The feature fusion module includes multiple ConvBNSILU modules, Concat modules, and C3 modules. The Concat operation included in the Concat module is used to superimpose two layers of features with the same input channel size and add the number of channels. The specific structure of the feature extraction module and the feature fusion module and the connection method of each layer can be referred to Figure 3 .

[0059] In step 206, feature extraction is performed on the image to be detected through multiple key feature extraction layers of the feature extraction module to obtain target backbone features of multiple scales in sequence. The target backbone features refer to the key image features obtained in sequence according to the connection order in the feature extraction module. As the network depth increases, the corresponding target backbone features carry less detailed information about the image to be detected and carry more semantic information about the image to be detected.

[0060] Among them, the key feature extraction layer can refer to Figure 3 The network layer where the C3 module is located. For example, the C3 module corresponding to the 4th layer network, the C3 module corresponding to the 6th layer network, and the C3 module corresponding to the 8th layer network. The connection method of the key feature extraction layer is as follows Figure 3 shown.

[0061] The target backbone feature can refer to the key image features extracted by the feature extraction module from the image to be detected. Different network depths correspond to different scales of the output target backbone features.

[0062] The network depth can refer to the depth of each layer in the target detection network. The depth of each layer can be as follows Figure 3 shown.

[0063] For example, the feature extraction module extracts features from the image to be detected, and the output of the feature extraction module is obtained to obtain the corresponding target backbone features. As the scale of the target backbone features decreases (the depth of the network increases), the detailed information about the image to be detected carried by them decreases, and the semantic information about the image to be detected carried by them increases.

[0064] In step 208 , target backbone features at multiple scales are weighted to obtain weighted backbone features at multiple scales.

[0065] The weighted backbone feature may refer to a feature obtained by weighting the target backbone feature according to a weight coefficient.

[0066] The weight coefficient is pre-set and can be modified according to actual conditions. For example, it can be set according to the scale of the target trunk feature and the scale of the target to be detected. For example, when detecting a smaller-scale target, the weight coefficient corresponding to the target trunk feature with a larger scale can be set larger, and the weight coefficient corresponding to the target trunk feature with a smaller scale can be set smaller, so that the small detection head of the target detection network pays more attention to the detail information. For example, the target trunk feature output by the C3 module corresponding to the 4th layer network has its corresponding weight coefficient named weight3, the target trunk feature output by the C3 module corresponding to the 6th layer network has its corresponding weight coefficient named weight4, and the target trunk feature output by the C3 module corresponding to the 8th layer network has its corresponding weight coefficient named weight5. When the image to be detected contains a large number of small targets, weight3 can be set to 2, weight4 to 1, and weight5 to 0.5. When the image to be detected contains almost no small targets or is basically a large target, all weight coefficients can be set to 0. In addition, other settings are also possible, and no longer given examples here.

[0067] Exemplarily, the target backbone features of each designated shallow layer output are weighted according to a pre-set weight coefficient to obtain a weighted backbone feature.

[0068] For example, the weight coefficient corresponding to the C3 module corresponding to the 4th layer of the network can be set to 2, and then the target backbone features output by the C3 module corresponding to the 4th layer of the network are weighted according to the weight coefficient to obtain the corresponding weighted backbone features. The weight coefficient corresponding to the C3 module corresponding to the 6th layer of the network can be set to 1, and then the target backbone features output by the C3 module corresponding to the 6th layer of the network are weighted according to the weight coefficient to obtain the corresponding weighted backbone features. The weight coefficient corresponding to the C3 module corresponding to the 8th layer of the network can be set to 0.5, and then the target backbone features output by the C3 module corresponding to the 8th layer of the network are weighted according to the weight coefficient to obtain the corresponding weighted backbone features.

[0069] In step 210 , the weighted backbone features of multiple scales are respectively input into the key fusion nodes of the corresponding levels in the feature fusion module, and are fused again with the original fusion features in the feature fusion module to obtain the target fusion features.

[0070] Among them, the key fusion nodes of the corresponding levels in the feature fusion module can refer to the Concat modules of pre-set levels, such as the 20th layer (belonging to the detection subnetwork), the 23rd layer, the 26th layer, and the 29th layer.

[0071] For example, the weighted backbone features output by the C3 module corresponding to the 4th layer of the network can be input into the Concat module of the 23rd layer for fusion, the weighted backbone features output by the C3 module corresponding to the 6th layer of the network can be input into the Concat module of the 26th layer for fusion, and the weighted backbone features output by the C3 module corresponding to the 8th layer of the network can be input into the Concat module of the 29th layer for fusion. The features output by these Concat modules after re-fusion have more detailed information.

[0072] In step 212, the target fusion feature is input into the detection head, and the output of the detection head is obtained to obtain the prediction box corresponding to the image to be detected, and the target detection result of the image to be detected is determined based on the prediction box.

[0073] A prediction box can be used to represent an object in the image to be detected. When there is only one object in the image to be detected, there is only one corresponding prediction box. When there are multiple objects in the image to be detected, there are multiple corresponding prediction boxes.

[0074] For example, after obtaining the target fusion feature, the target fusion feature is input into the detection head for processing, and the output of the detection head is obtained to obtain the prediction box corresponding to the image to be detected. If there is only one prediction box, it is directly used as the target detection result. If there are multiple prediction boxes, the multiple prediction boxes are used as the target detection results.

[0075] The technical solution of the embodiment of the present application is to input the weighted backbone features of multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module respectively, and fuse them with the original fusion features in the feature fusion module, so that the backbone features carrying more detail information and the original fusion features carrying richer semantic information are fused again to obtain the target fusion features. The fused features have more detail information, and the weighting mechanism of the target backbone features enables the small detection head of the target detection network to pay more attention to the detail information of the image to be detected, thereby facilitating the identification of small targets in the image to be detected, avoiding the target detection network from missing small targets in the image to be detected, and improving the accuracy of target detection.

[0076] See Figure 4 and Figure 5 In some embodiments, the target detection network further includes a detection sub-network, the output of the feature extraction module serves as the input of the detection sub-network, and the output of the detection sub-network serves as the input of the feature fusion module. Figure 4 shown.

[0077] The target detection method also includes but is not limited to the following steps:

[0078] Step 502: Obtain the maximum-scale target backbone features output by the feature extraction module to obtain high-resolution features.

[0079] The high-resolution feature may refer to the feature output by the first C3 module in the feature extraction module. The high-resolution feature is the feature with the most detailed information of the image to be tested and the largest scale among all the output target backbone features.

[0080] For example, obtaining Figure 3 The target backbone features output by the C3 module corresponding to the second layer network shown in the figure are used to obtain high-resolution features.

[0081] Step 504 : weighting the high-resolution features according to the weight coefficients to obtain weighted high-resolution features.

[0082] The weight coefficient is a preset coefficient, which can be set according to actual conditions.

[0083] For example, the weight coefficient corresponding to the second layer of the network can be set to 3, and then the high-resolution features are weighted according to the weight coefficient to obtain weighted high-resolution features.

[0084] In step 506, the weighted high-resolution features and the fusion features output by the corresponding FPN structure of YOLOv5 are input into the detection subnetwork to obtain high-detail detection features.

[0085] Among them, the fusion features output by the corresponding FPN structure of YOLOv5 can refer to the features output by the 19th layer network.

[0086] For example, the detection subnetwork fuses the weighted high-resolution features with the fused features output by the corresponding FPN structure of YOLOv5, and then processes them through the C3 module to obtain high-detail detection features. The high-detail detection features have rich detail information.

[0087] For example, the weighted high-resolution features can be input into the Concat module corresponding to the 20th layer of the network in the feature fusion module, so that the feature fusion layer can fuse the original FPN fusion features and high-resolution features output by the Upsample upsampling module corresponding to the 19th layer of the network again.

[0088] Step 508: Input the high-detail features into the corresponding high-resolution detection head and feature fusion module.

[0089] The high-resolution detection head may refer to a detection head used for performing target detection based on high-detail detection features.

[0090] For example, high-detail features are input to Figure 3The high-resolution detection head P2 in the image is used to input the high-detail features into the 22nd layer ConvBNSILU module, and then input into the feature fusion module to facilitate the detection of small targets.

[0091] The technical solution of the embodiment of the present application, by inputting the weighted high-resolution features of the feature extraction module into the detection sub-network, makes the detection sub-network pay more attention to detail information, obtains high-resolution features with particularly rich detail information, and makes the detection head of the sub-network pay more attention to detail information, further improving the detection accuracy of smaller-scale targets.

[0092] See Figure 6 In some embodiments, before the step of "obtaining an image to be detected", the target detection method further includes but is not limited to the following steps:

[0093] Step 602: Acquire the original image and the image resolution of the original image.

[0094] The original image may be an image captured by a device with a camera function, such as a drone, a road probe, or a satellite. The original image may be stored in a data storage system of the server 104 or in a local memory of the terminal 102.

[0095] Image resolution can refer to the amount of information stored in the original image, that is, how many pixels are there per inch in the image.

[0096] For example, the original image and the image resolution corresponding to the original image stored in the data storage system of the server 104 may be acquired through the network.

[0097] Step 604 : If the image resolution is greater than or equal to the resolution threshold, the original image is segmented to obtain a plurality of images to be detected; wherein the image to be detected and adjacent images to be detected have overlapping areas.

[0098] The resolution threshold is a preset threshold and can be modified according to specific circumstances.

[0099] For example, when the image resolution is greater than or equal to the resolution threshold, it indicates that the image resolution of the original image is too large. To facilitate detection, improve detection accuracy and efficiency, and reduce computational complexity, the original image is segmented to obtain multiple images to be detected. The image to be detected and adjacent images to be detected have overlapping areas.

[0100] For example, when the image resolution is greater than or equal to the resolution threshold, the original image is divided into 6 images to be detected, and there is a 25% overlap between the images to be detected and the adjacent images to be detected.

[0101] The technical solution of the embodiment of the present application judges the image resolution and resolution threshold of the original image, and when the image resolution is greater than or equal to the resolution threshold, the original image is segmented to obtain multiple images to be detected, so as to improve the accuracy and efficiency of the subsequent target detection network in detecting the original image and reduce the amount of calculation. Moreover, by setting the image to be detected and the adjacent images to be detected to have overlapping areas, it is possible to avoid missing detection of the target to be detected when there is a target to be detected in the cutting area, thereby making the detection more complete and improving the accuracy of target detection.

[0102] In some embodiments, the target detection method further includes: when the image resolution is less than a resolution threshold, using the original image as the image to be detected.

[0103] Specifically, when the image resolution is less than the resolution threshold, it means that the image resolution of the original image is not large. In order to save detection resources and improve detection efficiency, the original image can be directly used as the image to be detected.

[0104] See Figure 7 In some embodiments, the target detection method further includes but is not limited to the following steps:

[0105] Step 702: Obtain target detection results corresponding to the image to be detected that belongs to the same original image.

[0106] Among them, the images to be detected belonging to the same original image can refer to multiple images to be detected that are segmented from the original image when the original image is segmented. For example, when the original image is segmented, the images to be detected that are segmented can be named and marked. For example, for the original image A, when 6 images to be detected are obtained by segmentation, these 6 images to be detected can be named A-1, A-2, A-3, A-4, A-5, A-6, A-7 and A-8 respectively. When obtaining the target detection results corresponding to the images to be detected of the same original image, the target detection results corresponding to all images to be detected with the A mark can be obtained. The target detection results of each image to be detected can be stored in the data storage system of the server 104, or can be stored in the local memory of the terminal 102.

[0107] Exemplarily, the target detection results corresponding to the images to be detected with the same identification may be acquired through a network to obtain the target detection results corresponding to the images to be detected belonging to the same original image.

[0108] Step 704 : performing coordinate transformation on each target detection result to obtain the position of each target detection result in the original image.

[0109] The coordinate conversion may refer to a method of converting pixel points corresponding to each coordinate in the image to be detected to corresponding coordinates in the original image.

[0110] Exemplarily, the segmentation method used when segmenting the original image can be obtained, and the method of establishing the coordinate system of each image to be detected can be obtained. Then, the coordinates of each pixel point in each image to be detected are transformed according to the segmentation method and the method of establishing the coordinate system of each image to be detected to obtain the position of each pixel point in each image to be detected in the original image, and then determine the position of each target detection result in the original image.

[0111] Step 706 : Determine the detection result of the original image according to the position of each target detection result in the original image.

[0112] Exemplarily, all target detection results may be merged and duplicate target detection results may be removed to obtain the detection result of the original image.

[0113] The technical solution of the embodiment of the present application performs coordinate transformation on the target detection results corresponding to each image to be detected to determine the position of each target detection result in the original image, thereby facilitating the determination of the detection results of the original image and improving the accuracy of the detection results of the original image.

[0114] In some embodiments, the step of "determining the detection results of the original image based on the positions of the target detection results in the original image" includes: merging the target detection results according to the positions of the target detection results in the original image to obtain a merged detection result; and filtering the merged detection result according to the non-maximum suppression algorithm to obtain the detection results of the original image.

[0115] The non-maximum suppression algorithm may refer to a Non-Maximum Suppression (NMS algorithm), which refers to an algorithm for searching for local maximum values ​​and suppressing maximum values.

[0116] In this embodiment, the prediction frames of the images to be detected may overlap, that is, a large number of prediction frames will be generated at the position of the same target in this solution, and these prediction frames may overlap with each other. At this time, we need to use non-maximum suppression to find the best prediction frame and eliminate redundant prediction frames.

[0117] Specifically, all target detection results can be directly merged to obtain a merged detection result, which includes a large number of prediction boxes. Then, the NMS algorithm is used to filter out redundant prediction boxes, leaving the best prediction box for each target. These best prediction boxes constitute the detection result of the original image.

[0118] The technical solution of the embodiment of the present application removes redundant prediction boxes in the merged prediction results by taking the NMS maximum value, selects the best prediction box, and improves the accuracy of the detection results of the original image.

[0119] See Figure 8 Some embodiments of the present application also propose a method for training a target detection network, wherein the method for training a target detection network is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including but not limited to the following steps:

[0120] Step 802: Obtain a training image set and a ground truth box corresponding to each target to be detected in each training image in the training image set.

[0121] The training image set may refer to an image set used to train the original detection network to obtain an object detection network. The training image set includes multiple training images, and each of the training images in the training image set has a ground truth box labeled with the object to be detected. The training image set may be stored in server 104. The training image set may be the VisDrone2019 dataset or other image sets, and this application does not impose any specific limitations on this.

[0122] For example, a training image set and a real frame corresponding to each target to be detected in each training image may be obtained from a data storage system stored in the server 104 .

[0123] In step 804, the training image is input into the original detection network improved based on YOLOv5, wherein the original detection network includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module.

[0124] The original detection network can refer to an untrained target detection network, whose structure and connection relationship between each network layer are as follows: Figure 3 As shown in Figure 1, it includes a feature extraction module and a feature fusion module. The output of the feature extraction module serves as the input of the feature fusion module. The original detection network can be improved by the YOLOv5 network.

[0125] For example, the training image set can be divided into a training set (80%), a validation set (10%) and a test set (10%), and then the training images in the training set are input into Figure 3 In the target detection network shown, the target detection network is trained.

[0126] In step 806, feature extraction is performed on the image to be detected through multiple key feature extraction layers of the feature extraction module to obtain target backbone features of multiple scales in sequence. As the network depth increases, the target backbone features carry less detailed information about the image to be detected and more semantic information about the image to be detected.

[0127] Among them, the key feature extraction layer can refer to Figure 3The network layer where the C3 module is located. For example, the C3 module corresponding to the 4th layer network, the C3 module corresponding to the 6th layer network, and the C3 module corresponding to the 8th layer network. The connection method of the key feature extraction layer is as follows Figure 3 shown.

[0128] For example, the feature extraction module extracts features from the image to be detected, and obtains the output of each key feature extraction layer to obtain the corresponding target backbone features. As the scale of the target backbone features decreases (the depth of the network increases), the detailed information about the image to be detected that it carries decreases, and the semantic information about the image to be detected that it carries increases.

[0129] Step 808 : Weighting the target backbone features at multiple scales to obtain weighted backbone features.

[0130] The weighted backbone feature may refer to a feature obtained by weighting the target backbone feature according to a weight coefficient.

[0131] The weight coefficient is pre-set and can be modified according to actual conditions. For example, it can be set according to the scale of the target trunk feature and the scale of the target to be detected. For example, when detecting a smaller-scale target, the weight coefficient corresponding to the target trunk feature with a larger scale can be set larger, and the weight coefficient corresponding to the target trunk feature with a smaller scale can be set smaller, so that the small detection head of the target detection network pays more attention to the detail information. For example, the target trunk feature output by the C3 module corresponding to the 4th layer network has its corresponding weight coefficient named weight3, the target trunk feature output by the C3 module corresponding to the 6th layer network has its corresponding weight coefficient named weight4, and the target trunk feature output by the C3 module corresponding to the 8th layer network has its corresponding weight coefficient named weight5. When the image to be detected contains a large number of small targets, weight3 can be set to 2, weight4 to 1, and weight5 to 0.5. When the image to be detected contains almost no small targets or is basically a large target, all weight coefficients can be set to 0. In addition, other settings are also possible, and no longer given examples here.

[0132] Exemplarily, the target backbone features of each designated shallow layer output are weighted according to a pre-set weight coefficient to obtain a weighted backbone feature.

[0133] For example, the weight coefficient corresponding to the C3 module corresponding to the 4th layer network can be set to 2, and then the target backbone features output by the C3 module corresponding to the 4th layer network are weighted according to the weight coefficient to obtain the corresponding weighted backbone features. The weight coefficient corresponding to the C3 module corresponding to the 6th layer network can be set to 1, and then the target backbone features output by the C3 module corresponding to the 6th layer network are weighted according to the weight coefficient to obtain the corresponding weighted backbone features. The weight coefficient corresponding to the C3 module corresponding to the 8th layer network can be set to 0.5, and then the target backbone features output by the C3 module corresponding to the 8th layer network are weighted according to the weight coefficient to obtain the corresponding weighted backbone features.

[0134] In step 810 , the weighted backbone features of multiple scales are respectively input into the key fusion nodes of the corresponding levels in the feature fusion module, and are fused again with the original fusion features in the feature fusion module to obtain the target fusion features.

[0135] Among them, the key fusion nodes of the corresponding levels in the feature fusion module can refer to the Concat modules of pre-set levels, such as the 20th layer (belonging to the detection subnetwork), the 23rd layer, the 26th layer, and the 29th layer.

[0136] For example, the weighted backbone features output by the C3 module (key feature extraction layer) corresponding to the 4th layer of the network can be input into the Concat module of the 23rd layer for fusion, the weighted backbone features output by the C3 module (key feature extraction layer) corresponding to the 6th layer of the network can be input into the Concat module of the 26th layer for fusion, and the weighted backbone features output by the C3 module (key feature extraction layer) corresponding to the 8th layer of the network can be input into the Concat module of the 29th layer for fusion. The features output by these Concat modules after re-fusion have more detailed information.

[0137] In step 812, the target fusion feature is input into the detection head, and the output of the detection head is obtained to obtain the prediction box corresponding to the training image.

[0138] A prediction box can be used to represent an object in a training image. When there is only one object in a training image, there is only one prediction box. When there are multiple objects in a training image, there are multiple prediction boxes.

[0139] In step 814, the original detection network is adjusted according to the difference between the predicted box and the real box corresponding to the training image until the original detection network converges to obtain the target detection network.

[0140] Specifically, a loss function can be established based on the difference between the actual true frame of each target to be detected and the predicted frame, and then the original detection network can be adjusted according to the loss function to obtain a trained target detection network.

[0141] The technical solution of the embodiment of the present application is to input the weighted backbone features of multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module respectively, and fuse them with the original fusion features in the feature fusion module, so that the backbone features carrying more detail information and the original fusion features carrying richer semantic information are fused again. The fused features have more detail information, and the weighting mechanism of the target backbone features enables the small detection head of the target detection network to pay more attention to the detail information of the image to be detected, thereby facilitating the identification of small targets in the image to be detected, avoiding the target detection network from missing small targets in the image to be detected, and improving the accuracy of target detection.

[0142] It should be noted that when the original detection network also includes a detection subnetwork, the corresponding processing process is similar to the aforementioned embodiment. The weighted high-resolution features obtained by weighted processing of the high-resolution features and the fused features output by the corresponding FPN structure of YOLOv5 are the inputs of the detection subnetwork; the output of the detection subnetwork also serves as one of the new inputs of the feature fusion module. The training method of the target detection network also includes: obtaining the high-resolution features output by the shallowest key feature extraction layer in the feature extraction module; weighting the high-resolution features according to the weight coefficient to obtain weighted high-resolution features; inputting the weighted high-resolution features and the fused features output by the corresponding FPN structure of YOLOv5 into the detection subnetwork to obtain high-detail detection features; and inputting the high-detail features into the corresponding high-resolution detection head and feature fusion module.

[0143] The specific process and Figure 4 and Figure 5 The embodiment shown is similar and will not be repeated here. The processing process of the corresponding predicted frame is also to adjust the target detection network through the difference between the real frame and the predicted frame until the target detection network converges to obtain the target detection network.

[0144] See Figures 9 to 13 In some embodiments, the step of “adjusting the target detection network according to the difference between each true box and the predicted box” includes but is not limited to the following steps:

[0145] Step 902: Calculate the angle loss based on the angle difference between the real frame and the predicted frame.

[0146] The angle difference between the real box and the predicted box can be seen in Figure 10 , predicted box B and true box B GT The angle formed is Figure 10 For the α and β shown in , when α≤π / 4, the convergence process first minimizes α, otherwise it minimizes β first.

[0147] The definition of angle loss is as follows:

[0148]

[0149]

[0150]

[0151] In formulas (1) to (4), c h is the height difference between the center point of the real box and the predicted box, Λ is the angle loss, σ is the distance between the center point of the real box and the predicted box, is the coordinate of the center point of the real frame, (b cx ,b cy ) is the coordinate of the center point of the predicted box. It can be noted that when α is π / 2 or 0, the angle loss is 0.

[0152] The angle loss can be calculated using the above formulas (1) to (4).

[0153] In step 904, a distance loss is calculated based on the distance difference between the real frame and the predicted frame.

[0154] Among them, the distance difference between the real box and the predicted box is as follows Figure 11 As shown by Figure 11 It can be seen that when α→0, the contribution of distance loss is greatly reduced. On the contrary, the closer α is to π / 4, the greater the contribution of distance loss.

[0155] The distance loss definition can be calculated by the following formulas (5) and (6):

[0156]

[0157]

[0158] In formula (5) and formula (6), Δ represents the distance loss, (c w , c h ) is the width and height of the bottom bounding rectangle of the predicted box and the true box.

[0159] For example, the angle loss Λ can be calculated using formulas (1) to (4), and then the distance loss Δ can be calculated based on the angle loss Λ, formulas (5), and (6).

[0160] Step 906: Calculate the shape loss based on the scale information between the real box and the predicted box.

[0161] Among them, the scale information between the predicted box and the real box is as follows Figure 12 As shown, the definition of shape loss can be expressed by formula (7) and formula (8).

[0162] Ω=∑ t＝w,h (1-e -wt ) θ (7)

[0163]

[0164] In formula (7) and formula (8), Ω represents the shape loss, (w, h) represents the width and height of the prediction box, (w gt , h gt ) represents the width and height of the real box, and θ controls the degree of attention to the shape loss. In order to avoid paying too much attention to the shape loss and reducing the movement of the predicted box, the value range of θ can be [2,6], and θ can be 4.

[0165] Exemplarily, the shape loss can be calculated using formula (7) and formula (8).

[0166] Step 908: Calculate the overlap area loss based on the overlap area between the true box and the predicted box.

[0167] The overlapping area can be Figure 13 As shown, the overlap area loss can be defined by formula (9):

[0168]

[0169] Among them, IOU can refer to the overlapping area loss, B GT It can refer to the area of ​​the real box, and B can refer to the area of ​​the predicted box.

[0170] For example, the overlap area loss IOU can be calculated using the above formula (9).

[0171] In step 910 , a total loss is calculated based on the angle loss, distance loss, shape loss, and overlap area loss, and the original detection network is adjusted based on the total loss.

[0172] Specifically, the total loss can be expressed as follows:

[0173]

[0174] The total loss can be calculated by formula (10), and then the original detection network is adjusted according to the total loss through the back propagation algorithm until the original detection network converges to obtain the target detection network.

[0175] The technical solution of the embodiment of the present application, by adopting SIOU Loss as the total loss function, can facilitate the acquisition of the difference between the predicted box and the real box, thereby facilitating the adjustment of the original detection network to obtain the target detection network, thereby improving the accuracy of target detection by the subsequent target detection network.

[0176] The technical solution of the embodiment of the present application is that when training the original detection network, the graphics card used by the training platform is Tesla-P100, the training epoch is 300, and imgsz is set to 640, and finally the weight file corresponding to the trained target detection network is obtained.

[0177] On the test set, the original YOLOv5 network, YOLOv5-Snet network (introducing the detection subnetwork), YOLOv5-SIOU network (using SIOU Loss as the total loss function), YOLOv5-AW network (not using SIOU Loss as the total loss function, introducing weighted backbone features and detection subnetwork) and YOLOv5-AW-SIoU network (using SIOU Loss as the total loss function, introducing weighted backbone features and detection subnetwork) were compared. The results are shown in Table 1 below.

[0178] Table 1

[0179] mAP_0.5 The epoch used to reach the optimal value YOLOv5 0.41473 YOLOv5-SIOU 0.42830 YOLOv5-Snet 0.44166 YOLOv5-AW 0.46207 232 YOLOv5-AW-SIOU 0.46668 202

[0180] mAP_0.5 is the model's mAP value calculated when the IoU value is 0.5. mAP stands for meanAveragePrecision, which is calculated for each class and then averaged. This is a statistical concept in object detection, generally used to measure model performance. A smaller epoch indicates faster network training and a faster rate of reaching the optimal solution.

[0181] As shown in Table 1, the YOLOv5 network that uses SIOU Loss as the total loss function and introduces weighted backbone features and detection sub-networks significantly improves the detection efficiency of the original YOLOv5 network for smaller targets, and the MAP is increased by more than 5%.

[0182] See Figure 14 Some embodiments of the present application also provide a target detection method, including but not limited to the following steps:

[0183] Step 1402 : Obtain a training image set and a ground truth box corresponding to each target to be detected in each training image in the training image set.

[0184] In step 1404, the training image is input into the original detection network, where the original detection network includes a feature extraction module, a feature fusion module, and a detection sub-network. The output of the feature extraction module serves as the input of the feature fusion module and the detection sub-network, and the output of the detection sub-network also serves as the input of the feature fusion module.

[0185] In step 1406, feature extraction is performed on the training image through multiple key feature extraction layers of the feature extraction module, and multiple target backbone features output by each key feature extraction layer are obtained in turn. As the network depth increases, the detailed information about the image to be detected carried by the corresponding target backbone features decreases, and the semantic information about the image to be detected carried increases.

[0186] In step 1408 , target backbone features at multiple scales are weighted to obtain weighted backbone features at multiple scales.

[0187] In step 1410 , the weighted backbone features of multiple scales are respectively input into the key fusion nodes of the corresponding levels in the feature fusion module, and are fused again with the original fusion features in the feature fusion module to obtain the target fusion features.

[0188] In step 1412, the weighted high-resolution features and the fusion features output by the corresponding FPN structure of YOLOv5 are input into the detection subnetwork to obtain high-detail detection features.

[0189] In step 1414, the target fusion features are input into the detection head, the high-detail features are input into the corresponding high-resolution detection head and feature fusion module, and the output of the detection head is obtained to obtain the predicted box of each target to be detected in each training image.

[0190] In step 1416, an angle loss is calculated based on the angle difference between the real frame and the predicted frame.

[0191] In step 1418, a distance loss is calculated based on the distance difference between the true box and the predicted box.

[0192] In step 1420 , the shape loss is calculated based on the scale information between the true box and the predicted box.

[0193] In step 1422 , the overlapping area loss is calculated based on the overlapping area between the true box and the predicted box.

[0194] In step 1424, a total loss is calculated based on the angle loss, distance loss, shape loss, and overlap area loss, and the original detection network is adjusted based on the total loss until the original detection network converges to obtain a target detection network. The target detection network is used to perform target detection.

[0195] Step 1426 , obtaining the original image and the image resolution of the original image.

[0196] Step 1428: Determine whether the image resolution is greater than or equal to the resolution threshold.

[0197] If yes, go to step 1432; if no, go to step 1442.

[0198] Step 1430 , segmenting the original image to obtain multiple images to be detected; wherein the image to be detected and adjacent images to be detected have overlapping areas.

[0199] In step 1432, each image to be detected is input into the target detection network, and target detection results corresponding to the images to be detected belonging to the same original image are obtained.

[0200] Step 1434 , performing coordinate transformation on each target detection result to obtain the position of each target detection result in the original image.

[0201] Step 1436 , merging the target detection results according to their positions in the original image to obtain a merged detection result.

[0202] Step 1438: Filter the merged detection result according to the non-maximum suppression algorithm to obtain the detection result of the original image.

[0203] Step 1440: Input the original image into the target detection network, obtain the output of the target detection network, and obtain the detection result corresponding to the original image.

[0204] It should be noted that the specific steps of steps 1438 to 1440 can be found in Figures 2 to 13 The embodiments shown will not be described in detail here.

[0205] The technical solution of the embodiment of the present application is to input the weighted backbone features of multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module respectively, and fuse them with the original fusion features in the feature fusion module, so that the target backbone features carrying more detail information and the original fusion features carrying richer semantic information are fused again. The fused features have more detail information, and the weighting mechanism of the backbone features enables the small detection head of the target detection network to pay more attention to the detail information of the image to be detected, thereby facilitating the identification of small targets in the image to be detected, avoiding the target detection network from missing small targets in the image to be detected, and improving the accuracy of target detection.

[0206] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0207] Based on the same inventive concept, the present application also provides an object detection device for implementing the above-mentioned object detection method. The implementation solution provided by the device is similar to the implementation solution described in the above-mentioned method.

[0208] In one embodiment, Figure 15 As shown, a target detection device is provided, including: an image acquisition module 1502, an image input module 1504, an extraction module 1506, a weighting module 1508, a fusion module 1510 and a prediction module 1512, wherein:

[0209] An image acquisition module 1502 is used to acquire an image to be detected;

[0210] An image input module 1504 is configured to input an image to be detected into an object detection network improved based on YOLOv5, wherein the object detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module;

[0211] Extraction module 1506 is configured to extract features from the image to be detected through multiple key feature extraction layers of the feature extraction module, and sequentially obtain target backbone features of multiple scales. The target backbone features refer to the key image features obtained sequentially in the connection order in the feature extraction module. As the network depth increases, the corresponding target backbone features carry less detailed information about the image to be detected and more semantic information about the image to be detected.

[0212] A weighting module 1508 is used to weight target backbone features at multiple scales to obtain weighted backbone features at multiple scales;

[0213] The fusion module 1510 is used to input the weighted backbone features of multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module, and fuse them with the original fusion features in the feature fusion module to obtain the target fusion features;

[0214] The prediction module 1512 is used to input the target fusion feature into the detection head, obtain the output of the detection head, obtain the prediction box corresponding to the image to be detected, and determine the target detection result of the image to be detected based on the prediction box.

[0215] In some embodiments, the target detection device further includes:

[0216] The high-resolution feature acquisition module is used to obtain the target backbone features of the largest scale output by the feature extraction module to obtain high-resolution features.

[0217] The high-resolution feature weighting module is used to weight the high-resolution features according to the weight coefficient to obtain weighted high-resolution features.

[0218] The detection subnetwork input module is used to input the fusion features of the weighted high-resolution features and the output of the corresponding FPN structure of YOLOv5 into the detection subnetwork to obtain high-detail detection features.

[0219] The high-resolution feature fusion module is used to input high-detail features into the corresponding high-resolution detection head and feature fusion module.

[0220] In some embodiments, the target detection device further includes:

[0221] The original image acquisition module is used to acquire the original image and the image resolution of the original image.

[0222] The segmentation module is used to segment the original image if the image resolution is greater than or equal to the resolution threshold to obtain multiple images to be detected; wherein the image to be detected and the adjacent images to be detected have overlapping areas.

[0223] In some embodiments, the target detection device further includes:

[0224] The target detection result acquisition module is used to obtain the target detection results corresponding to the image to be detected belonging to the same original image.

[0225] The coordinate conversion module is used to convert the coordinates of each target detection result to obtain the position of each target detection result in the original image;

[0226] The detection result determination module is used to determine the detection result of the original image according to the position of each target detection result in the original image.

[0227] In some embodiments, the detection result determination module is also used to merge the target detection results according to their positions in the original image to obtain a merged detection result; and filter the merged detection result according to a non-maximum suppression algorithm to obtain a detection result of the original image.

[0228] like Figure 16 As shown, some embodiments of the present application further provide a training device for an object detection network, including:

[0229] The training data acquisition module 1602 is used to obtain a training image set and a ground truth box corresponding to each target to be detected in each training image in the training image set;

[0230] A training image input module 1604 is used to input the training image into the original detection network improved based on YOLOv5, wherein the original detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module;

[0231] A training image extraction module 1606 is configured to extract features from the image to be detected using multiple key feature extraction layers of the feature extraction module, sequentially obtaining target backbone features at multiple scales. The target backbone features are key image features obtained sequentially in the feature extraction module according to the connection order. As the network depth increases, the corresponding target backbone features carry less detailed information about the training image and more semantic information about the training image.

[0232] The training image feature weighting module 1608 is used to weight target backbone features of multiple scales to obtain weighted backbone features of multiple scales;

[0233] The training image feature fusion module 1610 is used to input the weighted backbone features of multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module, and fuse them with the original fusion features in the feature fusion module to obtain the target fusion features;

[0234] The training image prediction module 1612 is used to input the target fusion features into the detection head, obtain the output of the detection head, and obtain the prediction box corresponding to the training image.

[0235] The training adjustment module 1614 is used to adjust the original detection network according to the difference between the predicted box and the real box corresponding to the training image until the original detection network converges to obtain the target detection network.

[0236] In some embodiments, the training adjustment module is further used to calculate the angle loss based on the angle difference between the real box and the predicted box; calculate the distance loss based on the distance difference between the real box and the predicted box; calculate the shape loss based on the scale information between the real box and the predicted box; calculate the overlapping area loss based on the overlapping area between the real box and the predicted box; calculate the total loss based on the angle loss, distance loss, shape loss and overlapping area loss, and adjust the target detection network based on the total loss.

[0237] Each module in the above-mentioned object detection device or object detection network training device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in the form of software in a memory in the computer device, so that the processor can call and execute the corresponding operations of each of the above modules.

[0238] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 17 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a target detection method or a training method for a target detection network is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0239] Those skilled in the art will understand that Figure 17The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0240] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the above-mentioned target detection method or the training method of the target detection network are implemented.

[0241] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the target detection method or the training method of the target detection network are implemented.

[0242] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of a target detection method or a training method of a target detection network.

[0243] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0244] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0245] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An improved target detection method based on YOLOv5, characterized in that: The method comprises: Obtain the image to be detected; Inputting the image to be detected into an object detection network improved based on YOLOv5, wherein the object detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module; Performing feature extraction on the image to be detected through multiple key feature extraction layers of the feature extraction module to sequentially obtain target backbone features of multiple scales, wherein the target backbone features refer to key image features sequentially obtained in the feature extraction module according to a connection order, and as the network depth increases, the corresponding target backbone features carry less detail information about the image to be detected and carry more semantic information about the image to be detected; Performing weighted processing on the target backbone features of the multiple scales respectively to obtain weighted backbone features of the multiple scales; Inputting the weighted backbone features of the multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module respectively, and fusing them again with the original fusion features in the feature fusion module to obtain the target fusion features; The target fusion feature is input into the detection head, and the output of the detection head is obtained to obtain the prediction box corresponding to the image to be detected, and the target detection result of the image to be detected is determined according to the prediction box.

2. The method according to claim 1, characterized in that The target detection network further includes a detection subnetwork, the output of the feature extraction module serves as the input of the detection subnetwork, and the output of the detection subnetwork serves as the input of the feature fusion module; The method further comprises: Obtaining the maximum-scale target backbone feature output by the feature extraction module to obtain a high-resolution feature; Weighting the high-resolution features according to the weight coefficients to obtain weighted high-resolution features; Input the weighted high-resolution features and the fusion features output by the corresponding FPN structure of YOLOv5 into the detection subnetwork to obtain high-detail detection features; The high-detail detection features are input into the corresponding high-resolution detection head and the feature fusion module.

3. The method according to claim 1 or 2, characterized in that Before acquiring the image to be detected, the method further includes: Acquire an original image and an image resolution of the original image; If the image resolution is greater than or equal to the resolution threshold, the original image is segmented to obtain a plurality of images to be detected; wherein, the image to be detected and adjacent images to be detected have overlapping areas.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining target detection results corresponding to the image to be detected belonging to the same original image; Performing coordinate transformation on each of the target detection results to obtain a position of each of the target detection results in the original image; The detection result of the original image is determined according to the position of each target detection result in the original image.

5. The method according to claim 4, characterized in that Determining the detection result of the original image according to the position of each target detection result in the original image includes: Merging the target detection results according to their positions in the original image to obtain a merged detection result; The combined detection result is screened according to a non-maximum suppression algorithm to obtain a detection result of the original image.

6. A method for training a target detection network, characterized in that: The method comprises: Obtaining a training image set and a ground truth box corresponding to each target to be detected in each training image in the training image set; Inputting the training image into an original detection network improved based on YOLOv5, wherein the original detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module; Performing feature extraction on the image to be detected through multiple key feature extraction layers of the feature extraction module to sequentially obtain target backbone features of multiple scales, wherein the target backbone features refer to key image features obtained sequentially in the feature extraction module according to the connection order. As the network depth increases, the corresponding target backbone features carry less detailed information about the training image and more semantic information about the training image; Performing weighted processing on the target backbone features of the multiple scales respectively to obtain weighted backbone features of the multiple scales; Inputting the weighted backbone features of the multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module respectively, and fusing them again with the original fusion features in the feature fusion module to obtain the target fusion features; Inputting the target fusion feature into the detection head, obtaining the output of the detection head, and obtaining the prediction box corresponding to the training image; According to the difference between the predicted box corresponding to the training image and the real box, the original detection network is adjusted until the original detection network converges to obtain the object detection network.

7. The method according to claim 6, characterized in that The adjusting the original detection network according to the difference between the predicted box corresponding to the training image and the true box includes: Calculate the angle loss based on the angle difference between the real frame and the predicted frame; Calculate the distance loss based on the distance difference between the real frame and the predicted frame; Calculating a shape loss based on scale information between the true frame and the predicted frame; Calculate the overlap area loss based on the overlap area between the true box and the predicted box; A total loss is calculated according to the angle loss, distance loss, shape loss, and overlap area loss, and the original detection network is adjusted according to the total loss.

8. A target detection device, characterized in that: The device comprises: An image acquisition module, used to acquire an image to be detected; An image input module is used to input the image to be detected into a target detection network improved based on YOLOv5, wherein the target detection network improved based on YOLOv5 includes a feature extraction module and a feature fusion module, and the output of the feature extraction module serves as the input of the feature fusion module; An extraction module is configured to extract features from the image to be detected through multiple key feature extraction layers of the feature extraction module, and sequentially obtain target backbone features of multiple scales, wherein the target backbone features refer to key image features obtained sequentially in the feature extraction module according to the connection order, and as the network depth increases, the corresponding target backbone features carry less detailed information about the image to be detected and more semantic information about the image to be detected; A weighting module, configured to perform weighted processing on the target backbone features of the multiple scales to obtain weighted backbone features of the multiple scales; A fusion module is used to input the weighted backbone features of the multiple scales into the key fusion nodes of the corresponding levels in the feature fusion module, and fuse them with the original fusion features in the feature fusion module to obtain the target fusion features; The prediction module is used to input the target fusion feature into the detection head, obtain the output of the detection head, obtain the prediction box corresponding to the image to be detected, and determine the target detection result of the image to be detected based on the prediction box.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the processor implements the steps of the method according to any one of claims 1 to 5, or implements the steps of the method according to any one of claims 6 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented, or the steps of the method according to any one of claims 6 to 7 are implemented.

Images