Target detection method and device based on YOLO architecture

Abstract

The invention discloses a target detection method and device based on a YOLO architecture, and the method comprises the steps: carrying out the target detection of a labeled image and a non-labeled image based on an initial student model, and obtaining a student detection result of the labeled image and a student detection result of the non-labeled image; classifying the unlabeled images based on a teacher model to obtain false labels of the unlabeled images; scoring the pseudo labels based on a pseudo label distributor; determining a first supervised loss using the first supervised loss function; determining unsupervised loss by using an unsupervised loss function; determining a total loss based on the first supervised loss and the unsupervised loss; iteratively training the initial student model and the teacher model based on the total loss until the total loss is smaller than a second preset loss value, and determining the initial student model at the moment as a target student model; and inputting the to-be-recognized image into the target student model to obtain a recognition result of the to-be-recognized image. According to the invention, the training efficiency of the model and the detection accuracy of target detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a target detection method and device based on the YOLO architecture. Background Art

[0002] Traditional machine learning target detection and recognition algorithms require manual design and selection of appropriate features. The algorithm process design is complex and has poor adaptability to target changes, making it difficult to use in complex scenarios. However, deep convolutional neural networks have been proven to be effective in supervised learning training on large amounts of labeled data. This end-to-end deep convolutional neural network method can automatically extract useful features from data. After iterative training, the model can effectively detect and identify specific targets in complex scenarios.

[0003] The existing space target detection and recognition algorithms have the following shortcomings:

[0004] 1. Learning methods based on deep convolutional neural networks rely on large quantities of high-quality manual labels and sample data. However, it is difficult to collect large quantities of spatial target image data in the real world through visible light cameras. At the same time, it takes a lot of resources and time to annotate all the collected data with high quality.

[0005] 2. When the amount of sample data is small, it is difficult for the deep neural network to learn the effective features of the target from a small amount of data, resulting in poor training results of the model and very limited generalization ability in real scenarios.

[0006] That is, the existing technology leads to low model training efficiency and low target detection accuracy. Summary of the invention

[0007] The embodiments of the present application provide a target detection method and device based on the YOLO architecture, which can improve the training efficiency of the model and the detection accuracy of target detection.

[0008] In the first aspect, the target detection method based on the YOLO architecture provided by the present application includes:

[0009] Get labeled images and unlabeled images;

[0010] Performing target detection on the labeled image and the unlabeled image based on the initial student model to obtain a student detection result of the labeled image and a student detection result of the unlabeled image;

[0011] Classifying the unlabeled image based on the teacher model to obtain a pseudo label of the unlabeled image;

[0012] Scoring the pseudo-label of the unlabeled image based on the pseudo-label assigner to obtain a score of the pseudo-label;

[0013] Determine a first supervised loss based on the student detection result of the labeled image and the true label of the labeled image using a first supervised loss function;

[0014] Determining an unsupervised loss based on the pseudo-label of the unlabeled image and the student detection result of the unlabeled image using an unsupervised loss function;

[0015] determining a total loss based on the first supervised loss and the unsupervised loss;

[0016] Iteratively training the initial student model and the teacher model based on the total loss until the total loss is less than a second preset loss value, and determining the initial student model at this time as a target student model;

[0017] The image to be recognized is input into the target student model to obtain a recognition result of the image to be recognized.

[0018] Optionally, the target detection method based on the YOLO architecture includes:

[0019] Obtain a YOLOv5 model, wherein the YOLOv5 model includes a feature extraction backbone network, a Neck layer, and a detection head;

[0020] A dilated spatial convolutional pooling pyramid module is added to the feature extraction backbone network of the YOLOv5 model, the C3 module in the Neck layer of the YOLOv5 model is replaced with a C2f module, and the detection head of the YOLOv5 model is replaced with three parallel first detection heads, second detection heads and third detection heads, wherein the first detection head is used for classification, the second detection head is used for regression, and the third detection head is used to detect whether an area contains a target, so as to obtain an initial student model.

[0021] Optionally, performing target detection on the labeled image and the unlabeled image based on the initial student model to obtain a student detection result of the labeled image and a student detection result of the unlabeled image includes:

[0022] Get a domain adaptive classifier;

[0023] Using a second supervised loss function to train the initial student model and the domain adaptive classifier based on the labeled image and the unlabeled image to obtain an aging training model;

[0024] Based on the aging training model, target detection is performed on the labeled image and the unlabeled image to obtain a student detection result of the labeled image and a student detection result of the unlabeled image.

[0025] Optionally, the teacher model is obtained by updating the aging training model EMA.

[0026] Optionally, the student detection result of the labeled image includes the classification result of the labeled image output by the first detection head, the detection frame result of the labeled image output by the second detection head, and the detection confidence result of the labeled image output by the third detection head; the real label of the labeled image includes the real classification result of the labeled image, the real detection frame result of the labeled image, and the real confidence result of the labeled image, and the first supervised loss function satisfies the following formula,

[0027]

[0028] In the first supervised loss function, (h,w) represents the pixel point on the image with a height of h and a width of w. is the classification result of the labeled image output by the first detection head, represents the true classification result of the labeled image, and are respectively the detection frame result of the labeled image output by the second detection head and the detection confidence result of the labeled image output by the third detection head, and are the true detection box result of the labeled image and the true confidence result of the labeled image, respectively. CE represents the cross entropy loss. represents the CIoU calculation value between the detection box result of the labeled image output by the second detection head and the real detection box result of the labeled image, L s is the first supervised loss.

[0029] Optionally, the student detection result of the unlabeled image includes the classification result of the unlabeled image output by the first detection head, the detection frame result of the unlabeled image output by the second detection head, and the detection confidence result of the unlabeled image output by the third detection head; the real label of the unlabeled image includes the real classification result of the unlabeled image, the real detection frame result of the unlabeled image, and the real confidence result of the unlabeled image, and the unsupervised loss function satisfies the following formula,

[0030]

[0031]

[0032] Among them, in the unsupervised loss function, L u is an unsupervised loss, (h,w) represents the pixel point on the image with a height of h and a width of w. is the classification result of the unlabeled image output by the first detection head, and are respectively the detection frame result of the unlabeled image output by the second detection head and the detection confidence result of the unlabeled image output by the third detection head, are the classification result, regression result and detection confidence result of the pseudo label of the unlabeled image at the position (h, w), P (h,w) is the score of the pseudo-label at (h,w), 1 {·} is a discriminant function. If the condition {·} is met, the output is 1, otherwise the output is 0. τ1 is the first preset threshold, τ2 is the second preset threshold, and the first preset threshold is less than the second preset threshold.

[0033] Optionally, determining the total loss based on the first supervised loss and the unsupervised loss includes:

[0034] The first supervised loss and the unsupervised loss are weightedly summed based on a first preset weight coefficient and a second preset weight coefficient to obtain a total loss.

[0035] In the second aspect, the target detection device based on the YOLO architecture provided by the present application includes:

[0036] An acquisition module is used to acquire labeled images and unlabeled images;

[0037] A detection module, used for performing target detection on the labeled image and the unlabeled image based on an initial student model to obtain a student detection result of the labeled image and a student detection result of the unlabeled image;

[0038] A classification module, used for classifying the unlabeled image based on a teacher model to obtain a pseudo label of the unlabeled image;

[0039] A scoring module, used for scoring the pseudo-label of the unlabeled image based on the pseudo-label assigner to obtain a score of the pseudo-label;

[0040] A first determination module, configured to determine a first supervised loss based on a student detection result of the labeled image and a true label of the labeled image using a first supervised loss function;

[0041] A second determination module is used to determine the unsupervised loss based on the pseudo label of the unlabeled image and the student detection result of the unlabeled image using an unsupervised loss function;

[0042] A third determination module, configured to determine a total loss based on the first supervised loss and the unsupervised loss;

[0043] A training module, configured to iteratively train the initial student model and the teacher model based on the total loss until the total loss is less than a second preset loss value, and determine the initial student model at this time as a target student model;

[0044] The recognition module is used to input the image to be recognized into the target student model to obtain the recognition result of the image to be recognized.

[0045] In a third aspect, the electronic device provided in the present application includes a memory and a processor, the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the target detection method based on the YOLO architecture provided in the present application.

[0046] In a fourth aspect, the computer-readable storage medium provided in the present application stores a plurality of instructions, which are suitable for loading by a processor to implement the steps in the target detection method based on the YOLO architecture provided in the present application.

[0047] In a fifth aspect, the computer program product provided in the present application includes a computer program or instructions, which, when executed by a processor, implements the steps in the target detection method based on the YOLO architecture provided in the present application.

[0048] In the present application, compared with the related art, labeled images and unlabeled images are obtained; target detection is performed on the labeled images and the unlabeled images based on the initial student model to obtain student detection results of the labeled images and the unlabeled images; the unlabeled images are classified based on the teacher model to obtain pseudo labels of the unlabeled images; the pseudo labels of the unlabeled images are scored based on the pseudo label allocator to obtain scores of the pseudo labels; a first supervised loss is determined based on the student detection results of the labeled images and the true labels of the labeled images using a first supervised loss function; an unsupervised loss is determined based on the pseudo labels of the unlabeled images, the scores of the pseudo labels, and the student detection results of the unlabeled images using an unsupervised loss function; a total loss is determined based on the first supervised loss and the unsupervised loss; the initial student model and the teacher model are iteratively trained based on the total loss until the total loss is less than a second preset loss value, and the initial student model at this time is determined as the target student model; the image to be recognized is input into the target student model to obtain a recognition result of the image to be recognized. This application adopts a combination of supervised and unsupervised methods, which performs better than conventional supervised learning algorithms, can improve the accuracy of target detection, and does not rely on large quantities of high-quality manual labels and sample data, reducing the difficulty of collecting image data and the resources and time for high-quality annotation of all image data, thereby improving the training efficiency of the model and the detection accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 Schematic diagram of a scene of a target detection system based on the YOLO architecture provided in an embodiment of the present application;

[0051] Figure 2 It is a flowchart of an embodiment of a target detection method based on the YOLO architecture provided in an embodiment of the present application;

[0052] Figure 3 It is a schematic diagram of the overall framework of an embodiment of a target detection method based on the YOLO architecture provided in an embodiment of the present application;

[0053] Figure 4 It is a structural schematic diagram of a dilated spatial convolution pooling pyramid module of an embodiment of a target detection method based on a YOLO architecture provided in an embodiment of the present application;

[0054] Figure 5It is a structural diagram of a C2f module of an embodiment of a target detection method based on a YOLO architecture provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structures of the first detection head, the second detection head, and the third detection head of an embodiment of the target detection method based on the YOLO architecture provided in an embodiment of the present application;

[0056] Figure 7 is a schematic diagram of the structure of a target detection device based on the YOLO architecture provided in an embodiment of the present application;

[0057] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] It should be noted that the principles of the present application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application and should not be considered as limiting other specific embodiments of the present application that are not described in detail herein.

[0059] In the following description of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0060] In the following description of the present application, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0062] In order to improve the effect of target detection based on the YOLO architecture, the embodiments of the present application provide a target detection method based on the YOLO architecture, a target detection device based on the YOLO architecture, an electronic device, a computer-readable storage medium, and a computer program product. Among them, the target detection method based on the YOLO architecture can be executed by the target detection device based on the YOLO architecture, or by an electronic device integrated with the target detection device based on the YOLO architecture.

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0064] Please refer to Figure 1 , this application also provides a target detection system based on the YOLO architecture, such as Figure 1 As shown, the target detection system based on the YOLO architecture includes an electronic device 100, and the electronic device 100 is integrated with the target detection device based on the YOLO architecture provided by the present application.

[0065] Among them, the electronic device 100 can be any device equipped with a processor and has processing capabilities, such as mobile electronic devices with processors such as smart phones, tablet computers, PDAs, laptops, smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, industrial equipment, etc.

[0066] In addition, if Figure 1 As shown, the target detection system based on the YOLO architecture may also include a memory 200 for storing original data, intermediate data, and result data.

[0067] In the embodiment of the present application, the memory may be a cloud memory. Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0068] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0069] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disks), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0070] It should be noted that Figure 1 The scenario diagram of the target detection system based on the YOLO architecture shown is only an example. The target detection system and scenario based on the YOLO architecture described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can know that with the evolution of the target detection system based on the YOLO architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0071] It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0072] Please refer to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of a target detection method based on the YOLO architecture provided in an embodiment of the present application. Figure 2 As shown, the process of the target detection method based on the YOLO architecture provided in this application is as follows:

[0073] 201. Obtain labeled images and unlabeled images.

[0074] Specifically, Figure 3 As shown, unlabeled data is obtained, and Mosaic enhancement and Mixup enhancement are performed on the unlabeled data to obtain an unlabeled image. Labeled data is obtained, and Mosaic enhancement and Mixup enhancement are performed on the labeled data to obtain a labeled image.

[0075] in, Figure 3 The student models in represent the initial student model, the aged training model and the target student model at different training stages.

[0076] 202. Perform target detection on labeled images and unlabeled images based on the initial student model to obtain student detection results of labeled images and student detection results of unlabeled images.

[0077] (1) Obtain the YOLOv5 model.

[0078] The YOLOv5 model was publicly released by Ultralytics on June 9, 2020. The YOLOv5 model is an improvement on the YOLOv3 model, and there are four models: YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. The YOLOv5 model consists of a backbone network, a Neck layer, and a detection head.

[0079] The backbone network of the YOLOv5 model is mainly composed of Focus, BottleneckCSP and SSP networks, which mainly include Focus, Conv convolutional blocks, BottleneckCSP and SSP modules. First, input a picture of size 640*640*3, where 3 is the number of convolutions, slice it through the Focus module, reduce the height and width of the image, and output the image size of 320*320. Then, use Concat to integrate the height and width of the sliced ​​image and increase the number of channels of the input image. At this time, the number of image channels is 64.

[0080] Secondly, the integrated image is subjected to feature extraction through a Conv convolution module with a size of 3 and a stride of 2, and the output image size is 160*160*128. Then, the extracted feature map is subjected to three sets of BottleneckCSP1 and Conv convolution operations to obtain a feature map with an image size of 20*20*1024. The SSP module is used to improve the model accuracy for the 20*20 feature map. The SSP module performs 1*1, 5*5, 9*9, and 13*134 maximum pooling operations on the image to extract features from multiple aspects, and the four sets of pooled feature maps are aggregated through Concat.

[0081] BottleneckCSP1 and BottleneckCSP2 used in the Neck layer of YOLOv5 can improve the network speed while ensuring accuracy. The Neck layer of YOLOv5 uses the BottleneckCSP2 module to reduce the number of model parameters. The feature map of size 80*80*512 is upsampled. The upsampling process is completed by connecting 2 groups of BottleneckCSP2, Conv, Upsample and Concat with a size of 1 and a step length of 1. In Concat[i,j], j represents the feature map obtained by the j-th layer operation in the network. The feature map of 80*80*512 is then downsampled to obtain three feature maps of different sizes: 80*80*512, 40*40*512, and 20*20*512.

[0082] The detection head of YOLOv5m uses multi-scale feature maps for detection, using large images to detect small targets and small images to detect large targets. The three different scale feature maps of the Neck layer are subjected to Conv2d convolution operations, and finally three feature maps of sizes 80*80*255, 40*40*255, and 20*20*255 are obtained.

[0083] Generate candidate boxes on three different scale feature maps. Since this paper has only two categories of smoke detection, smoke and non-smoke, three different sizes of candidate boxes are used to predict smoke on the three scale feature maps of YOLOv5m. Therefore, for the self-made data set in Chapter 3, there are a total of 21 candidate boxes. Finally, the target box is screened by weighted non-maximum method. The 21 generated candidate boxes are screened and the target classification and bounding box regression are output.

[0084] Among them, the YOLOv5 model includes a feature extraction backbone network, a Neck layer, and a detection head.

[0085] (2) Add a dilated spatial convolutional pooling pyramid module to the feature extraction backbone network of the YOLOv5 model, replace the C3 module in the Neck layer of the YOLOv5 model with a C2f module, and replace the detection head of the YOLOv5 model with three parallel first detection heads, second detection heads, and third detection heads. The first detection head is used for classification, the second detection head is used for regression, and the third detection head is used to detect whether the area contains the target, thereby obtaining the initial student model.

[0086] like Figure 4 As shown, Figure 4 The Atrous Spatial Pyramid Pooling (ASPP) module is shown in the figure. The Atrous Spatial Pyramid Pooling (ASPP) module is introduced into the bottom design of the feature extraction backbone network of Darknet-53 of YOLOv5. The ASPP module is constructed by multiple parallel atrous convolutions with different sampling rates. Figure 4 As shown in Figure 1, the ASPP module is constructed by multiple parallel dilated convolutions with different sampling rates. The features extracted by each sampling rate are further processed in an independent branch and then fused to generate the final feature representation. This structure maintains the resolution while expanding the receptive field and obtains multi-scale contextual information of the target in the image.

[0087] like Figure 5 As shown, Figure 5 The C2f module is shown in the figure. The C3 module in the Neck layer of the YOLOv5 model is replaced by the C2f module. Its structure is as follows Figure 5As shown in the figure. The C2f module is a multi-level convolutional module for feature fusion. It reduces the number of parameters by stacking multiple Bottlenecks, effectively reduces the computational complexity and consumption of the model, and improves the operating efficiency of the model. The introduced residual connection enables the network to more effectively learn and utilize the correlation information between features, thereby realizing information transmission and fusion between feature maps at different levels. This module design enhances the model's perception of targets of different scales, reduces information loss, and thus improves the detection performance of the target detection algorithm for targets of different scales.

[0088] like Figure 6 As shown in Figure 1, considering that the features that recognition classification and positioning focus on and rely on are different, the detection head design of the YOLOv5 model is improved to a decoupled detection head. The detection head of the YOLOv5 model is replaced by three parallel first detection heads, second detection heads, and third detection heads. The first detection head is used for classification, the second detection head is used for regression, and the third detection head is used to detect whether the area contains the target, as shown in Figure 1. Figure 6 shown.

[0089] Among them, Cls., Reg. and Obj. represent three different loss functions: the first detection head is used for classification, the second detection head is used for regression, and the third detection head is used to detect whether the region contains the target. This structure decouples the input feature fusion layer into different branches for processing, learns the key features of the target location and category information respectively, and then outputs the predicted results. This structural design of different network branches effectively reduces the number of parameters and computational complexity, and enhances the generalization ability and robustness of the model.

[0090] The dense detector in this framework uses the YOLO model as the main architecture, and performs a new optimization design on the detection and recognition model, which enhances the performance and generalization ability of the model, making it more suitable for space target detection scenarios.

[0091] In an embodiment of the present application, target detection is performed on labeled images and unlabeled images based on an initial student model to obtain student detection results of the labeled images and student detection results of the unlabeled images, including: obtaining a domain adaptive classifier; training the initial student model and the domain adaptive classifier based on the labeled images and the unlabeled images using a second supervised loss function to obtain an aging training model; performing target detection on the labeled images and the unlabeled images based on the aging training model to obtain student detection results of the labeled images and student detection results of the unlabeled images.

[0092] Specifically, a second supervised loss is determined based on the student detection result of the labeled image, the true label of the labeled image, and the adaptive classification result of the labeled image by the domain adaptive classifier. When the second supervised loss is not less than the first preset loss value, the domain adaptive classifier and the initial student model are gradient updated. When the second supervised loss is less than the first preset loss value, the initial student model at this time is determined as the aging training model.

[0093] Among them, the student detection results of the labeled images include the classification results of the labeled images output by the first detection head, the detection frame results of the labeled images output by the second detection head, and the detection confidence results of the labeled images output by the third detection head; the real labels of the labeled images include the real classification results of the labeled images, the real detection frame results of the labeled images, and the real confidence results of the labeled images.

[0094] In the embodiment of the present application, an iterative round allocator is used in the semi-supervised learning training, and the training process is divided into two stages: aging training and joint training. First, a certain number of iterations of aging training are performed to enable the model to have a certain pseudo-label generation capability, and an aging training model is obtained. Then, the model is jointly trained using labeled and unlabeled data. In the aging training stage, the model is also allowed to view unlabeled data, reducing feature inconsistencies in the aging training process and speeding up the overall learning process of the model for unlabeled data.

[0095] During joint training, each pair of labeled and unlabeled images is used to train a domain adaptive classifier. The domain adaptive loss function is as follows:

[0096]

[0097] Among them, P (h,w) is the classification result of the domain adaptive classifier. D = 0 is the labeled image, and D = 1 is the unlabeled image. The gradient reversal layer is introduced to use the ordinary gradient descent to train the domain adaptive classifier. When the gradient reversal layer is propagated, the sign of the gradient will be reversed to optimize the network.

[0098] Therefore, the second supervised loss function in the burn-in training phase is as follows:

[0099]

[0100] Among them, in the second supervised loss function, L s is the second supervised loss, (h,w) represents the pixel point with height h and width w on the image, is the classification result of the labeled image output by the first detection head, represents the true classification result of the labeled image, and They are the detection box results of the labeled image output by the second detection head and the detection confidence results of the labeled image output by the third detection head, and They are the true detection box results of the labeled image and the true confidence results of the labeled image, CE represents the cross entropy loss, It represents the CIoU calculation value between the detection box result of the labeled image output by the second detection head and the real detection box result of the labeled image. λ is the hyperparameter of the domain adaptation weight, which defaults to 0.1.

[0101] 203. Classify the unlabeled images based on the teacher model to obtain pseudo labels for the unlabeled images.

[0102] In the embodiment of the present application, the aging training model EMA obtained after aging training according to the initial student model is updated to obtain the teacher model. The exponential moving average (EMA) is an averaging method that gives higher weight to recent data.

[0103] 204. Determine a first supervised loss based on the student detection result of the labeled image and the true label of the labeled image using the first supervised loss function.

[0104] Among them, the student detection results of the labeled images include the classification results of the labeled images output by the first detection head, the detection frame results of the labeled images output by the second detection head, and the detection confidence results of the labeled images output by the third detection head; the real labels of the labeled images include the real classification results of the labeled images, the real detection frame results of the labeled images, and the real confidence results of the labeled images.

[0105] In the embodiment of the present application, the student detection result of the labeled image includes the classification result of the labeled image output by the first detection head, the detection frame result of the labeled image output by the second detection head, and the detection confidence result of the labeled image output by the third detection head; the real label of the labeled image includes the real classification result of the labeled image, the real detection frame result of the labeled image, and the real confidence result of the labeled image. The first supervised loss function satisfies the following formula:

[0106]

[0107] In the first supervised loss function, (h,w) represents the pixel point on the image with a height of h and a width of w. is the classification result of the labeled image output by the first detection head, represents the true classification result of the labeled image, and They are the detection box results of the labeled image output by the second detection head and the detection confidence results of the labeled image output by the third detection head, and They are the true detection box results of the labeled image and the true confidence results of the labeled image, CE represents the cross entropy loss, represents the CIoU calculation value between the detection box result of the labeled image output by the second detection head and the real detection box result of the labeled image, L s is the first supervised loss.

[0108] 205. Score the pseudo-labels of the unlabeled images based on the pseudo-label assigner to obtain pseudo-label scores.

[0109] In the embodiment of the present application, the pseudo-label distributor divides pseudo-labels into two types to alleviate the inconsistency problem of pseudo-labels. Figure 3 As shown, in the pseudo-label allocator, the pseudo-labels obtained after the non-maximum suppression algorithm is processed are divided into two categories: reliable pseudo-label data and uncertain pseudo-label data. The first preset threshold τ1 and the second preset threshold τ2 of the pseudo-label score are used to determine the two types of pseudo-label scores. The first preset threshold τ1 and the second preset threshold τ2 of the pseudo-label score are used to design the unsupervised loss function using uncertain pseudo-labels in the framework.

[0110] The pseudo-label assigner assigns unreliable pseudo-labels as soft labels. The optimization goal is to improve the quality of pseudo labels in semi-supervised learning training. Unreliable pseudo labels are divided into two categories: pseudo labels with high classification scores and pseudo labels with low classification scores. The τ1 and τ2 thresholds are used as the conditional activation of the classification scores. Convert more unreliable pseudo labels into true positives. When iteratively training the model, most uncertain pseudo labels are false positives due to inaccurate prediction boxes. This soft label learning mechanism is used to suppress the inconsistency of pseudo labels without affecting the selection of reliable pseudo labels.

[0111] 206. An unsupervised loss is determined based on the pseudo-labels of the unlabeled images, the scores of the pseudo-labels, and the student detection results for the unlabeled images using an unsupervised loss function.

[0112] In the embodiment of the present application, the unsupervised loss function satisfies the following formula:

[0113]

[0114] Among them, in the unsupervised loss function, L u is an unsupervised loss, (h,w) represents the pixel point on the image with a height of h and a width of w. is the classification result of the unlabeled image output by the first detection head, and They are the detection box results of the unlabeled image output by the second detection head and the detection confidence results of the unlabeled image output by the third detection head, They are the classification results, regression results, and detection confidence results of the pseudo-label of the unlabeled image at position (h, w), P (h,w) is the pseudo-label score of the pseudo-label at (h,w). {·} is a discriminant function. If the condition {·} is met, the output is 1, otherwise the output is 0. τ1 is the first preset threshold, τ2 is the second preset threshold, and the first preset threshold is less than the second preset threshold.

[0115] 207. Determine the total loss based on the first supervised loss and the unsupervised loss.

[0116] In the embodiment of the present application, the first supervised loss and the unsupervised loss are weighted and summed based on the first preset weight coefficient and the second preset weight coefficient to obtain the total loss. For example, the first preset weight coefficient is 1, the second preset weight coefficient is λ, and the calculation formula of the total loss L is as follows:

[0117] L=L s +λL u

[0118] Among them, L is the total loss, L s represents the first supervised loss calculated on the labeled image, L u represents the unsupervised loss calculated on the unlabeled image, λ represents the weight of the unsupervised loss, and λ is used to balance the first supervised loss and the unsupervised loss.

[0119] 208. Iteratively train the initial student model and the teacher model based on the total loss until the total loss is less than a second preset loss value, and determine the initial student model at this time as the target student model.

[0120] Specifically, when the total loss is not less than the second preset loss value, the initial student model and the teacher model are gradient updated, and when the total loss is less than the second preset loss value, the initial student model at this time is determined as the aging training model.

[0121] When the total loss is not less than the second preset loss value, the gradient updates the aged training model and the teacher model. When the total loss is less than the second preset loss value, the aged training model at this time is determined as the target student model.

[0122] 209. Input the image to be recognized into the target student model to obtain a recognition result of the image to be recognized.

[0123] In the embodiment of the present application, after the target student model is obtained through training, the image to be identified is input into the target student model to obtain the recognition result of the image to be identified. The recognition result includes the detection frame of the image to be identified, the classification result of the detection frame, and the confidence of the detection frame.

[0124] In order to better implement the target detection method based on the YOLO architecture provided in the embodiment of the present application, the embodiment of the present application also provides a target detection device based on the YOLO architecture based on the target detection method based on the YOLO architecture. The meaning of the nouns is the same as that in the target detection method based on the YOLO architecture. For specific implementation details, please refer to the description in the above method embodiment.

[0125] Please refer to Figure 7 , Figure 7 A schematic diagram of the structure of a target detection device based on the YOLO architecture provided in an embodiment of the present application, wherein the target detection device based on the YOLO architecture may include:

[0126] An acquisition module 701 is used to acquire labeled images and unlabeled images;

[0127] A detection module 702 is used to perform target detection on the labeled image and the unlabeled image based on an initial student model to obtain a student detection result of the labeled image and a student detection result of the unlabeled image;

[0128] A classification module 703 is used to classify the unlabeled image based on the teacher model to obtain a pseudo label of the unlabeled image;

[0129] A scoring module 704 is used to score the pseudo-label of the unlabeled image based on the pseudo-label assigner to obtain a score of the pseudo-label;

[0130] A first determination module 705 is used to determine a first supervised loss based on the student detection result of the labeled image and the true label of the labeled image using a first supervised loss function;

[0131] A second determination module 706 is used to determine the unsupervised loss based on the pseudo label of the unlabeled image and the student detection result of the unlabeled image using an unsupervised loss function;

[0132] A third determination module 707, configured to determine a total loss based on the first supervised loss and the unsupervised loss;

[0133] A training module 708 is used to iteratively train the initial student model and the teacher model based on the total loss until the total loss is less than a second preset loss value, and determine the initial student model at this time as a target student model;

[0134] The recognition module 709 is used to input the image to be recognized into the target student model to obtain the recognition result of the image to be recognized.

[0135] The specific implementation of each of the above modules can be found in the previous embodiments and will not be described in detail here.

[0136] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the processor is used to execute the steps in the target detection method based on the YOLO architecture provided in this embodiment by calling a computer program stored in the memory.

[0137] Please refer to Figure 8 , Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0138] The electronic device may include components such as a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, and an input unit 104. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0139] The processor 101 is the control center of the electronic device, which uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 102, and calling data stored in the memory 102. Optionally, the processor 101 may include one or more processing cores; optionally, the processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 101.

[0140] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.

[0141] The electronic device also includes a power supply 103 for supplying power to each component. Optionally, the power supply 103 can be logically connected to the processor 101 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 103 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0142] The electronic device may further include an input unit 104, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0143] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically in this embodiment, the processor 101 in the electronic device will load the executable code corresponding to one or more computer programs into the memory 102 according to the following instructions, and the processor 101 will execute the steps in the target detection method based on the YOLO architecture provided in this application, such as:

[0144] Acquire labeled images and unlabeled images; perform target detection on the labeled images and the unlabeled images based on the initial student model to obtain student detection results for the labeled images and the unlabeled images; classify the unlabeled images based on the teacher model to obtain pseudo labels for the unlabeled images; score the pseudo labels for the unlabeled images based on the pseudo label assigner to obtain scores for the pseudo labels; determine the first supervised loss based on the student detection results for the labeled images and the true labels for the labeled images using the first supervised loss function; determine the unsupervised loss based on the pseudo labels for the unlabeled images, the scores for the pseudo labels, and the student detection results for the unlabeled images using the unsupervised loss function; determine the total loss based on the first supervised loss and the unsupervised loss; iteratively train the initial student model and the teacher model based on the total loss until the total loss is less than a second preset loss value, and determine the initial student model at this time as the target student model; input the image to be recognized into the target student model to obtain the recognition result of the image to be recognized.

[0145] It should be noted that the electronic device provided in the embodiment of the present application belongs to the same concept as the target detection method based on the YOLO architecture in the above embodiment. The specific implementation process is detailed in the above related embodiments and will not be repeated here.

[0146] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program stored therein is executed on the processor of the electronic device provided in the embodiment of the present application, the processor of the electronic device executes the steps in the target detection method based on the YOLO architecture provided in the present application. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0147] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes various optional implementations of the above-mentioned object detection method based on the YOLO architecture.

[0148] The above is a detailed introduction to a target detection method and device based on the YOLO architecture provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0149] It should be noted that when the above embodiments of the present application are applied to specific products or technologies, the relevant data of the user is involved, and the user's permission or consent is required, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A target detection method based on the YOLO architecture, characterized in that: The target detection method based on the YOLO architecture includes: Get labeled images and unlabeled images; Performing target detection on the labeled image and the unlabeled image based on the initial student model to obtain a student detection result of the labeled image and a student detection result of the unlabeled image; Classifying the unlabeled image based on the teacher model to obtain a pseudo label of the unlabeled image; Scoring the pseudo-label of the unlabeled image based on the pseudo-label assigner to obtain a score of the pseudo-label; Determine a first supervised loss based on the student detection result of the labeled image and the true label of the labeled image using a first supervised loss function; Determine an unsupervised loss using an unsupervised loss function based on the pseudo-labels of the unlabeled images, the scores of the pseudo-labels, and the student detection results of the unlabeled images; determining a total loss based on the first supervised loss and the unsupervised loss; Iteratively training the initial student model and the teacher model based on the total loss until the total loss is less than a second preset loss value, and determining the initial student model at this time as a target student model; The image to be recognized is input into the target student model to obtain a recognition result of the image to be recognized.

2. The target detection method based on the YOLO architecture according to claim 1, characterized in that: The target detection method based on the YOLO architecture includes: Obtain a YOLOv5 model, wherein the YOLOv5 model includes a feature extraction backbone network, a Neck layer, and a detection head; A dilated spatial convolutional pooling pyramid module is added to the feature extraction backbone network of the YOLOv5 model, the C3 module in the Neck layer of the YOLOv5 model is replaced with a C2f module, and the detection head of the YOLOv5 model is replaced with three parallel first detection heads, second detection heads and third detection heads, wherein the first detection head is used for classification, the second detection head is used for regression, and the third detection head is used to detect whether an area contains a target, so as to obtain an initial student model.

3. The target detection method based on the YOLO architecture according to claim 2, characterized in that: The performing target detection on the labeled image and the unlabeled image based on the initial student model to obtain the student detection result of the labeled image and the student detection result of the unlabeled image includes: Get a domain adaptive classifier; Using a second supervised loss function to train the initial student model and the domain adaptive classifier based on the labeled image and the unlabeled image to obtain an aging training model; Based on the aging training model, target detection is performed on the labeled image and the unlabeled image to obtain a student detection result of the labeled image and a student detection result of the unlabeled image.

4. The target detection method based on the YOLO architecture according to claim 3, characterized in that: The teacher model is obtained by updating the aging training model EMA.

5. The target detection method based on the YOLO architecture according to claim 4, characterized in that: The student detection result of the labeled image includes the classification result of the labeled image output by the first detection head, the detection frame result of the labeled image output by the second detection head, and the detection confidence result of the labeled image output by the third detection head; The true label of the labeled image includes the true classification result of the labeled image, the true detection frame result of the labeled image, and the true confidence result of the labeled image. The first supervised loss function satisfies the following formula: In the first supervised loss function, (h,w) represents the pixel point on the image with a height of h and a width of w. is the classification result of the labeled image output by the first detection head, represents the true classification result of the labeled image, and are respectively the detection frame result of the labeled image output by the second detection head and the detection confidence result of the labeled image output by the third detection head, and are the true detection box result of the labeled image and the true confidence result of the labeled image, respectively. CE represents the cross entropy loss. represents the CIoU calculation value between the detection box result of the labeled image output by the second detection head and the real detection box result of the labeled image, L s is the first supervised loss.

6. The target detection method based on the YOLO architecture according to claim 5, characterized in that: The student detection result of the unlabeled image includes the classification result of the unlabeled image output by the first detection head, the detection frame result of the unlabeled image output by the second detection head, and the detection confidence result of the unlabeled image output by the third detection head; The true label of the unlabeled image includes the true classification result of the unlabeled image, the true detection frame result of the unlabeled image, and the true confidence result of the unlabeled image. The unsupervised loss function satisfies the following formula: Among them, in the unsupervised loss function, L u is an unsupervised loss, (h,w) represents the pixel point on the image with a height of h and a width of w. is the classification result of the unlabeled image output by the first detection head, and are respectively the detection frame result of the unlabeled image output by the second detection head and the detection confidence result of the unlabeled image output by the third detection head, are the classification result, regression result and detection confidence result of the pseudo label of the unlabeled image at the position (h, w), P (h,w) is the score of the pseudo-label at (h,w), 1 {·} is a discriminant function. If the condition {·} is met, the output is 1, otherwise the output is 0. τ1 is the first preset threshold, τ2 is the second preset threshold, and the first preset threshold is less than the second preset threshold.

7. The target detection method based on the YOLO architecture according to claim 6, characterized in that: The determining of the total loss based on the first supervised loss and the unsupervised loss comprises: The first supervised loss and the unsupervised loss are weightedly summed based on a first preset weight coefficient and a second preset weight coefficient to obtain a total loss.

8. A target detection device based on the YOLO architecture, characterized in that: include: An acquisition module is used to acquire labeled images and unlabeled images; A detection module, used for performing target detection on the labeled image and the unlabeled image based on an initial student model to obtain a student detection result of the labeled image and a student detection result of the unlabeled image; A classification module, used for classifying the unlabeled image based on a teacher model to obtain a pseudo label of the unlabeled image; A scoring module, used for scoring the pseudo-label of the unlabeled image based on the pseudo-label assigner to obtain a score of the pseudo-label; A first determination module, configured to determine a first supervised loss based on a student detection result of the labeled image and a true label of the labeled image using a first supervised loss function; A second determination module is used to determine the unsupervised loss based on the pseudo label of the unlabeled image and the student detection result of the unlabeled image using an unsupervised loss function; A third determination module, configured to determine a total loss based on the first supervised loss and the unsupervised loss; A training module, configured to iteratively train the initial student model and the teacher model based on the total loss until the total loss is less than a second preset loss value, and determine the initial student model at this time as a target student model; The recognition module is used to input the image to be recognized into the target student model to obtain the recognition result of the image to be recognized.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the steps in the target detection method based on the YOLO architecture according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, which are suitable for a processor to load to execute the steps in the target detection method based on the YOLO architecture according to any one of claims 1 to 7.

Images