Target detection method, system and device based on semi-supervised learning and storage medium

Abstract

The invention relates to the technical field of target detection, and particularly provides a semi-supervised learning-based target detection method, system and device and a storage medium, and the method comprises the steps: obtaining a data set which comprises a marked image and an unmarked image; generating a pseudo label for the unmarked image based on the marked image by using a dense detector; dividing the pseudo labels into reliable labels and soft labels according to the scores of the pseudo labels by using a pseudo label distributor; the pseudo label is updated to the data set, the updated data set is utilized to train a target detection model, and the target detection model identifies a target object in the sample image after training; the soft label is set to only participate in the preset partial loss calculation in the training process so as to reduce the training error caused by the soft label. According to the invention, an efficient semi-supervised learning target detection framework is constructed, and the accuracy of a target detection model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and specifically relates to a target detection method, system, device and storage medium based on semi-supervised learning. Background Art

[0002] As a commonly used model in the field of image recognition, the object detection model plays a key role in many practical application scenarios. However, the training process of this model requires a large number of labeled image samples. It is worth noting that labeling images is an extremely tedious task that consumes a lot of manpower, material resources, time and energy.

[0003] In recent years, semi-supervised learning methods have gradually emerged and applied to the field of object detection. The significant advantage of this method is that it does not require a large number of labeled images like traditional methods.

[0004] The core of semi-supervised learning for object detection lies in the allocation of positive and negative samples. The commonly used allocation method is based on the IoU threshold strategy, that is, samples with pseudo-label IoU greater than a certain threshold are set as positive samples, and samples with IoU less than the threshold are set as negative samples. However, as training progresses, the score of the pseudo-label may gradually increase, which may cause inaccurate pseudo-labels to be regarded as reliable labels during the allocation process, thereby introducing training errors. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a target detection method, system, device and storage medium based on semi-supervised learning to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a target detection method based on semi-supervised learning, comprising: Obtain a data set, the data set comprising labeled images and unlabeled images; Generate pseudo labels for unlabeled images based on labeled images using dense detectors; Using a pseudo-label allocator to divide pseudo-labels into reliable labels and soft labels according to their scores; Updating the pseudo labels to the dataset, and using the updated dataset to train a target detection model, wherein the target detection model recognizes target objects in the sample images after training; Set the soft labels to participate in only part of the preset loss calculation during training to reduce the training error caused by soft labels; The dense detector comprises: Improved RetinaNet based on ResNet-50-FPN architecture, the improved structure includes: The output layers are reduced from 5 to 3 to change the input resolution from 1333 to 640; heads at different levels no longer share weights.

[0007] In an optional implementation, using a pseudo-label distributor to divide pseudo-labels into reliable labels and soft labels according to the scores of pseudo-labels includes: Setting a first threshold and a second threshold, wherein the first threshold is smaller than the second threshold; Generate the classification score of the pseudo label according to the probability value given by the Softmax function when generating the pseudo label, and generate the target detection score of the soft label by comparing the complete intersection of the predicted box of the soft label with the real box; Calculate the product of the classification score of the pseudo label and the target detection score, and use the product as the label score of the pseudo label; Label the pseudo labels with label scores higher than the second threshold as reliable labels. The default reliable labels are used in the same way as the labeled images during training. Marking pseudo labels with label scores between a first threshold and a second threshold as soft labels; Pseudo labels with label scores lower than the first threshold are classified as negative samples.

[0008] In an optional implementation, setting the soft labels to participate in only a preset portion of loss calculations during training to reduce training errors caused by the soft labels includes: Filter out the soft labels whose classification scores reach the preset classification score threshold as the first category of soft labels; Filter out the soft labels whose target detection scores reach the preset target detection score threshold as the second type of soft labels; The first type of soft labels are set to participate in the calculation of target loss during training, and the second type of soft labels are set to participate in the calculation of regression loss during training.

[0009] In an optional implementation, setting the first type of soft labels to participate in calculating the target loss during the training process, and setting the second type of soft labels to participate in calculating the regression loss during the training process, includes: The first type of soft labels will not be divided into positive samples and negative samples, and only participate in the calculation of target loss :

[0010] in, is the classification result output by the target detection model, is the classification result of label assignment, is the binary cross entropy loss, is the object detection score of the soft label, is the classification score of the pseudo-label, is the indicator function, h and w represent the height and width of the soft label respectively; Confirm that the target detection score of the second type of soft label is greater than 0.99, and the second type of soft label is used to calculate the regression loss :

[0011] in, is the detection box parameter output by the target detection model, Parameters of the bounding box assigned to the label.

[0012] In an optional embodiment, the method further comprises: Adding a round adapter to control the parameters or strategies of the training process, wherein the round adapter includes a domain adaptation strategy and a distribution adaptation strategy; The domain adaptation strategy includes introducing a domain adaptation loss function into the loss calculation of labeled data. :

[0013] in, is the output of the domain classifier, D=0 for labeled data and D=1 for unlabeled data; During the training process, the domain classifier is trained using the common gradient descent method. During this process, the domain classifier tries to accurately determine whether the input data is labeled data or unlabeled data, and continuously adjusts its own parameters; When the gradient passes through GRL, the sign of the gradient parameter is reversed, and this reversed gradient is used to optimize the base network so that the optimization direction of the base network is opposite to that of the domain classifier; The distribution adaptive strategy includes calculating a first threshold and a second threshold in each round, wherein the first threshold and the second threshold are used to distinguish and divide reliable labels and soft labels, and the first threshold of round k and the second threshold The calculation methods include:

[0014]

[0015] in, is a fixed parameter and can be set to 60 based on experience. It is The categories in The pseudo-label score on rounds, and Indicates the amount of labeled data and the amount of unlabeled data, Indicates The round is in The number of true label annotations computed by the round adapter on each category.

[0016] In a second aspect, the present invention provides a target detection system based on semi-supervised learning, comprising: A data acquisition module, used to acquire a data set, wherein the data set includes labeled images and unlabeled images; a label generation module for generating pseudo labels for unlabeled images based on labeled images using a dense detector; A label classification module, for dividing pseudo labels into reliable labels and soft labels according to the scores of pseudo labels using a pseudo label distributor; A data updating module, used to update the pseudo-labels to the data set, and train the target detection model using the updated data set, wherein the target detection model recognizes the target object in the sample image after training; The training control module is used to set the soft labels to participate in only a preset part of the loss calculation during the training process, so as to reduce the training error caused by the soft labels; The dense detector comprises: Improved RetinaNet based on ResNet-50-FPN architecture, the improved structure includes: The output layers are reduced from 5 to 3 to change the input resolution from 1333 to 640; heads at different levels no longer share weights.

[0017] In an optional implementation, the tag classification module includes: A threshold setting unit, used to set a first threshold and a second threshold, wherein the first threshold is smaller than the second threshold; The first scoring unit is used to generate a classification score of the pseudo label according to the probability value given by the Softmax function when the pseudo label is generated, and to generate an object detection score of the soft label by comparing the complete intersection-union ratio of the predicted box of the soft label with the real box; A second scoring unit, used to calculate the product of the classification score of the pseudo label and the target detection score, and use the product as the label score of the pseudo label; A first partitioning unit is used to mark pseudo labels with label scores higher than a second threshold as reliable labels, and the default reliable labels are used in the same way as the labeled images in the training process; A second dividing unit, configured to mark pseudo labels with label scores between the first threshold and the second threshold as soft labels; The third classification unit is used to classify the pseudo labels whose label scores are lower than the first threshold as negative samples.

[0018] In an optional implementation, the training control module includes: A first screening unit, used to screen out soft labels whose classification scores reach a preset classification score threshold as first-category soft labels; A second screening unit is used to screen out soft labels whose target detection scores reach a preset target detection score threshold as second-category soft labels; The training setting unit is used to set the first type of soft labels to participate in the calculation of the target loss during the training process, and the second type of soft labels to participate in the calculation of the regression loss during the training process.

[0019] In a third aspect, a device is provided, comprising: A memory, used for storing a target detection program based on semi-supervised learning; A processor is used to implement the steps of the target detection method based on semi-supervised learning provided in the first aspect when executing the target detection program based on semi-supervised learning.

[0020] In a fourth aspect, a computer-readable storage medium is provided, on which a target detection program based on semi-supervised learning is stored. When the target detection program based on semi-supervised learning is executed by a processor, the steps of the target detection method based on semi-supervised learning provided in the first aspect are implemented.

[0021] The beneficial effects of the present invention are that the target detection method, system, device and storage medium based on semi-supervised learning provided by the present invention construct an efficient semi-supervised learning target detection framework, improve the quality of pseudo-labels by introducing object scores and pseudo-label allocators, and reduce the distribution difference between labeled data and unlabeled data through round adapters, and have high feasibility and accuracy.

[0022] The pseudo-label allocator uses two thresholds: a low threshold and a high threshold. Labels between these two thresholds are considered uncertain, while labels greater than the high threshold are considered reliable labels. Pseudo-labels with inconsistent classification and regression effects are processed by soft labels without affecting the loss calculation of reliable pseudo-labels. This effectively balances the impact of classification and regression, thereby improving the overall performance of the model without negatively affecting the training effect of reliable pseudo-labels.

[0023] The final pseudo-label score is obtained by multiplying the objectness score and the classification score, which can generate a stronger pseudo-label response in the case of small resolution input.

[0024] The round adapter alleviates the overfitting of the model to labeled data. The appropriate threshold is adaptively calculated in each epoch to distinguish between reliable pseudo labels and uncertain pseudo labels, which can more accurately distinguish between reliable pseudo labels and uncertain pseudo labels.

[0025] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0028] Figure 2 It is a schematic diagram of an unlabeled coal mine conveyor belt image used for training the method according to an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram of the pseudo-label score generation principle of the existing RetinaNet structure.

[0030] Figure 4 It is a schematic diagram of the pseudo label score generation principle of the Dense Detector of the method of an embodiment of the present invention.

[0031] Figure 5 is a schematic block diagram of a system according to an embodiment of the present invention.

[0032] Figure 6 A schematic diagram of the structure of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0035] The following is an explanation of the professional terms that appear in this article: torchvision.datasets.ImageFolder is a class for loading image datasets provided by the torchvision library in PyTorch. It can easily load data from image files organized in a specific directory structure on disk, which is very suitable for image classification tasks.

[0036] RetinaNet is a deep learning model for object detection, proposed by researchers from Facebook AI Research (FAIR) in 2017 in a paper titled "Focal Loss for Dense Object Detection". This model has a significant impact in the field of object detection, and its core contribution is the introduction of the Focal Loss function, which effectively solves the problem of imbalanced number of positive and negative samples in dense object detection.

[0037] Efficient-teacher is a supervised and semi-supervised object detection library for the YOLO series developed by the Alibaba research team. Its main modules include: Dense Detector: a baseline model that extends RetinaNet through dense sampling technology inspired by YOLOv5, providing a basis for subsequent pseudo-label generation and training; PseudoLabel Assigner: a novel pseudo-label assignment mechanism introduced, which makes more sophisticated use of the pseudo-labels generated by Dense Detector, and can prevent the bias caused by a large number of low-quality pseudo-labels in the student-teacher mutual learning mechanism; Epoch Adaptor: a stable and efficient end-to-end semi-supervised training plan is implemented for Dense Detector, which uses domain and distribution adaptation to enable Dense Detector to learn globally distributed features, making training independent of the proportion of labeled data.

[0038] "Complete Intersection over Union" (commonly referred to as CIoU) is a metric used in the field of computer vision to evaluate the similarity between two bounding boxes.

[0039] At present, the deep learning networks that are commonly used in the field of deep learning, such as the first-stage YOLO series algorithm and SSD algorithm, and the second-stage Fast-R CNN and Faster-R CNN, have shined in target detection and gradually replaced traditional algorithms. The YOLO series algorithm has fast detection speed, simple network structure, high robustness and accuracy.

[0040] The target detection method based on semi-supervised learning provided in the embodiment of the present invention is executed by a computer device, and accordingly, the target detection system based on semi-supervised learning runs in the computer device.

[0041] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a target detection system based on semi-supervised learning. According to different requirements, the order of the steps in the flowchart may be changed, and some may be omitted.

[0042] like Figure 1 As shown, the method includes: S1, obtaining a data set, where the data set includes labeled images and unlabeled images.

[0043] Use torchvision.datasets.ImageFolder to load labeled and unlabeled image datasets and perform batch processing through DataLoader (DataLoader is a tool widely used in deep learning and machine learning for efficient loading and processing of data).

[0044] S2, uses dense detectors to generate pseudo labels for unlabeled images based on labeled images.

[0045] Define a simple DenseDetector model and pre-train it using labeled images. Run the model on unlabeled images and generate pseudo labels based on the objectness score and classification score.

[0046] S3, uses the pseudo-label allocator to divide the pseudo-labels into reliable labels and soft labels according to their scores.

[0047] Pseudo labels are divided into reliable labels and soft labels according to the classification score threshold.

[0048] S4, updating the pseudo labels to the data set, and using the updated data set to train the target detection model, wherein the target detection model recognizes the target object in the sample image after training.

[0049] Pseudo-labels are combined with unlabeled images into a new dataset for training the TargetDetectionModel. During training, the losses of reliable labels and soft labels are calculated separately, and the influence of soft labels is controlled by adjusting the weights.

[0050] S5, setting the soft label to participate in only a preset part of the loss calculation during the training process to reduce the training error caused by the soft label.

[0051] It is possible to specify which soft labels can participate in the calculation of losses to avoid cumulative errors.

[0052] In order to explain the technical solution in detail, the foreign body recognition of the conveyor belt image of the coal mine is taken as an example for explanation.

[0053] The target detection model used is Yolov5x. The specific structure of Yolov5x includes: Backbone part, Neck part and Head part.

[0054] Backbone: Responsible for feature extraction of input images. YOLOv5x uses the CSPDarknet53 architecture and combines Cross Stage Partial Networks (CSPNet) to reduce the amount of computation and improve the learning ability of the model. It extracts multi-scale features through dense convolutional layers, which helps capture rich visual information. Neck: used for feature fusion to strengthen feature representation at different scales. YOLOv5x combines FPN (Feature Pyramid Network) and PANet (Path Aggregation Network) to enhance the information flow between features at different levels, thereby improving the ability to detect small objects and complex scenes. Head: Generates the final detection results, including bounding boxes and category predictions. This part consists of multiple convolutional layers and is responsible for outputting the category and position offset of each anchor point. Through optimized loss functions (such as IoU Loss), YOLOv5x improves the accuracy of bounding box regression.

[0055] Use a binocular camera to capture images of the conveyor belt scene. The captured images are as follows: Figure 2 As shown in the figure, the images are mainly conveyor belts in the mining area, which mainly carry bulk coal. The image resolution captured by the camera is 1920*1080, and a total of 9327 conveyor belt images are collected, of which 932 images are annotated, accounting for 10% of the entire dataset. The image annotations include "bulk coal", "iron wire", "root bolt" and "other", among which "iron wire", "root bolt" and "other" are pollutants.

[0056] The efficient-teacher network architecture is used to train the target detection model. The teacher model includes a dense detector, a pseudo-label assigner (PLA) and an epoch adapter, and the student model uses the YOLOv5x model.

[0057] Dense detectors include: Improved RetinaNet based on ResNet-50-FPN architecture, the improved structure includes: The output layers are reduced from 5 to 3 to change the input resolution from 1333 to 640; heads at different levels no longer share weights.

[0058] In the existing RetinaNet, the classification branch only uses the classification score to represent the pseudo label score, such as Figure 3 shown.

[0059] like Figure 4 As shown, the Dense detector of this application introduces an objectness score based on the existing RetinaNet, further enhancing the detection capability of the model. The objectness score is a numerical indicator in the field of computer vision used to measure the possibility that an image area contains an object. The Dense detector obtains the objectness score by calculating the Complete Intersection over Union (CIoU) between the predicted box and the true box (GT box). This score can reflect the position quality of the pseudo-label. In the case of small-resolution input, the Dense detector can generate a more accurate pseudo-label score based on the objectness score and the classification score. Specifically, the pseudo-label score is the product of the objectness score and the classification score.

[0060] The structure of RetinaNet of the ResNet-50-FPN architecture of this application includes: In the esNet-50 backbone network, the input image first passes through a 7×7 convolution layer with a step size of 2 and 64 channels, followed by batch normalization and ReLU activation function operations, and then passes through a maximum pooling layer with a pooling kernel size of 3×3 and a step size of 2 to further downsample the image; it is then mainly composed of 4 residual module groups, denoted as layer1, layer2, layer3 and layer4, each of which consists of multiple residual blocks. As the number of network layers increases, the resolution of the feature map gradually decreases, but the semantic information gradually becomes richer; The feature pyramid network takes the output feature maps C3, C4, and C5 of ResNet-50 as input. First, a 1×1 convolution is performed on the highest-level feature map C5 to reduce the number of channels to obtain P5, so that its number of channels is fixed to 256. Then, P5 is doubled upsampled by the nearest neighbor to obtain a feature map with the same length and width as C4, so that element-by-element addition can be performed later. The same operation is performed on the obtained upsampled feature maps in turn to obtain P4 and P3. At the same time, 1×1 convolution is used to transform the channels of C3 and C4 so that their number of channels is consistent with that of the upsampled feature maps. After adding the upsampled high semantic features and the shallow positioning detail features, the generated P3, P4, etc. are re-fused using 3×3 convolution to generate the final feature map. RetinaHead (output layer) includes classification subnetwork and regression subnetwork: The classification subnetwork consists of multiple 3×3 convolutional layers and 1×1 convolutional layers. The 3×3 convolutional layers are used for feature extraction and fusion, and the 1×1 convolutional layers are used to map the feature maps to a specific dimension to output the classification score of each anchor box. The regression subnetwork, consisting of multiple 3×3 convolutional layers and 1×1 convolutional layers, is used to predict the offset of each anchor box relative to the true target box, including the offset of the center coordinates and the scaling factors of width and height.

[0061] The pseudo label assigner (PLA) uses two thresholds: low threshold and high threshold , labels between these two thresholds are considered uncertain, while labels greater than the high threshold are considered reliable labels. For uncertain labels, PLA introduces a soft loss strategy for processing.

[0062] In a specific example, the pseudo-label division method includes: (1) Setting a first threshold and a second threshold, wherein the first threshold is smaller than the second threshold; (2) Generate the classification score of the pseudo-label according to the probability value given by the Softmax function when the pseudo-label is generated, and generate the target detection score of the soft label by comparing the complete intersection of the predicted box of the soft label with the real box; (3) Calculate the product of the classification score of the pseudo label and the target detection score, and use the product as the label score of the pseudo label; (4) Pseudo labels with a label score higher than the second threshold are marked as reliable labels, i.e., positive samples. The default reliable labels are used in the same way as the labeled images in the training process. (5) Marking pseudo labels with label scores between the first threshold and the second threshold as soft labels; (6) Pseudo labels with label scores lower than the first threshold are classified as negative samples.

[0063] For uncertain labels (soft labels), PLA introduces a soft loss strategy for processing. The soft loss strategy includes: (1) Filter out the soft labels whose classification scores reach the preset classification score threshold as the first category of soft labels; (2) Filter out the soft labels whose target detection scores reach the preset target detection score threshold as the second category of soft labels; (3) Set the first type of soft labels to participate in the calculation of target loss during training, and the second type of soft labels to participate in the calculation of regression loss during training: The first type of soft labels will not be divided into positive samples and negative samples, and only participate in the calculation of target loss :

[0064] in, is the classification result output by the target detection model, is the classification result of label assignment, is the binary cross entropy loss, is the object detection score of the soft label, is the classification score of the pseudo-label, is the indicator function, h and w represent the height and width of the soft label respectively; Confirm that the target detection score of the second type of soft label is greater than 0.99, and the second type of soft label is used to calculate the regression loss :

[0065] in, is the detection box parameter output by the target detection model, Parameters of the detection box assigned to the label. PLA uses Convert pseudo labels into real training samples.

[0066] PLA can handle pseudo labels with inconsistent classification and regression effects through soft labels without affecting the loss calculation of reliable pseudo labels. In this way, PLA effectively balances the impact of classification and regression when dealing with uncertain pseudo labels, thereby improving the overall performance of the model without negatively affecting the training effect of reliable pseudo labels.

[0067] An Epoch Adaptor is added to control the parameters or strategies of the training process. The Epoch Adaptor includes a domain adaptation strategy and a distribution adaptation strategy. Specifically, the Epoch Adaptor makes the training process of semi-supervised object detection (SSOD) faster and smoother through the domain adaptation strategy and the distribution adaptation strategy. The purpose of domain adaptation is to narrow the distribution gap between labeled data and unlabeled data, thereby reducing the impact of domain differences on model training. Distribution adaptation is a dynamic method used to estimate the threshold of pseudo labels in each epoch to adapt to changes in data distribution. This dual adaptive strategy effectively improves the stability of training and the generalization ability of the model.

[0068] The domain adaptation strategy includes introducing a domain adaptation loss function into the loss calculation of labeled data. :

[0069] in, is the output of the domain classifier, D=0 for labeled data and D=1 for unlabeled data; During the training process, the domain classifier is trained using the common gradient descent method. During this process, the domain classifier tries to accurately determine whether the input data is labeled data or unlabeled data, and continuously adjusts its own parameters; When the gradient passes through GRL, the sign of the gradient parameter will be reversed. This reversed gradient is used to optimize the base network so that the optimization direction of the base network is opposite to that of the domain classifier. This strategy allows the base network to reduce the ability of the domain classifier while improving its robustness to differences between different domains, thereby achieving better domain adaptation effects.

[0070] The distribution adaptive strategy includes calculating a first threshold and a second threshold in each round, wherein the first threshold and the second threshold are used to distinguish and divide reliable labels and soft labels. The first threshold of round k and the second threshold The calculation methods include:

[0071]

[0072] in, is a fixed parameter and can be set to 60 based on experience. It is The categories in The pseudo-label score on rounds, and Indicates the amount of labeled data and the amount of unlabeled data, Indicates The round is in The number of true label annotations computed by the round adapter on each category.

[0073] The process of forward propagation, loss calculation, backpropagation, and parameter update is repeated until the preset number of training rounds is reached or other stopping conditions are met.

[0074] By using the trained target detection algorithm to process the latest collected coal mine conveyor belt images, foreign objects can be accurately identified.

[0075] In some embodiments, the target detection system based on semi-supervised learning may include multiple functional modules composed of computer program segments. The computer program of each program segment in the target detection system based on semi-supervised learning may be stored in the memory of a computer device and executed by at least one processor to perform (see Figure 1 Description) Functions for object detection based on semi-supervised learning.

[0076] In this embodiment, the target detection system based on semi-supervised learning can be divided into multiple functional modules according to the functions it performs, such as Figure 5 As shown. The functional modules of the system may include: a data acquisition module, a label generation module, a label classification module, a data update module and a training control module. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0077] A data acquisition module, used to acquire a data set, wherein the data set includes labeled images and unlabeled images; a label generation module for generating pseudo labels for unlabeled images based on labeled images using a dense detector; A label classification module, for dividing pseudo labels into reliable labels and soft labels according to the scores of pseudo labels using a pseudo label distributor; A data updating module, used to update the pseudo-labels to the data set, and train the target detection model using the updated data set, wherein the target detection model recognizes the target object in the sample image after training; The training control module is used to set the soft labels to participate in only a preset part of the loss calculation during the training process to reduce the training error caused by the soft labels.

[0078] Optionally, as an embodiment of the present invention, the dense detector includes: Improved RetinaNet based on ResNet-50-FPN architecture, the improved structure includes: The output layers are reduced from 5 to 3 to change the input resolution from 1333 to 640; heads at different levels no longer share weights.

[0079] Optionally, as an embodiment of the present invention, the label classification module includes: A threshold setting unit, used to set a first threshold and a second threshold, wherein the first threshold is smaller than the second threshold; The first scoring unit is used to generate a classification score of the pseudo label according to the probability value given by the Softmax function when the pseudo label is generated, and to generate an object detection score of the soft label by comparing the complete intersection-union ratio of the predicted box of the soft label with the real box; A second scoring unit, used to calculate the product of the classification score of the pseudo label and the target detection score, and use the product as the label score of the pseudo label; A first partitioning unit is used to mark pseudo labels with label scores higher than a second threshold as reliable labels, and the default reliable labels are used in the same way as the labeled images in the training process; A second dividing unit, configured to mark pseudo labels with label scores between the first threshold and the second threshold as soft labels; The third classification unit is used to classify the pseudo labels whose label scores are lower than the first threshold as negative samples.

[0080] Optionally, as an embodiment of the present invention, the training control module includes: A first screening unit, used to screen out soft labels whose classification scores reach a preset classification score threshold as first-category soft labels; A second screening unit is used to screen out soft labels whose target detection scores reach a preset target detection score threshold as second-category soft labels; The training setting unit is used to set the first type of soft labels to participate in the calculation of the target loss during the training process, and the second type of soft labels to participate in the calculation of the regression loss during the training process.

[0081] Figure 6The target detection method based on semi-supervised learning provided for the embodiment of the present application can be applied to equipment. It will be appreciated by those skilled in the art that the equipment structure involved in the embodiment of the present invention does not constitute a limitation on the equipment, and the equipment may include more or less components than shown, or combine certain components, or arrange different components. In an embodiment of the present invention, the equipment includes but is not limited to laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The equipment may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the embodiments of the present application described herein and / or required.

[0082] The device 600 may include: a processor 610, a memory 620 and a communication unit 630. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0083] The memory 620 may be used to store the execution instructions of the processor 610, and the memory 620 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 620 are executed by the processor 610, the device 600 is enabled to perform some or all of the steps in the following method embodiments.

[0084] The processor 610 is the control center of the storage device, and uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 620, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of a plurality of packaged ICs with the same or different functions. For example, the processor 610 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0085] The communication unit 630 is used to establish a communication channel so that the storage device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0086] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0087] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer device (which can be a personal computer, a server, or a second device, a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0088] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0089] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0090] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0092] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A target detection method based on semi-supervised learning, characterized in that: include: Obtain a data set, the data set comprising labeled images and unlabeled images; Generate pseudo labels for unlabeled images based on labeled images using dense detectors; Using a pseudo-label allocator to divide pseudo-labels into reliable labels and soft labels according to their scores; Updating the pseudo labels to the dataset, and using the updated dataset to train a target detection model, wherein the target detection model recognizes target objects in the sample images after training; Set the soft labels to participate in only part of the preset loss calculation during training to reduce the training error caused by soft labels; The dense detector comprises: Improved RetinaNet based on ResNet-50-FPN architecture, the improved structure includes: The output layers are reduced from 5 to 3 to change the input resolution from 1333 to 640; heads at different levels no longer share weights.

2. The method according to claim 1, characterized in that The pseudo-label allocator is used to divide the pseudo-labels into reliable labels and soft labels according to their scores, including: Setting a first threshold and a second threshold, wherein the first threshold is smaller than the second threshold; Generate the classification score of the pseudo label according to the probability value given by the Softmax function when generating the pseudo label, and generate the target detection score of the soft label by comparing the complete intersection of the predicted box of the soft label with the real box; Calculate the product of the classification score of the pseudo label and the target detection score, and use the product as the label score of the pseudo label; Label the pseudo labels with label scores higher than the second threshold as reliable labels. The default reliable labels are used in the same way as the labeled images during training. Marking pseudo labels with label scores between a first threshold and a second threshold as soft labels; Pseudo labels with label scores lower than the first threshold are classified as negative samples.

3. The method according to claim 1, characterized in that Set the soft labels to participate in only part of the preset loss calculation during the training process to reduce the training error caused by soft labels, including: Filter out the soft labels whose classification scores reach the preset classification score threshold as the first category of soft labels; Filter out the soft labels whose target detection scores reach the preset target detection score threshold as the second type of soft labels; The first type of soft labels are set to participate in the calculation of target loss during training, and the second type of soft labels are set to participate in the calculation of regression loss during training.

4. The method according to claim 2, characterized in that: Set the first type of soft labels to participate in the calculation of target loss during training, and the second type of soft labels to participate in the calculation of regression loss during training, including: The first type of soft labels will not be divided into positive samples and negative samples, and only participate in the calculation of target loss : in, is the classification result output by the target detection model, is the classification result of label assignment, is the binary cross entropy loss, is the object detection score of the soft label, is the classification score of the pseudo-label, is the indicator function, h and w represent the height and width of the soft label respectively; Confirm that the target detection score of the second type of soft label is greater than 0.99, and the second type of soft label is used to calculate the regression loss : in, is the detection box parameter output by the target detection model, Parameters of the bounding box assigned to the label.

5. The method according to claim 1, characterized in that The method further comprises: Adding a round adapter to control the parameters or strategies of the training process, wherein the round adapter includes a domain adaptation strategy and a distribution adaptation strategy; The domain adaptation strategy includes introducing a domain adaptation loss function into the loss calculation of labeled data. : in, is the output of the domain classifier, D=0 for labeled data and D=1 for unlabeled data; During the training process, the domain classifier is trained using the common gradient descent method. During this process, the domain classifier tries to accurately determine whether the input data is labeled data or unlabeled data, and continuously adjusts its own parameters; When the gradient passes through GRL, the sign of the gradient parameter is reversed, and this reversed gradient is used to optimize the base network so that the optimization direction of the base network is opposite to that of the domain classifier; The distribution adaptive strategy includes calculating a first threshold and a second threshold in each round, wherein the first threshold and the second threshold are used to distinguish and divide reliable labels and soft labels, and the first threshold of round k and the second threshold The calculation methods include: in, is a fixed parameter and can be set to 60 based on experience. It is The categories in The pseudo-label score on rounds, and Indicates the amount of labeled data and the amount of unlabeled data, Indicates The round is in The number of true label annotations computed by the round adapter on each category.

6. A target detection system based on semi-supervised learning, characterized in that: include: A data acquisition module, used to acquire a data set, wherein the data set includes labeled images and unlabeled images; a label generation module for generating pseudo labels for unlabeled images based on labeled images using a dense detector; A label classification module, for dividing pseudo labels into reliable labels and soft labels according to the scores of pseudo labels using a pseudo label distributor; A data updating module, used to update the pseudo-labels to the data set, and train the target detection model using the updated data set, wherein the target detection model recognizes the target object in the sample image after training; The training control module is used to set the soft labels to participate in only a preset part of the loss calculation during the training process, so as to reduce the training error caused by the soft labels; The dense detector comprises: Improved RetinaNet based on ResNet-50-FPN architecture, the improved structure includes: The output layers are reduced from 5 to 3 to change the input resolution from 1333 to 640; heads at different levels no longer share weights.

7. The system according to claim 6, characterized in that The label classification module includes: A threshold setting unit, used to set a first threshold and a second threshold, wherein the first threshold is smaller than the second threshold; The first scoring unit is used to generate a classification score of the pseudo label according to the probability value given by the Softmax function when the pseudo label is generated, and to generate an object detection score of the soft label by comparing the complete intersection-union ratio of the predicted box of the soft label with the real box; A second scoring unit, used to calculate the product of the classification score of the pseudo label and the target detection score, and use the product as the label score of the pseudo label; A first partitioning unit is used to mark pseudo labels with label scores higher than a second threshold as reliable labels, and the default reliable labels are used in the same way as the labeled images in the training process; A second dividing unit, configured to mark pseudo labels with label scores between the first threshold and the second threshold as soft labels; The third classification unit is used to classify the pseudo labels whose label scores are lower than the first threshold as negative samples.

8. The system according to claim 6, characterized in that The training control module comprises: A first screening unit, used to screen out soft labels whose classification scores reach a preset classification score threshold as first-category soft labels; A second screening unit is used to screen out soft labels whose target detection scores reach a preset target detection score threshold as second-category soft labels; The training setting unit is used to set the first type of soft labels to participate in the calculation of the target loss during the training process, and the second type of soft labels to participate in the calculation of the regression loss during the training process.

9. A device, characterized in that: include: A memory, used for storing a target detection program based on semi-supervised learning; A processor, used to implement the steps of the target detection method based on semi-supervised learning as described in any one of claims 1 to 5 when executing the target detection program based on semi-supervised learning.

10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a target detection program based on semi-supervised learning, and when the target detection program based on semi-supervised learning is executed by a processor, the steps of the target detection method based on semi-supervised learning as described in any one of claims 1 to 5 are implemented.

Images