Training of target detection network and target detection method, device and equipment

Abstract

The present disclosure provides a target detection network training method, device and equipment and a target detection method, device and equipment. The training method comprises: obtaining a labeled training image sample set and an unlabeled training image sample set; inputting each labeled training image sample into a student network for training to determine a first loss function; inputting each unlabeled training image sample into a teacher network to obtain first target detection behavior information; training the student network according to each unlabeled training image sample and the first target detection behavior information to obtain second target detection behavior information, determining a second loss function, and determining a target loss function according to the first loss function and the second loss function; and obtaining a target detection network when the number of iterations reaches a threshold or the target loss function meets a preset convergence condition.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a method, apparatus, device, and storage medium for training a target detection network and for target detection. Background Technology

[0002] In related technologies, semi-supervised object detection algorithms often employ the Mean-Teacher method. The teacher network (teacher model) is generated by the student network (student model) using an Exponential Moving Average (EMA). This method primarily relies on pseudo-labels generated by the teacher network to guide the student network's learning. For object detection tasks, pseudo-labels include not only category labels but also candidate boxes representing the target's location. However, the candidate boxes generated by the teacher network often have high noise levels, and inaccurately located candidate boxes reduce the accuracy of the student network's training, resulting in a lower accuracy of the final trained object detection model and ultimately reducing the overall accuracy of object detection. Summary of the Invention

[0003] This disclosure provides at least one method, apparatus, device, and storage medium for training an object detection network and for object detection.

[0004] In a first aspect, embodiments of this disclosure provide a method for training an object detection network, the method comprising:

[0005] Obtain the labeled training image sample set and the unlabeled training image sample set;

[0006] Each labeled training image sample in the labeled training image sample set is input into the student network for training, and a first loss function is determined, wherein the first loss function is the loss function between the category identified by the student network and the labeled category of the labeled training image sample;

[0007] Each unlabeled training image sample in the unlabeled training image sample set is input into the teacher network to obtain the first target detection behavior information for each unlabeled training image sample.

[0008] The student network is trained based on the unlabeled training image samples and the first target detection behavior information to obtain the second target detection behavior information for each unlabeled training image sample. A second loss function is determined, and a target loss function is determined based on the first loss function and the second loss function. When the number of iterations reaches a threshold or the target loss function satisfies a preset convergence condition, a target detection network is obtained. The second loss function is the loss function between the second target detection behavior information and the first target detection behavior information.

[0009] In this embodiment of the disclosure, iterative training is performed using labeled training image sample sets and unlabeled training image sample sets. Specifically, for the unlabeled training image sample set, a loss function is constructed between the first object detection behavior information of the teacher network and the second object detection behavior information of the student network. This can fully utilize the supervision information generated by the teacher network and improve the performance of the object detection network.

[0010] In an optional implementation, when multiple first candidate boxes identified in the first target detection behavior information are clustered into multiple clusters, a target loss function is determined based on the first loss function and the second loss function, specifically including:

[0011] For each of the plurality of clusters, a second loss function corresponding to that cluster is obtained;

[0012] The third loss function is obtained by weighted averaging of the multiple second loss functions.

[0013] The target loss function is obtained based on the third loss function and the first loss function.

[0014] In this embodiment of the disclosure, a second loss function can be obtained for each of the multiple clusters obtained by clustering, and the final target loss function can be obtained by weighted averaging, thereby improving the efficiency and reliability of the calculation.

[0015] In one optional implementation, for each of the plurality of clusters, a second loss function corresponding to the cluster is obtained, including:

[0016] For each of the multiple clusters, a category loss function is obtained by considering the loss function between the category of each second candidate box of the target identified by the student network and the category of the first candidate box with the highest confidence identified by the teacher network.

[0017] Based on the loss function between each second candidate box of the target identified by the student network and the first candidate box with the highest confidence identified by the teacher network, a regression loss function is obtained;

[0018] Based on the ranking distribution of the second candidate boxes of the target identified by the student network and the ranking distribution of the first candidate boxes identified by the teacher network, a ranking distribution loss function is obtained.

[0019] The second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function.

[0020] In this embodiment of the disclosure, a category loss function, a regression loss function, and a ranking distribution loss function are constructed for each cluster to guide the training of the student network. This can make full use of the supervision information generated by the teacher network during the object detection process and improve the training performance.

[0021] In one optional implementation, the second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function, specifically including:

[0022] The sum between the category loss function and the regression loss function is obtained, and the product between the ranking distribution loss function and the first weight value is obtained. The sum and the product are added together to obtain the second loss function, wherein the first weight value is used to control the loss weight of the ranking distribution.

[0023] In this embodiment of the disclosure, the loss weights of different information can be controlled by the weight values, thereby controlling the training of the student network and obtaining a target detection network that meets the requirements.

[0024] In one optional implementation, obtaining the target detection network when the number of iterations reaches a threshold or the target loss function satisfies a preset convergence condition includes:

[0025] For the current number of iterations, the student network is trained based on the labeled training image samples, the unlabeled training image samples, and the first target detection behavior information. The model parameters of the teacher network for the next iteration are determined based on the model parameters of the student network obtained from the current number of iterations, until the number of iterations reaches a threshold or the target loss function converges, thus obtaining the target detection network.

[0026] In this embodiment of the disclosure, the student network is updated in each round of training iteration, and the teacher network used for the next training iteration is also updated, thereby improving the convergence and accuracy of the object detection network.

[0027] In one optional implementation, obtaining the target detection network specifically includes:

[0028] The teacher network or the student network obtained through iterative training when the number of iterations reaches a threshold or when the target loss function converges is used as the target detection network.

[0029] Secondly, embodiments of this disclosure also provide a target detection method, including:

[0030] Acquire the image to be detected;

[0031] The target detection network generated using the target detection network training method described in the first aspect above is used to perform target detection on the image to be detected, thereby obtaining the category of the target detected in the image to be detected.

[0032] In this embodiment, the object detection network uses a teacher network to iteratively train a student network on the object detection behavior information of each target training image sample in the target training image sample set. This ensures that the object detection behavior information of the student network and the teacher network satisfies a preset similarity condition for the same target training image sample. Then, during object detection, the network acquires the image to be detected and performs object detection on it, thereby obtaining the category of the detected target. By using the teacher network's object detection behavior information to supervise the training of the student network, allowing the student network to mimic the teacher network's object detection behavior to obtain category labels, richer supervisory information can be mined and utilized, improving the accuracy and reliability of training, and thus enhancing the accuracy of object detection.

[0033] In one optional implementation, the image to be detected is a surveillance image captured by a monitoring device. Then, the category of the target detected from the image to be detected is obtained, including:

[0034] Using the target detection network, human targets are detected in the surveillance image to be detected to determine whether a human body exists in the surveillance image to be detected, and if a human body is found, whether the human body is the target user.

[0035] In this embodiment of the disclosure, the human body category is detected for the acquired monitoring image to be detected, thereby identifying the target user in the monitoring image to be detected. This can improve the tracking and identification of target users in the field of intelligent monitoring.

[0036] In one optional implementation, if the image to be detected is a traffic image of a defined road segment, then obtaining the category of the target detected from the image to be detected includes:

[0037] Using the target detection network, target detection is performed on the traffic image to be detected to determine the targets belonging to the vehicle category in the traffic image to be detected, and the number of targets belonging to the vehicle category is determined.

[0038] Based on the target number of vehicles belonging to a given vehicle category, the traffic congestion level of the designated road segment is obtained.

[0039] In this embodiment of the disclosure, target detection of vehicles in the traffic image to be detected can determine the number of vehicles in a certain road segment, thereby enabling real-time analysis of the traffic situation in the certain road segment and providing a basis for intelligent transportation.

[0040] Thirdly, embodiments of this disclosure also provide a training apparatus for an object detection network, comprising:

[0041] The first acquisition module is used to acquire the labeled training image sample set and the unlabeled training image sample set;

[0042] The first training branch module is used to input each labeled training image sample in the labeled training image sample set into the student network for training, and determine the first loss function, wherein the first loss function is the loss function between the category identified by the student network and the labeled category of the labeled training image sample;

[0043] The second training branch module is used to input each unlabeled training image sample from the unlabeled training image sample set into the teacher network to obtain first target detection behavior information for each unlabeled training image sample; train the student network based on each unlabeled training image sample and the first target detection behavior information to obtain second target detection behavior information for each unlabeled training image sample, determine a second loss function, and determine a target loss function based on the first loss function and the second loss function; and obtain a target detection network when the number of iterations reaches a threshold or the target loss function meets a preset convergence condition, wherein the second loss function is a loss function between the second target detection behavior information and the first target detection behavior information.

[0044] In an optional implementation, when multiple first candidate boxes identified in the first target detection behavior information are clustered into multiple clusters, the second training branch module is used to determine the target loss function based on the first loss function and the second loss function when determining the target loss function:

[0045] For each of the plurality of clusters, a second loss function corresponding to that cluster is obtained;

[0046] The third loss function is obtained by weighted averaging of the multiple second loss functions.

[0047] The target loss function is obtained based on the third loss function and the first loss function.

[0048] In one optional implementation, when obtaining the second loss function corresponding to each of the plurality of clusters, the second training branch module is used to:

[0049] For each of the multiple clusters, a category loss function is obtained by considering the loss function between the category of each second candidate box of the target identified by the student network and the category of the first candidate box with the highest confidence identified by the teacher network.

[0050] Based on the loss function between each second candidate box of the target identified by the student network and the first candidate box with the highest confidence identified by the teacher network, a regression loss function is obtained;

[0051] Based on the ranking distribution of the second candidate boxes of the target identified by the student network and the ranking distribution of the first candidate boxes identified by the teacher network, a ranking distribution loss function is obtained.

[0052] The second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function.

[0053] In an optional implementation, when determining the second loss function based on the category loss function, the regression loss function, and the ranking distribution loss function, the second training branch module is used to:

[0054] The sum between the category loss function and the regression loss function is obtained, and the product between the ranking distribution loss function and the first weight value is obtained. The sum and the product are added together to obtain the second loss function, wherein the first weight value is used to control the loss weight of the ranking distribution.

[0055] In an optional implementation, when the number of iterations reaches a threshold or the target loss function satisfies a preset convergence condition, the second training branch module is used to:

[0056] For the current number of iterations, the student network is trained based on the labeled training image samples, the unlabeled training image samples, and the first target detection behavior information. The model parameters of the teacher network for the next iteration are determined based on the model parameters of the student network obtained from the current number of iterations, until the number of iterations reaches a threshold or the target loss function converges, thus obtaining the target detection network.

[0057] In one optional implementation, when obtaining the target detection network, the first acquisition module is used to:

[0058] The teacher network or the student network obtained when the number of iterations reaches a threshold or when the target loss function converges is used as the target detection network.

[0059] Fourthly, an optional implementation of this disclosure also provides a target detection device, comprising:

[0060] The second acquisition module is used to acquire the image to be detected;

[0061] The detection module is used to perform target detection on the image to be detected using a target detection network generated based on the training method of the target detection network described in the first aspect above, and to obtain the category of the target detected in the image to be detected.

[0062] In one optional implementation, the image to be detected is a monitoring image captured by a monitoring device. When obtaining the category of the target detected from the image to be detected, the detection module is used to:

[0063] Using the target detection network, human targets are detected in the surveillance image to be detected to determine whether a human body exists in the surveillance image to be detected, and if a human body is found, whether the human body is the target user.

[0064] In one optional implementation, if the image to be detected is a traffic image of a defined road segment, then when obtaining the category of the target detected from the image to be detected, the detection module is used to:

[0065] Using the target detection network, target detection is performed on the traffic image to be detected to determine the targets belonging to the vehicle category in the traffic image to be detected, and the number of targets belonging to the vehicle category is determined.

[0066] Based on the target number of vehicles belonging to a given vehicle category, the traffic congestion level of the designated road segment is obtained.

[0067] Fifthly, an optional implementation of this disclosure also provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the steps of the first aspect described above, or any possible implementation of the first aspect, are performed; or, when the machine-readable instructions are executed by the processor, the steps of the second aspect described above, or any possible implementation of the second aspect, are performed.

[0068] In a sixth aspect, an optional implementation of this disclosure also provides a computer-readable storage medium storing a computer program that, when run, performs the steps of the first aspect or any possible implementation thereof, or, when the machine-readable instructions are executed by the processor, performs the steps of the second aspect or any possible implementation thereof.

[0069] For a description of the effects of the aforementioned target detection device, computer equipment, and computer-readable storage medium, please refer to the description of the target detection method above, which will not be repeated here.

[0070] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0071] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0072] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0073] Figure 1 A flowchart of a target detection method provided by an embodiment of this disclosure is shown;

[0074] Figure 2 A flowchart illustrating the training method for the object detection network provided in an embodiment of this disclosure is shown;

[0075] Figure 3 A schematic diagram illustrating the overall logic principle of the training method for the target detection network provided in this embodiment is shown.

[0076] Figure 4 The diagram illustrates the logical principle of the training method for the target detection network provided in this embodiment, specifically for an unlabeled training image sample set.

[0077] Figure 5 A schematic diagram of the target detection device provided in an embodiment of this disclosure is shown;

[0078] Figure 6A schematic diagram of a training apparatus for an object detection network provided in an embodiment of this disclosure is shown;

[0079] Figure 7 A schematic diagram of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown herein can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0081] To facilitate understanding of the technical solutions disclosed herein, the technical terms used in the embodiments of this disclosure will first be explained:

[0082] Semi-supervised learning (SSL) is a learning method that combines supervised and unsupervised learning. It mainly considers how to use a small number of labeled samples and a large number of unlabeled samples for training and classification.

[0083] The Mean-Teacher method: Its core idea is that the network acts as both a student network and a teacher network. In this embodiment, the student network can also be called a student model, and the teacher network can also be called a teacher model. As a teacher network, it is used to generate the learning objectives of the student network. As a student network, it uses the objectives generated by the teacher network to learn. The network parameters of the teacher network are obtained by taking the network parameters of the student network through an exponential moving average (EMA). The version update of the teacher network is based on the weighted sum of the current student network and the teacher network in the previous iteration of training, i.e., the EMA smoothed version.

[0084] Object detection is a fundamental computer vision task with wide applications in fields such as intelligent surveillance, robotics, and autonomous driving. Semi-supervised object detection algorithms, which can utilize a small amount of labeled data and a large amount of unlabeled data, are of great significance. How to efficiently improve the accuracy and performance of object detection using large amounts of unlabeled data is a major research topic in semi-supervised object detection.

[0085] In related technologies, semi-supervised object detection algorithms often employ the Mean-Teacher method. The teacher network is generated from the student network via Emulsion Method (EMA). A weak data augmentation method is applied to the teacher network to generate pseudo-labels, while the student network is subjected to a strong data augmentation method and uses these pseudo-labels as supervisory signals for learning. This method primarily relies on the pseudo-labels generated by the teacher network to guide the student network's learning. However, for object detection tasks, pseudo-labels include not only category labels but also candidate boxes representing object locations. The pseudo-labels generated by the teacher network are noisy, and the accuracy of the generated candidate boxes is low. This can mislead the student network's label assignment method when using candidate boxes to supervise its training, causing many inaccurately located candidate boxes to be misclassified as positive samples. This affects the training of the student network, resulting in lower accuracy of the final trained model and reduced object detection accuracy.

[0086] Furthermore, current semi-supervised object detection algorithms typically set a high score threshold to filter out obviously erroneous pseudo-labels, resulting in very sparse pseudo-labels. Consequently, the student network can only obtain sparse and limited supervision information from the teacher network, reducing the accuracy of model training.

[0087] Based on the above research, this disclosure provides a training method and a target detection method for an object detection network. Specifically, the training method for the object detection network involves obtaining a set of labeled training image samples and a set of unlabeled training image samples. Each labeled training image sample from the labeled training image sample set is input into a student network for training to determine a first loss function. Each unlabeled training image sample from the unlabeled training image sample set is input into a teacher network to obtain first target detection behavior information for each unlabeled training image sample. Based on the unlabeled training image samples and the first target detection behavior information, the student network is trained to obtain second target detection behavior information for each unlabeled training image sample, and a second loss function is determined. Finally, a target loss function is determined based on the first and second loss functions. The training continues until a threshold number of iterations is reached. Alternatively, if the target loss function satisfies a preset convergence condition, an object detection network is obtained. Then, during object detection, the image to be detected is acquired, and based on the trained object detection network, object detection is performed on the image to be detected, thereby obtaining the category of the object detected in the image. In this way, the object detection behavior information of the teacher network during the object detection process is used as supervision information to train the student network. Compared with related technologies that only use the candidate box with the highest score as supervision information, the embodiments of this disclosure can mine denser supervision information from the teacher network, make fuller use of the supervision information in the teacher network, and allow the student network to obtain the category label of the student network by imitating the object detection behavior information of the teacher network, thereby improving the accuracy and performance of the object detection network training and thus improving the accuracy of object detection.

[0088] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0089] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0090] To facilitate understanding of this embodiment, a target detection method disclosed in this disclosure will first be described in detail. The target detection method provided in this disclosure generally executes a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the target detection method can be implemented by a processor calling computer-readable instructions stored in memory.

[0091] For example, the terminal device obtains the image to be detected input by the user and sends the image to be detected to the server. The server performs target detection on the image to be detected based on the trained target detection network, obtains candidate boxes representing the target location region and the category of the target corresponding to the candidate box, and then the server can return the detection results to the terminal. In addition, the target detection network involved in the embodiments of this disclosure needs to be trained in advance. The training process can also be performed by the terminal device, the server or other processing devices. In one possible implementation, due to the performance limitations of the terminal device, it can usually be performed by the server side.

[0092] The target detection method provided in this disclosure embodiment will be described below using a server as the execution subject as an example.

[0093] See Figure 1 The diagram shows a flowchart of a target detection method provided in an embodiment of this disclosure. The method includes steps S101 to S102, wherein:

[0094] S101: Acquire the image to be detected.

[0095] In this embodiment of the disclosure, the target detection method can be applied to fields such as intelligent monitoring, robotics, autonomous driving, intelligent transportation, and smart homes, without limitation. For example, a captured face image can be obtained as the image to be detected, and target detection can be performed on the face in the face image. Or, for example, a traffic image of a certain road segment can be obtained as the image to be detected, and target detection can be performed on the vehicles in the traffic image, thereby determining the congestion situation of the road segment.

[0096] S102: Based on the trained object detection network, perform object detection on the image to be detected and obtain the category of the object detected in the image.

[0097] The target detection network uses the teacher network to iteratively train the student network with the target detection behavior information of each target training image sample in the target training image sample set, so that the target detection behavior information of the student network and the teacher network meet the preset similarity conditions for the same target training image sample.

[0098] When performing step 102, the specific steps include:

[0099] Based on a trained object detection network, the system takes the image to be detected as input, performs object detection on the image, obtains candidate boxes of the detected objects, and obtains the category of the object corresponding to the candidate box.

[0100] In this embodiment, the method can be applied to an image to be detected that includes only one target. The target detection network can then obtain candidate bounding boxes containing the target, and the image feature information of the image region containing each candidate bounding box can be classified to obtain the category of the target. In another possible embodiment, the method can also be applied to an image to be detected that includes multiple targets. In this case, the target detection network can obtain candidate bounding boxes containing each target separately, and the image feature information of the image region containing each target's candidate bounding box can be classified separately to obtain the category of each target.

[0101] In this embodiment of the disclosure, the target detection method mainly relies on the target detection network. The accuracy and performance of the target detection network determine the target detection result of the image to be detected. In order to improve the accuracy and performance of the target detection network, this embodiment of the disclosure provides a training method for the target detection network. Specifically, the target detection network uses the teacher network to iteratively train the student network on the target detection behavior information of each target training image sample in the target training image sample set, so that the target detection behavior information of the student network and the teacher network meet the preset similarity condition for the same target training image sample.

[0102] Among them, the target detection behavior information represents the behavior data in the target detection process, including but not limited to each identified candidate box, the clustering information of each candidate box, and the ranking distribution of candidate boxes in each cluster formed by the clustering.

[0103] The target training image samples can be an unlabeled training image sample set. In this embodiment, the training of the target detection network is mainly applied to a semi-supervised training scenario. The target training image samples also include a labeled training image sample set. The target detection network is obtained by iterative training based on the labeled training image sample set and the unlabeled training image sample set. For the labeled training image sample set, it can be directly input into the student network for training. For the unlabeled training image sample set, in this embodiment, the target detection behavior information of the teacher network is used as supervision information to train the student network.

[0104] Thus, in this embodiment of the disclosure, for unlabeled training image samples, the teacher network iteratively trains the student network using the target detection behavior information of the unlabeled training image samples. This ensures that the target detection behavior information of the student network and the teacher network meets a preset similarity condition for the same training image sample. This allows for the extraction of richer supervisory information from the teacher network. Furthermore, by having the student network learn and imitate the target detection behavior information of the teacher network, training labels for the student network are obtained, improving the accuracy of the student network training. This results in a more accurate target detection network. Consequently, when performing target detection, the trained target detection network is used to detect targets in the image to be detected, obtaining the category of the target detected in the image, thereby improving the accuracy and reliability of target detection.

[0105] The training method for object detection networks is explained in detail below. (See also...) Figure 2 The diagram shown is a flowchart of a training method for an object detection network in this embodiment of the present disclosure, including:

[0106] S201: Obtain the labeled training image sample set and the unlabeled training image sample set.

[0107] The labeled training image sample set includes each labeled training image sample, and each labeled training image sample includes a labeled category. The unlabeled training image sample set includes each unlabeled training image sample.

[0108] In this embodiment, the training scenario of a semi-supervised object detection method is mainly targeted. It requires a small number of labeled training image samples and a large number of unlabeled training image samples. During the training process, each round of training iteration requires both labeled and unlabeled training image samples. The number of labeled and unlabeled training image samples can be mixed in a certain proportion. In this way, the teacher network can predict the category of the unlabeled training image sample based on the labeled category of the labeled training image sample and use the prediction result as the category label of the unlabeled training image sample.

[0109] S202: Input the labeled training image samples from each labeled training image sample set into the student network for training, and determine the first loss function, wherein the first loss function is the loss function between the category identified by the student network and the labeled category of the labeled training image sample.

[0110] In other words, for labeled training image samples, since they have clearly defined labeled categories, they can be directly input into the student network for training. The first loss function in this process is the loss function between the category identified by the student network and the labeled category, such as the cross-entropy function, without any restrictions. The first loss function can be denoted as...

[0111] S203: Input each unlabeled training image sample from the unlabeled training image sample set into the teacher network to obtain the first target detection behavior information for each unlabeled training image sample.

[0112] In this embodiment, the teacher network typically uses Non-Maximum Suppression (NMS) to filter out sparse detection boxes from dense candidate boxes during object detection, which are then used as the final detection results. However, this embodiment reveals that the teacher network's behavior during NMS can provide effective supervisory information. Specifically, the teacher network's behavior during NMS can be divided into two parts: 1) how it clusters candidate boxes into multiple clusters; and 2) the ranking of candidate boxes within each cluster. Therefore, in this embodiment, the training of the student network can be supervised based on these two parts of the behavior.

[0113] The first object detection behavior information includes at least the identified first candidate boxes, the clustering information of each first candidate box, and the ranking distribution of the first candidate boxes in each cluster formed by the clusters. There are no restrictions on this information. The purpose is to mine more dense and richer information of the teacher network in the object detection process, so as to use it as supervisory information to guide the training of the student network and improve the training accuracy.

[0114] S204: Train the student network based on each unlabeled training image sample and the first target detection behavior information to obtain the second target detection behavior information for each unlabeled training image sample, determine the second loss function, and determine the target loss function based on the first loss function and the second loss function. When the number of iterations reaches a threshold or the target loss function meets the preset convergence condition, obtain the target detection network. The second loss function is the loss function between the second target detection behavior information and the first target detection behavior information.

[0115] In one possible embodiment, step S204 specifically includes:

[0116] S1. Based on each unlabeled training image sample and the first target detection behavior information, train the student network to obtain the second target detection behavior information for each unlabeled training image sample.

[0117] The second object detection behavior information refers to the behavioral data of the student network during the object detection process. This second object detection behavior information includes at least the identified second candidate boxes, the clustering information of each second candidate box, and the sorting distribution of the second candidate boxes in each cluster formed by the clustering.

[0118] S2. Determine the second loss function, and determine the target loss function based on the first and second loss functions.

[0119] The second loss function is the loss function between the second target detection behavior information and the first target detection behavior information.

[0120] In this embodiment of the disclosure, when training the student network, the training category label of the student network is obtained by having the student network imitate the target detection behavior information of the teacher network. Based on the behavior of the teacher network, the training of the student network in this embodiment of the disclosure can also be divided into two parts: 1) Reverse NMS clustering, the purpose of which is to let the student network learn the NMS process of the teacher network and obtain the same NMS clustering cluster as the teacher network. The basic principle is to statistically analyze the NMS behavior of the teacher network, determine which candidate boxes the teacher network clusters into a cluster, and consider that the candidate boxes in a cluster usually detect the same target. Therefore, when training the student network, the candidate boxes in the same cluster can be assigned the same target, that is, the same category. 2) NMS ranking matching: Within each cluster, candidate boxes can be ranked based on confidence level, and the candidate box with the highest confidence level is retained. Therefore, the ranking distribution of candidate boxes within a cluster can also include rich information. For example, it can be considered that the feature information of candidate boxes with higher confidence level is more related to the target corresponding to the cluster, and the feature information of candidate boxes with lower confidence level is less related to the target corresponding to the cluster. In this embodiment, the student network can also be trained based on the candidate box ranking distribution of each cluster output by the teacher network. In this way, after the student network can generate the same clusters as the teacher network, the candidate box ranking distribution of the teacher network and the student network in each cluster can be further aligned, so that the student network can generate the same NMS process as the teacher network, that is, the second target detection behavior information of the student network is similar to the first target detection behavior information of the teacher network.

[0121] Furthermore, during the object detection process, the identified candidate boxes are usually clustered into one or more clusters. When they are clustered into one cluster, the target loss function can be obtained based on the second loss function corresponding to that cluster and the first loss function.

[0122] If multiple first candidate boxes identified in the first object detection behavior information are clustered into multiple clusters, then the target loss function is determined based on the first loss function and the second loss function, specifically including:

[0123] 1) For each of the multiple clusters, obtain the second loss function corresponding to the cluster.

[0124] For example, if the number of clusters is M, then the second loss function corresponding to the j-th cluster can be denoted as:

[0125] 2) Based on the weighted average of the obtained second loss functions, the third loss function is obtained.

[0126] For example, the third loss function can be:

[0127] That is, the third loss function can be expressed as the sum of the second loss functions of each cluster, and then divided by the number of clusters.

[0128] 3) Based on the third loss function and the first loss function, the target loss function is obtained.

[0129] For example, the target loss function is denoted as The first loss function is The target loss function is then:

[0130]

[0131] Here, α is a weight value that can be used to control the loss contribution between labeled training image samples and unlabeled training image samples. The value of α is proportional to the contribution of unlabeled training image samples.

[0132] In this embodiment of the disclosure, for each of the multiple clusters, a second loss function corresponding to that cluster is obtained. A possible implementation method is also provided, specifically including:

[0133] 1) For each of the multiple clusters, obtain the category loss function based on the loss function between the category of each second candidate box of the target identified by the student network and the category of the first candidate box with the highest confidence identified by the teacher network.

[0134] For example, in this embodiment of the disclosure, the teacher network performs object detection on unlabeled training image samples and can identify multiple first candidate boxes. These multiple first candidate boxes are clustered into multiple clusters, and the j-th cluster of the teacher network can be defined as... Where b and c represent the candidate box and the category, respectively, and N j represents the number of candidate boxes included in the j-th cluster, and t represents the teacher network.

[0135] Within each cluster, candidate boxes belonging to the same cluster can be considered to have the same category and should predict the same target. Therefore, the candidate boxes with the highest scores or confidence levels can be used. Let represent the prediction target corresponding to the j-th cluster. It will also be used to inversely assign training class labels to the corresponding unlabeled training image samples in the student network. That is, it can be understood that the predicted class c corresponding to the cluster can be used as the training class label required by the student network during training.

[0136] Similarly, the j-th cluster of the student network can be defined as Where s represents the student network.

[0137] For the j-th cluster, the class loss function can be expressed as:

[0138]

[0139] in, Let f represent the class loss function f between the i-th second candidate box and class c in the j-th cluster of the student network. cls This represents the Focal loss function. Of course, other loss functions can also be used in the embodiments of this disclosure, and there is no limitation.

[0140] 2) Based on the loss function between each second candidate box of the target identified by the student network and the first candidate box with the highest confidence identified by the teacher network, the regression loss function is obtained.

[0141] In this embodiment of the disclosure, for the regression task during the training process, the first candidate box with the highest confidence level identified by the teacher network can be used. As the corresponding second candidate box in the student network If the regression objective is to be determined, then the regression loss function can be expressed as:

[0142]

[0143] in, express f was obtained after being transferred to the student network space. reg The regression loss can be represented as, for example, the Smooth L1 loss function, but this embodiment of the present disclosure does not impose any limitations on it.

[0144] 3) Based on the loss function between the ranking distribution of the second candidate boxes of the target identified by the student network and the ranking distribution of the first candidate boxes identified by the teacher network, obtain the ranking distribution loss function.

[0145] For example, for the j-th cluster, based on the score distribution, we model and obtain the candidate box ranking distribution of the teacher network and the student network in that cluster respectively:

[0146]

[0147] Where t represents the teacher network, s represents the student network, and p i This represents the probability that the i-th candidate box belongs to the target category, and T is a preset coefficient.

[0148] This allows for the ranking and distribution of each second candidate box based on the student network. The ranking distribution of the first candidate boxes in the teacher network Obtain the ranking distribution loss function; for example, the KL divergence loss function can be used, and there are no restrictions on this.

[0149]

[0150] The candidate box ranking distribution not only provides feature association information for the target category, but also allows the student network to retain the same candidate boxes as the teacher network in each NMS cluster.

[0151] 4) Determine the second loss function based on the category loss function, regression loss function, and ranking distribution loss function.

[0152] Specifically, this disclosure provides a possible implementation method, which obtains the sum between the category loss function and the regression loss function, and obtains the product between the ranking distribution loss function and the first weight value. The sum and the product are added together to obtain the second loss function, wherein the first weight value is used to control the loss weight of the ranking distribution.

[0153] For example, the second loss function corresponding to the j-th cluster is:

[0154]

[0155] Wherein, β is the first weight value, which can be set according to actual needs. It can represent the degree of contribution of the ranking distribution loss function to the second loss function. The larger the value, the greater the contribution of the ranking distribution loss function.

[0156] S3. Obtain the object detection network when the number of iterations reaches the threshold or the target loss function meets the preset convergence condition.

[0157] In this embodiment of the disclosure, during each round of iterative training, the student network is supervised and trained based on the first target detection behavior information of the teacher network. After one round of iteration, the teacher network needs to be updated based on the student network for the next round of iterative training. Specifically, this embodiment of the disclosure provides a possible implementation method:

[0158] For the current iteration of training, the student network is trained based on each labeled training image sample, as well as each unlabeled training image sample and the first target detection behavior information. Based on the model parameters of the student network obtained from the current iteration of training, the model parameters of the teacher network for the next iteration of training are determined. This process continues until the number of iterations reaches a threshold or the target loss function converges, thus obtaining the target detection network.

[0159] For example, the EMA method can be used to exponentially average the student network to obtain the teacher network.

[0160] In this way, each round of iterative training updates the student network and the teacher network, improving the accuracy of the next iteration of training, and thus improving the overall training accuracy.

[0161] Specifically, obtaining the target detection network includes using the teacher network or student network obtained when the number of iterations reaches a threshold or the target loss function converges as the target detection network.

[0162] In other words, in this embodiment of the disclosure, during the training of the object detection network, the student network is continuously trained based on the teacher network, and the teacher network is iteratively updated based on the student network. When the final number of iterations reaches the threshold or the target loss function converges, the final teacher network or student network can be used as the object detection network after training is completed. This embodiment of the disclosure does not impose any restrictions. In one possible embodiment, since the teacher network is obtained from the student network through the EMA method, the teacher network after training can be used as the object detection network.

[0163] In this embodiment of the disclosure, since the teacher network is updated in each iteration of training, the teacher network has more stable performance. Therefore, at the end of the final iteration, the final teacher network can be directly used as the object detection network for object detection.

[0164] In this embodiment, a target detection network is trained using both labeled and unlabeled training image sample sets. The labeled training image sample set is directly input into the student network for training, determining a first loss function. The unlabeled training image sample set is input into the teacher network to obtain the teacher network's first target detection behavior information for each unlabeled training image sample. Based on this first target detection behavior information, the student network is trained to obtain the student network's second target detection behavior information for each unlabeled training image sample, determining a second loss function. Finally, a target loss function is determined based on the first and second loss functions. During iteration... When the number of training iterations reaches a threshold or the target loss function meets the preset convergence condition, the target detection network is obtained. In this way, for unlabeled training image samples, denser first target detection behavior information is extracted from the teacher network to supervise the training of the student network, without relying on the pseudo-labels output by the teacher network in related technologies. This allows for more full utilization of the supervision information in the teacher network. Furthermore, by having the student network imitate the behavior of the teacher network, the training labels of the student network are obtained, eliminating the need for label assignment techniques and reducing the negative impact of inaccurate pseudo-labels. This, in turn, improves training accuracy and enhances the performance and accuracy of target detection.

[0165] Based on the above embodiments, the logical principles of the training process of the object detection network are briefly explained below. (See attached document.) Figure 3 The diagram shown is a schematic diagram of the overall logic principle of the training method for the target detection network in this embodiment of the present disclosure.

[0166] like Figure 3 As shown, the training methods for object detection networks can be divided into the following two aspects:

[0167] The first aspect: targeting labeled training image samples.

[0168] The labeled training image samples are input into the student network for training, and a first loss function is determined between the categories identified by the student network and the labeled categories of the labeled training image samples.

[0169] For example, such as Figure 3 In the process, the labeled training image sample 1 contains human body category and object category. The labeled training image sample 1 is input into the student network, and the student network performs object detection on the labeled training image sample 1. Based on the identified category and the labeled category, the first loss function is determined, and the student network is trained.

[0170] The second aspect: targeting unlabeled training image samples.

[0171] 1) To improve data diversity, data augmentation methods can be used to augment unlabeled training image samples. For example, one or more of the following methods can be used: resizing, flipping, rotating, scaling, cropping, translating, adding Gaussian noise, changing image brightness, or changing image saturation and contrast. The specific method is not limited.

[0172] like Figure 3 In this context, weak data augmentation can be understood as a simple data transformation, such as resizing, while strong data augmentation can be understood as a complex data transformation, such as Gaussian noise. In order to ensure the performance of the teacher network, weak data augmentation is adopted for the teacher network.

[0173] 2) Input unlabeled training image samples into the teacher network to obtain the first detection result output by the teacher network, for example... Figure 3 The prediction map represents all identified candidate boxes. Each dot in the map can be understood as a candidate box. The first object detection behavior information in the object detection process is input into the student network to supervise the training of the student network, instead of using sparse pseudo-labels in related technologies. In this way, richer and denser supervision information can be extracted from the teacher network, which can improve the training accuracy of the student network.

[0174] See Figure 4 The diagram shown illustrates the logical principle of the training method for the object detection network in this embodiment of the present disclosure, specifically for an unlabeled training image sample set. Figure 4 As shown, Figure 4 The upper half describes the NMS process of the teacher network. The teacher network identifies multiple candidate boxes from unlabeled training image samples and clusters these candidate boxes, for example... Figure 4 Three clusters were generated, each containing multiple candidate boxes. For each cluster, scores were sorted, and the candidate box with the highest confidence was selected from the multiple candidate boxes to represent the target to be detected for that cluster.

[0175] Therefore, in this embodiment of the disclosure, from Figure 4 As shown, the teacher network outputs the first object detection behavior information during the object detection process to the student network as supervisory information for training. The first object detection behavior information includes the behavior performance of each cluster obtained by clustering. Each cluster includes category labels, candidate boxes, candidate box ranking distribution, etc. In this embodiment, training the student network can include two main aspects: 1) Reverse NMS clustering: that is, letting the student network learn the NMS process of the teacher network and generate the same NMS clusters as the teacher network. This part is mainly based on the category of the candidate box with the highest confidence identified by the teacher network, and the candidate box with the highest confidence. The candidate box information here can be understood as the location region information of the candidate box. The category of the candidate box with the highest confidence can be used as the category label, and the candidate box with the highest confidence can be used as the target basis when calculating the regression loss function; such as Figure 4 In the reverse NMS clustering, the "star" symbol represents the category of the candidate box with the highest confidence, and the "triangle" symbol represents the candidate box with the highest confidence. In this embodiment, the category loss function for training the student network is obtained based on the category of the candidate box with the highest confidence identified by the teacher network, and the regression loss function for training the student network is obtained based on the candidate box with the highest confidence identified by the teacher network. 2) NMS ranking matching: that is, ensuring that the candidate box ranking distribution of the teacher network and the student network is the same in each cluster, and then using the first object detection information mined from the teacher network to supervise the training of the student network, to obtain the finally trained object detection network, such as Figure 4 The ranking distribution of candidate boxes identified by the teacher network is input into the student network to obtain the ranking distribution loss function when training the student network.

[0176] Thus, in this embodiment, the training of the student network can be based on denser and richer supervisory information from the teacher network. Compared with related technologies, the use of pseudo-labels makes the semi-supervised object detection network more accurate and efficient. Furthermore, by imitating the object detection behavior information of the teacher network through the student network, the training class labels of the student network can be obtained without relying on the label assignment method. By not generating the training class labels of the student network through the label assignment method, the impact of inaccurate pseudo-labels can also be reduced, thereby improving the accuracy and performance of the student network training and enhancing the accuracy and reliability of the object detection network.

[0177] In addition, it should be noted that the application scenarios of the target detection method and the target detection network training method in the embodiments of this disclosure are not limited. The target detection network obtained by training on the training image samples corresponding to different application scenarios can be applied to the corresponding application scenarios for target detection. Specifically, this disclosure provides several possible implementation methods for target detection methods in different application scenarios.

[0178] 1) In the case where the image to be detected is a monitoring image captured by a monitoring device, a possible implementation method is provided for obtaining the category of the target detected from the image to be detected: using a target detection network, human body is detected in the monitoring image to be detected, to determine whether a human body exists in the monitoring image to be detected, and if a human body is determined to exist, to determine whether the human body is the target user.

[0179] In other words, in this embodiment of the present disclosure, in an intelligent monitoring scenario, a target user can be detected by a target detection network generated based on the training method of the target detection network in this embodiment of the present disclosure, and the monitoring data can be analyzed automatically.

[0180] 2) When the image to be detected is a traffic image of a set road segment, a possible implementation method is provided for obtaining the category of the target detected from the image to be detected: using a target detection network, target detection is performed on the traffic image to be detected, the targets belonging to the vehicle category in the traffic image to be detected are determined, and the number of targets belonging to the vehicle category is determined; based on the determined number of targets belonging to the vehicle category, the traffic congestion level of the set road segment is obtained.

[0181] In this embodiment, the target detection method can also be applied to intelligent transportation scenarios. It can perform real-time monitoring and analysis of designated road segments to determine the traffic congestion level of those segments. Based on the determined congestion level, it can then suggest corresponding traffic strategies or control relevant traffic equipment. For example, if the current traffic congestion level is high, an alarm can be issued to alert relevant personnel to take measures to alleviate congestion. Alternatively, traffic light timings can be intelligently adjusted based on the congestion level to minimize congestion.

[0182] Of course, the target detection method in this embodiment can also be applied to other application scenarios, such as autonomous driving and medical fields, and there are no limitations on this.

[0183] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0184] Based on the same inventive concept, this disclosure also provides a target detection device corresponding to the target detection method and a target detection training device corresponding to the target detection network training method. Since the principle of the device in this disclosure for solving the problem is similar to the target detection method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0185] Reference Figure 5 The diagram shown is a schematic representation of a target detection device provided in an embodiment of this disclosure. The device includes:

[0186] The second acquisition module 50 is used to acquire the image to be detected;

[0187] The detection module 51 is used to perform target detection on the image to be detected based on a trained target detection network, and obtain the category of the target detected in the image to be detected.

[0188] The trained target detection network is generated based on the training method of the target detection network in the embodiments of this disclosure.

[0189] In an optional implementation, if the image to be detected is a monitoring image captured by a monitoring device, then when obtaining the category of the target detected from the image to be detected, the detection module 51 is used to:

[0190] Using the target detection network, human targets are detected in the surveillance image to be detected to determine whether a human body exists in the surveillance image to be detected, and if a human body is found, whether the human body is the target user.

[0191] In an optional implementation, if the image to be detected is a traffic image of a defined road segment, then when obtaining the category of the target detected from the image to be detected, the detection module 51 is used to:

[0192] Using the target detection network, target detection is performed on the traffic image to be detected to determine the targets belonging to the vehicle category in the traffic image to be detected, and the number of targets belonging to the vehicle category is determined.

[0193] Based on the target number of vehicles belonging to a given vehicle category, the traffic congestion level of the designated road segment is obtained.

[0194] Reference Figure 6 The diagram shown is a schematic of a training apparatus for an object detection network provided in an embodiment of this disclosure. The apparatus includes:

[0195] The first acquisition module 60 is used to acquire a set of labeled training images and a set of unlabeled training images.

[0196] The first training branch module 61 is used to input each labeled training image sample in the labeled training image sample set into the student network for training, and determine the first loss function, wherein the first loss function is the loss function between the category identified by the student network and the labeled category of the labeled training image sample;

[0197] The second training branch module 62 is used to input each unlabeled training image sample in the unlabeled training image sample set into the teacher network to obtain first target detection behavior information for each unlabeled training image sample; train the student network according to each unlabeled training image sample and the first target detection behavior information to obtain second target detection behavior information for each unlabeled training image sample, determine a second loss function, and determine a target loss function according to the first loss function and the second loss function; and obtain a target detection network when the number of iterations reaches a threshold or the target loss function meets a preset convergence condition, wherein the second loss function is a loss function between the second target detection behavior information and the first target detection behavior information.

[0198] In an optional implementation, when multiple first candidate boxes identified in the first target detection behavior information are clustered into multiple clusters, the second training branch module 62 is used to: determine the target loss function based on the first loss function and the second loss function when determining the target loss function.

[0199] For each of the plurality of clusters, a second loss function corresponding to that cluster is obtained;

[0200] The third loss function is obtained by weighted averaging of the multiple second loss functions.

[0201] The target loss function is obtained based on the third loss function and the first loss function.

[0202] In one optional implementation, when obtaining the second loss function corresponding to each of the plurality of clusters, the second training branch module 62 is used to:

[0203] For each of the multiple clusters, a category loss function is obtained by considering the loss function between the category of each second candidate box of the target identified by the student network and the category of the first candidate box with the highest confidence identified by the teacher network.

[0204] Based on the loss function between each second candidate box of the target identified by the student network and the first candidate box with the highest confidence identified by the teacher network, a regression loss function is obtained;

[0205] Based on the ranking distribution of the second candidate boxes of the target identified by the student network and the ranking distribution of the first candidate boxes identified by the teacher network, a ranking distribution loss function is obtained.

[0206] The second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function.

[0207] In an optional implementation, when determining the second loss function based on the category loss function, the regression loss function, and the ranking distribution loss function, the second training branch module 62 is used to:

[0208] The sum between the category loss function and the regression loss function is obtained, and the product between the ranking distribution loss function and the first weight value is obtained. The sum and the product are added together to obtain the second loss function, wherein the first weight value is used to control the loss weight of the ranking distribution.

[0209] In an optional implementation, when the number of training iterations reaches a threshold or the target loss function satisfies a preset convergence condition, the second training branch module 62 is used to:

[0210] For the current number of iterations, the student network is trained based on the labeled training image samples, the unlabeled training image samples, and the first target detection behavior information. The model parameters of the teacher network for the next iteration are determined based on the model parameters of the student network obtained from the current number of iterations, until the number of iterations reaches a threshold or the target loss function converges, thus obtaining the target detection network.

[0211] In one optional implementation, when obtaining the target detection network, the first acquisition module 60 is used to:

[0212] The teacher network or the student network obtained when the number of iterations reaches a threshold or when the target loss function converges is used as the target detection network.

[0213] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0214] This disclosure also provides a computer device, such as... Figure 7 The diagram shown is a schematic representation of a computer device structure provided in an embodiment of this disclosure, including:

[0215] A processor 71 and a memory 72; the memory 72 stores machine-readable instructions executable by the processor 71, and the processor 71 executes the machine-readable instructions stored in the memory 72. When the machine-readable instructions are executed by the processor 71, the processor 71 performs the following steps:

[0216] Obtain the labeled training image sample set and the unlabeled training image sample set;

[0217] Each labeled training image sample in the labeled training image sample set is input into the student network for training, and a first loss function is determined, wherein the first loss function is the loss function between the category identified by the student network and the labeled category of the labeled training image sample;

[0218] Each unlabeled training image sample in the unlabeled training image sample set is input into the teacher network to obtain the first target detection behavior information for each unlabeled training image sample.

[0219] The student network is trained based on the unlabeled training image samples and the first target detection behavior information to obtain the second target detection behavior information for each unlabeled training image sample. A second loss function is determined, and a target loss function is determined based on the first loss function and the second loss function. When the number of iterations reaches a threshold or the target loss function satisfies a preset convergence condition, a target detection network is obtained. The second loss function is the loss function between the second target detection behavior information and the first target detection behavior information.

[0220] In an optional implementation, when multiple first candidate boxes identified in the first target detection behavior information are clustered into multiple clusters, a target loss function is determined based on the first loss function and the second loss function. Specifically, the processor 71 is used to:

[0221] For each of the plurality of clusters, a second loss function corresponding to that cluster is obtained;

[0222] The third loss function is obtained by weighted averaging of the multiple second loss functions.

[0223] The target loss function is obtained based on the third loss function and the first loss function.

[0224] In one optional implementation, for each of the plurality of clusters, a second loss function corresponding to the cluster is obtained, and the processor 71 is used to:

[0225] For each of the multiple clusters, a category loss function is obtained by considering the loss function between the category of each second candidate box of the target identified by the student network and the category of the first candidate box with the highest confidence identified by the teacher network.

[0226] Based on the loss function between each second candidate box of the target identified by the student network and the first candidate box with the highest confidence identified by the teacher network, a regression loss function is obtained;

[0227] Based on the ranking distribution of the second candidate boxes of the target identified by the student network and the ranking distribution of the first candidate boxes identified by the teacher network, a ranking distribution loss function is obtained.

[0228] The second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function.

[0229] In one optional implementation, the second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function, and the processor 71 is configured to:

[0230] The sum between the category loss function and the regression loss function is obtained, and the product between the ranking distribution loss function and the first weight value is obtained. The sum and the product are added together to obtain the second loss function, wherein the first weight value is used to control the loss weight of the ranking distribution.

[0231] In an optional implementation, when the number of iterations reaches a threshold or the target loss function satisfies a preset convergence condition, the target detection network is obtained, and the processor 71 is used to:

[0232] For the current number of iterations, the student network is trained based on the labeled training image samples, the unlabeled training image samples, and the first target detection behavior information. The model parameters of the teacher network for the next iteration are determined based on the model parameters of the student network obtained from the current number of iterations, until the number of iterations reaches a threshold or the target loss function converges, thus obtaining the target detection network.

[0233] In one optional implementation, the processor 71 is configured to: use the teacher network or the student network obtained by iterative training when the number of iterations reaches a threshold or the target loss function converges as the target detection network.

[0234] In another possible embodiment, processor 71 is used to perform the following steps:

[0235] Acquire the image to be detected;

[0236] The target detection network generated using the target detection network training method described in the first aspect above is used to perform target detection on the image to be detected, thereby obtaining the category of the target detected in the image to be detected.

[0237] In one optional implementation, the image to be detected is a monitoring image captured by a monitoring device. Then, the processor 71 obtains the category of the target detected in the image to be detected and is used to:

[0238] Using the target detection network, human targets are detected in the surveillance image to be detected to determine whether a human body exists in the surveillance image to be detected, and if a human body is found, whether the human body is the target user.

[0239] In one optional implementation, if the image to be detected is a traffic image of a defined road segment, then the processor 71 obtains the category of the target detected in the image to be detected and is used to:

[0240] Using the target detection network, target detection is performed on the traffic image to be detected to determine the targets belonging to the vehicle category in the traffic image to be detected, and the number of targets belonging to the vehicle category is determined.

[0241] Based on the target number of vehicles belonging to a given vehicle category, the traffic congestion level of the designated road segment is obtained.

[0242] The aforementioned memory 72 includes a main memory 721 and an external memory 722; the main memory 721, also known as internal memory, is used to temporarily store the computational data in the processor 71, as well as the data exchanged with external memory 722 such as a hard disk. The processor 71 exchanges data with the external memory 722 through the main memory 721.

[0243] The specific execution process of the above instructions can be referred to the steps of the target detection method described in the embodiments of this disclosure, and will not be repeated here.

[0244] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the target detection method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0245] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the target detection method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0246] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0247] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate; the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of this embodiment.

[0248] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0249] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0250] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A training method for an object detection network, characterized in that, include: Obtain the labeled training image sample set and the unlabeled training image sample set; Each labeled training image sample in the labeled training image sample set is input into the student network for training, and a first loss function is determined, wherein the first loss function is the loss function between the category identified by the student network and the labeled category of the labeled training image sample; Each unlabeled training image sample in the unlabeled training image sample set is input into the teacher network to obtain the first target detection behavior information for each unlabeled training image sample. The student network is trained based on the unlabeled training image samples and the first target detection behavior information to obtain the second target detection behavior information for each unlabeled training image sample. A second loss function is determined, and a target loss function is determined based on the first loss function and the second loss function. When the number of iterations reaches a threshold or the target loss function satisfies a preset convergence condition, a target detection network is obtained. The second loss function is the loss function between the second target detection behavior information and the first target detection behavior information. Where, if multiple first candidate boxes identified in the first target detection behavior information are clustered into multiple clusters, then the target loss function is determined based on the first loss function and the second loss function, including: For each of the plurality of clusters, a second loss function corresponding to that cluster is obtained; The third loss function is obtained by weighted averaging of the multiple second loss functions. The target loss function is obtained based on the third loss function and the first loss function.

2. The method according to claim 1, characterized in that, For each of the plurality of clusters, a second loss function corresponding to the cluster is obtained, including: For each of the multiple clusters, a category loss function is obtained by considering the loss function between the category of each second candidate box of the target identified by the student network and the category of the first candidate box with the highest confidence identified by the teacher network. Based on the loss function between each second candidate box of the target identified by the student network and the first candidate box with the highest confidence identified by the teacher network, a regression loss function is obtained; Based on the ranking distribution of the second candidate boxes of the target identified by the student network and the ranking distribution of the first candidate boxes identified by the teacher network, a ranking distribution loss function is obtained. The second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function.

3. The method according to claim 2, characterized in that, The second loss function is determined based on the category loss function, the regression loss function, and the ranking distribution loss function, including: The sum between the category loss function and the regression loss function is obtained, and the product between the ranking distribution loss function and the first weight value is obtained. The sum and the product are added together to obtain the second loss function, wherein the first weight value is used to control the loss weight of the ranking distribution.

4. The method according to any one of claims 1 to 3, characterized in that, When the number of iterations reaches a threshold or the target loss function satisfies a preset convergence condition, obtaining the target detection network includes: For the current number of iterations, the student network is trained based on the labeled training image samples, the unlabeled training image samples, and the first target detection behavior information. The model parameters of the teacher network for the next iteration are determined based on the model parameters of the student network obtained from the current number of iterations, until the number of iterations reaches a threshold or the target loss function converges, thus obtaining the target detection network.

5. The method according to claim 4, characterized in that, Obtaining the target detection network includes: The teacher network or the student network obtained when the number of iterations reaches a threshold or when the target loss function converges is used as the target detection network.

6. A target detection method, characterized in that, include: Acquire the image to be detected; Using a trained target detection network generated based on the target detection network training method of any one of claims 1 to 5, target detection is performed on the image to be detected to obtain the category of the target detected in the image to be detected.

7. The method according to claim 6, characterized in that, If the image to be detected is a monitoring image captured by a monitoring device, then the categories of targets detected from the image to be detected are obtained, including: Using the target detection network, human targets are detected in the surveillance image to be detected to determine whether a human body exists in the surveillance image to be detected, and if a human body is found, whether the human body is the target user.

8. The method according to claim 6, characterized in that, If the image to be detected is a traffic image of a defined road segment, then the categories of targets detected from the image to be detected are obtained, including: Using the target detection network, target detection is performed on the traffic image to be detected to determine the targets belonging to the vehicle category in the traffic image to be detected, and the number of targets belonging to the vehicle category is determined. Based on the target number of vehicles belonging to a given vehicle category, the traffic congestion level of the designated road segment is obtained.

9. A training device for an object detection network, characterized in that, include: The first acquisition module is used to acquire the labeled training image sample set and the unlabeled training image sample set; The first training branch module is used to input each labeled training image sample in the labeled training image sample set into the student network for training, and determine the first loss function, wherein the first loss function is the loss function between the category identified by the student network and the labeled category of the labeled training image sample; The second training branch module is used to input each unlabeled training image sample from the unlabeled training image sample set into the teacher network to obtain first target detection behavior information for each unlabeled training image sample; train the student network based on each unlabeled training image sample and the first target detection behavior information to obtain second target detection behavior information for each unlabeled training image sample, determine a second loss function, and determine a target loss function based on the first loss function and the second loss function; and obtain a target detection network when the number of iterations reaches a threshold or the target loss function meets a preset convergence condition, wherein the second loss function is a loss function between the second target detection behavior information and the first target detection behavior information; Wherein, when multiple first candidate boxes identified in the first target detection behavior information are clustered into multiple clusters, the second training branch module is used to: For each of the plurality of clusters, a second loss function corresponding to that cluster is obtained; The third loss function is obtained by weighted averaging of the multiple second loss functions. The target loss function is obtained based on the third loss function and the first loss function.

10. A target detection device, characterized in that, include: The second acquisition module is used to acquire the image to be detected; The detection module is used to perform target detection on the image to be detected using a target detection network generated based on the training method of the target detection network according to any one of claims 1 to 5, and to obtain the category of the target detected in the image to be detected.

11. A computer device, characterized in that, include: The processor and the memory, wherein the memory stores machine-readable instructions executable by the processor, the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the processor performs the steps of the training method for the target detection network as described in any one of claims 1 to 5, or the target detection method as described in any one of claims 6 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer device, performs the steps of the training method for the target detection network as described in any one of claims 1 to 5, or the target detection method as described in any one of claims 6 to 8.

Images