Target detection methods, apparatus, equipment and storage media

Abstract

This application provides a target detection method, apparatus, device, and storage medium. The method includes: acquiring an image to be detected; inputting the image to be detected into a pre-trained target detection model to obtain a target recognition result for the image; wherein the target detection model is obtained by sequentially setting one of multiple detection models as a teacher model and the remaining models as student models, and training the student models based on the teacher model. Since the target detection model is trained from multiple detection models, it can absorb the correct detection capabilities of different detection models, thereby improving the accuracy of target detection. Therefore, there is no need to introduce a complex backbone network or a complex bottleneck layer into the target detection model; a high target detection accuracy can be achieved through a simple network architecture, and the relatively simple network architecture also improves the efficiency of target detection.

Description

[0001] The priority information claimed in this application is as follows: Application No.: 202210665491.6, Application Title: Training Method, Apparatus, Device and Storage Medium for Detection Model, Application Date: June 14, 2022. Technical Field

[0002] This application relates to the field of computer technology, and in particular to a target detection method, apparatus, device, and storage medium. Background Technology

[0003] With the continuous development of deep learning, it has been widely applied in the field of image detection, for example, to identify the category and / or location information of target objects in images using neural networks. Typically, to improve the accuracy of target detection in image data, the network structure in the neural network can be further optimized by introducing more complex backbone networks or bottleneck layers. However, in actual image detection processes, the introduction of more complex backbone networks or bottleneck layers makes the detection network quite complex and large, leading to slower target detection efficiency. Summary of the Invention

[0004] To address the technical problems of traditional technologies, embodiments of this application provide a target detection method, apparatus, device, and storage medium.

[0005] In a first aspect, embodiments of this application provide a target detection method, including:

[0006] Acquire the image to be detected;

[0007] The image to be detected is input into a pre-trained target detection model to obtain the target recognition result of the image to be detected; wherein, the target detection model is obtained by sequentially setting one of a plurality of detection models as a teacher model and the remaining models as student models, and training the student model based on the teacher model.

[0008] Secondly, embodiments of this application provide a target detection device, comprising:

[0009] The acquisition module is used to acquire the image to be detected;

[0010] The processing module is used to input the image to be detected into a pre-trained target detection model to obtain the target recognition result of the image to be detected; wherein, the target detection model is obtained by sequentially setting one of a plurality of detection models as a teacher model and the remaining models as student models, and training the student model based on the teacher model.

[0011] Thirdly, embodiments of this application provide an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the processors implement the steps of the target detection method provided in the first aspect of embodiments of this application.

[0012] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the target detection method provided in the first aspect of embodiments of this application.

[0013] The technical solution provided in this application, after acquiring the image to be detected, inputs the image to be detected into a pre-trained target detection model to obtain the target recognition result of the image to be detected. Since the target detection model is obtained by sequentially setting one of multiple detection models as the teacher model and the remaining models as student models, and training the student models based on the teacher model, the target detection model is trained through multiple detection models. This allows it to absorb the correct detection capabilities of different detection models, thereby improving the accuracy of target detection. Therefore, there is no need to introduce complex backbone networks or complex bottleneck layers into the target detection model; a high target detection accuracy can be achieved through a simple network architecture, and the relatively simple network architecture also improves the efficiency of target detection. Attached Figure Description

[0014] Figure 1 This is a schematic flowchart of a target detection method provided in an embodiment of this application;

[0015] Figure 2 A schematic flowchart illustrating the training process of the target detection model provided in this application embodiment;

[0016] Figure 3 A flowchart illustrating the determination of a first loss value provided in an embodiment of this application;

[0017] Figure 4 A schematic diagram illustrating the principle of determining a first loss value provided in an embodiment of this application;

[0018] Figure 5 A flowchart illustrating the determination of a second loss value provided in an embodiment of this application;

[0019] Figure 6 A schematic diagram illustrating the principle of determining a second loss value provided in an embodiment of this application;

[0020] Figure 7 A flowchart illustrating the determination of a third loss value provided in an embodiment of this application;

[0021] Figure 8 A schematic diagram illustrating the principle of determining a third loss value provided in an embodiment of this application;

[0022] Figure 9 This is a schematic diagram of the target detection device provided in an embodiment of this application;

[0023] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit its scope. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0025] Figure 1 This is a schematic flowchart of a target detection method provided in an embodiment of this application. Figure 1 As shown, the method may include:

[0026] S101. Obtain the image to be detected.

[0027] S102. Input the image to be detected into a pre-trained target detection model to obtain the target recognition result of the image to be detected.

[0028] Specifically, the aforementioned image to be detected can be an image for which target object detection is required. The target object can be a person, animal, object, vehicle, or other similar object. In practical applications, the image to be detected can be acquired using a camera device or obtained from a corresponding storage device. After obtaining the image to be detected, it can be input into a pre-trained target detection model. This target detection model can include a backbone network, a bottleneck layer, and a detection head. The target detection model extracts features from the image to be detected and determines the target recognition result based on the extracted features.

[0029] The aforementioned object detection model is obtained by sequentially designating one detection model as the teacher model and the rest as student models, and training the student models based on the teacher model. The teacher model guides the student models in learning aspects such as image feature loss at the bottleneck layer, object relevance (also known as correlation) loss, and object regression and classification loss. Each detection model acts as both a teacher and a student model, with all models participating in knowledge guidance and collectively learning the correct detection capabilities of other models. During the training phase, they continuously improve their own detection performance and reduce false positive rates, while also imparting correct detection knowledge to other models, thereby improving the detection performance of all models. Therefore, the detection model does not require a complex backbone network or a complex bottleneck layer; a high object detection accuracy can be achieved through a simple network architecture.

[0030] The technical solution provided in this application, after acquiring the image to be detected, inputs the image to be detected into a pre-trained target detection model to obtain the target recognition result of the image to be detected. Since the target detection model is obtained by sequentially setting one of multiple detection models as the teacher model and the remaining models as student models, and training the student models based on the teacher model, it can absorb the correct detection capabilities of different detection models, thereby improving the accuracy of target detection. Therefore, there is no need to introduce a complex backbone network or a complex bottleneck layer into the target detection model; a high target detection accuracy can be achieved through a simple network architecture, and the relatively simple network architecture also improves the efficiency of target detection.

[0031] In one embodiment, the training process of the above-described object detection model is also provided. Based on the above embodiment, optionally, as follows... Figure 2 As shown, prior to S101 above, the method further includes:

[0032] S201. Obtain multiple training images and multiple detection models to be trained.

[0033] S202. Set one of the multiple detection models as the teacher model and the rest as the student models.

[0034] In this embodiment, multiple detection models can be created and trained synchronously. During training, each detection model can be sequentially designated as a teacher model, and the remaining models as student models. This allows the student models to learn more of the black-box knowledge from the teacher models. Specifically, one detection model can be designated as the teacher model, and the rest as student models. Taking n detection models as an example, detection model 1 can be designated as the teacher model, and detection models 2 through n can be designated as student models. The model parameters and sizes of the multiple detection models can be different.

[0035] S203. Use the teacher model and each student model to identify the target object in each training image, and obtain the initial prediction results of the teacher model and each student model for each training image.

[0036] Each detection model may include a corresponding backbone network, bottleneck layer, and detection head. The aforementioned training images serve as training data for each detection model. After obtaining the training data, it is input into the teacher model and each student model. Feature extraction is performed through the backbone network and bottleneck layer in the teacher and student models, and the initial prediction results for each training image are obtained through the prediction of the detection head.

[0037] S204. Based on the expected recognition result of each training image, the initial prediction result corresponding to the teacher model is filtered to obtain the target prediction result.

[0038] Here, the desired recognition result (i.e., prior knowledge) refers to the label data corresponding to each training image, which can be manually labeled information about the target object, providing accurate information. During mutual learning, the prior knowledge corresponding to each training image is combined to filter the initial prediction results output by the teacher model, eliminating erroneous initial prediction results and retaining correct ones, thereby obtaining the target prediction result. Optionally, the prior knowledge here can be manually labeled prior boxes (i.e., the desired recognition result). Intersection-over-Union (IoU) processing is performed on the prior boxes and candidate boxes (i.e., the initial prediction results), and candidate boxes with IoU results greater than a preset threshold are determined as target candidate boxes (i.e., the target prediction results). The preset threshold can be set based on requirements; optionally, it can be set to 0.4. The target candidate boxes here can be considered as candidate boxes with high accuracy.

[0039] Furthermore, based on the target prediction results, the student model is guided to learn the correct knowledge of the teacher model, and learns the knowledge of the teacher model in the following three directions: bottleneck layer, relevance of the detected target, and regression and classification of the detected target. Through mutual learning in these three directions, the detection accuracy of the student model for images can be improved.

[0040] S205. Based on the target prediction results and the initial prediction results corresponding to each student model, determine the first loss value of each student model in terms of learning the teacher model in terms of the bottleneck layer, the second loss value in terms of the relevance of the detected target, and the third loss value in terms of the classification and regression of the detected target.

[0041] In this process, the initial prediction results output by the teacher model are filtered based on prior knowledge to select the correct target prediction results. The loss values ​​of each student model in the three aspects mentioned above are then determined using the target prediction results and the initial prediction results output by each student model. For example, L2 loss can be calculated based on the target prediction results of the teacher model and the initial prediction results of the student models. Continuing with the example in S202 above, after steps S203-S205, the loss values ​​of student models 2 to n in learning from teacher model 1 in the three aspects are obtained.

[0042] S206. Set the next detection model among multiple detection models as the teacher model and the remaining models as student models, and repeat the step of using the teacher model and each student model to identify the target object in each training image until the last detection model among multiple detection models is set as the teacher model and the remaining models are set as student models, thereby obtaining the first loss value set of each student model in terms of the bottleneck layer, the second loss value set in terms of the relevance of the detected target, and the third loss value set in terms of the classification and regression of the detected target.

[0043] Furthermore, another detection model is reset to the teacher model, and the remaining models are student models. Continuing with the example in S202, detection model 2 out of the n detection models is again set as the teacher model, and detection models 1, 3 to n are all set as student models. Next, continue executing the above S203-S205 process until detection model n is set as the teacher model and detection models 1 to n-1 are set as student models. Continue executing the S203-S205 process to obtain the first loss value set of detection model 1 as a student model, the second loss value set of detection target relevance, and the third loss value set of detection target regression and classification, which are learned from n-1 teacher models (i.e., detection models 2 to n). Similarly, when detection model 2 is a student model, it learns the first loss value set of detection model 1 as a student model, the second loss value set of detection target relevance, and the third loss value set of detection target regression and classification, which are learned from n-1 teacher models (i.e., detection models 1, 3 to n).

[0044] The first, second, and third loss sets include the loss values ​​learned by the student model from multiple teacher models. By processing the first, second, and third loss sets, each student model, guided by multiple teacher models, selects the optimal processing method. This allows the student model to compensate for its own model's shortcomings to the greatest extent, enabling it to learn the knowledge of the best teacher model and comprehensively learn the knowledge of each teacher model.

[0045] S207. Process the first loss value set, the second loss value set, and the third loss value set respectively to obtain the corresponding first target loss value, second target loss value, and third target loss value.

[0046] Optionally, the process of determining the first target loss value for mutual learning among student models at the bottleneck layer can be as follows: the maximum loss value in each set of first loss values ​​is determined as the first target loss value for mutual learning among student models at the bottleneck layer. In this way, the student models are optimized using this first target loss value, enabling them to learn the knowledge of the best teacher model among multiple teacher models. Taking detection model 1 as an example, the first loss set corresponding to detection model 1 includes first loss value 1, first loss value 2, ..., first loss value n-1. First loss value 1 represents the loss value for mutual learning between detection model 1 and detection model 2 at the bottleneck layer; first loss value 2 represents the loss value for mutual learning between detection model 1 and detection model 3 at the bottleneck layer; and so on. First loss value n-1 represents the loss value for mutual learning between detection model 1 and detection model n at the bottleneck layer. Thus, the maximum loss value can be determined from first loss value 1, first loss value 2 to first loss value n-1. Assuming the maximum loss value is first loss value 2, then first loss value 2 can be determined as the first target loss value for mutual learning among detection model 1 at the bottleneck layer. Other detection models can be used as student models for calculation with reference to detection model 1, which will not be elaborated here.

[0047] Optionally, the process of determining the second objective loss value for mutual learning among student models in terms of the relevance of the detection target can be as follows: The average of each loss value in each set of second loss values ​​is calculated, and the result is determined as the second objective loss value for mutual learning among student models in terms of the relevance of the detection target. In this way, the student models are optimized using this second objective loss value, enabling them to comprehensively learn the knowledge of multiple teacher models. Taking detection model 1 as an example, the second loss set corresponding to detection model 1 includes second loss value 1, second loss value 2, ..., second loss value n-1. Second loss value 1 represents the loss value for mutual learning between detection model 1 and detection model 2 in terms of the relevance of the detection target; second loss value 2 represents the loss value for mutual learning between detection model 1 and detection model 3 in terms of the relevance of the detection target; and so on. Second loss value n-1 represents the loss value for mutual learning between detection model 1 and detection model n in terms of the relevance of the detection target. Therefore, the average of second loss values ​​1, 2, to n-1 can be calculated, and the result is determined as the second objective loss value for mutual learning between detection model 1 and detection model n in terms of the relevance of the detection target.

[0048] Optionally, the process of determining the third objective loss value for mutual learning among the student models in terms of regression and classification of the detection target can be as follows: calculate the mean of each loss value in each set of second loss values, and determine the result as the third objective loss value for mutual learning among the student models in terms of regression and classification of the detection target. In this way, the student models are optimized through this third objective loss value, enabling them to comprehensively learn the knowledge of multiple teacher models. Taking detection model 1 as the student model as an example, the third loss set corresponding to detection model 1 includes third loss value 1, third loss value 2, ..., third loss value n-1. Third loss value 1 represents the loss value of detection model 1 learning from detection model 2 in terms of regression and classification of the detection target. Third loss value 2 represents the loss value of detection model 1 learning from detection model 3 in terms of regression and classification of the detection target, and so on. Third loss value n-1 represents the loss value of detection model 1 learning from detection model n in terms of regression and classification of the detection target. In this way, the average value of third loss value 1, third loss value 2 to third loss value n-1 can be calculated, and the calculation result can be determined as the third target loss value of detection model 1 learning from each other in terms of regression and classification of the detection target.

[0049] S208. Based on the first target loss value, the second target loss value, the third target loss value, and the fourth target loss value, train each detection model to obtain the corresponding target detection model.

[0050] The fourth target loss value is determined based on the expected recognition result and the initial prediction result corresponding to each student model. Specifically, the fourth target loss value can be the regression loss value and classification loss value of each detection model learning the expected recognition result (i.e., prior knowledge) of each training image. In other words, the regression loss value can be calculated using prior knowledge and the location information of the target object obtained after processing the training image by the detection model, and the classification loss value can be calculated using prior knowledge and the category of the target object obtained after processing the training image by the detection model. The first, second, and third target loss values ​​obtained during mutual learning, and the fourth target loss value (i.e., regression and classification loss values) obtained during self-learning are summed to obtain the total loss value of each detection model. Based on each total loss value, independent backpropagation is performed on each detection model to optimize the parameters of each detection model until each detection model converges, thereby obtaining the corresponding target detection model and realizing the simultaneous training of multiple detection models.

[0051] In this embodiment, all detection models serve as both teacher and student models. All models participate in knowledge guidance and learn correct black-box knowledge from other detection models. During the training phase, they continuously improve their own detection performance and reduce false positive rates, while also imparting correct knowledge to other detection models, thereby enhancing the detection performance of all models. Furthermore, prior knowledge is used to filter the detection outputs of the teacher models, enabling the student models to learn the correct knowledge from the teacher models, further improving the detection performance of the target detection models.

[0052] In one embodiment, alternatively, such as Figure 3 and Figure 4 As shown, the process of determining the first loss value of each student model in the bottleneck layer based on the target prediction result and the initial prediction result corresponding to each student model can be as follows:

[0053] S301. Map the target prediction result to the bottleneck layer to obtain the target image feature vector output by the teacher model at the bottleneck layer.

[0054] After obtaining the target prediction result, the target prediction result (i.e. target candidate box) is mapped to the bottleneck layer of the teacher model to obtain the target image feature vector (i.e. processed feature value) output by the teacher model at the bottleneck layer.

[0055] Optionally, since the feature values ​​output by the bottleneck layer are limited due to the influence of traditional convolution, a self-attention module can be used to perform a self-attention operation on the target image feature vector (i.e., the processed feature values), making the feature values ​​extracted by the teacher model more global and representative. Furthermore, optionally, a self-attention operation can also be performed on the initial image feature vector (or feature values) output by the bottleneck layer of each student model to expand the receptive field of the feature values, allowing the student models to learn more connections between pixels, making the feature values ​​extracted by the student models more global and representative.

[0056] S302. Map each initial prediction result to the bottleneck layer to obtain the initial image feature vector output by each student model at the bottleneck layer.

[0057] S303. Map the target image feature vector to each student model to obtain the mapped image feature vector of each student model.

[0058] In this process, the target image feature vector is mapped to each student model, based on each student model, to obtain the mapped image feature vector (or mapped feature value) of each student model.

[0059] S304. Based on the feature vectors of each initial image and the feature vector of the mapped image, determine the first loss value of each student model in learning the teacher model at the bottleneck layer.

[0060] In this context, the processed feature values ​​can be considered foreground feature values, while the feature values ​​in the training image other than the foreground feature values ​​can be considered background feature values. Therefore, the number of foreground and background feature values ​​can be counted based on the processed feature values. After obtaining the initial image feature vector output from the bottleneck layer of each student model, the mapped image feature vector, the number of background feature values, and the number of foreground feature values, the loss value (loss) for each student model learning the teacher model at the bottleneck layer can be determined according to the following formula 1. fea .

[0061] Formula 1:

[0062] in, The mapped image feature vector (i.e., the mapped feature value) is... The initial image feature vector (i.e., eigenvalues) output by the bottleneck layer of the student model, num fg num represents the number of foreground eigenvalues. bg C represents the number of background feature values, C represents the number of channels in the bottleneck layer of the teacher model, and W represents the number of background feature values. true For the width of the bottleneck layer in the teacher model, H true For the bottleneck layer of the teacher model, α and β are hyperparameters, which can be set to 0.1 in this embodiment. When calculating foreground feature values, M... i,j When M is set to 1, it is used for background feature value calculation. i,j Take 0.

[0063] It should be noted that since the foreground feature values ​​are fewer than the background feature values, the loss values ​​corresponding to the foreground feature values ​​and the background feature values ​​are calculated separately, and then the individual loss values ​​are summed. This operation can increase the influence of the foreground feature values ​​on the detection model and reduce the influence of the background feature values ​​on the detection model.

[0064] In this embodiment, prior knowledge is used to filter the output of the teacher model, enabling the teacher model to provide more accurate knowledge. At the same time, self-attention operation is performed on the feature values ​​output by the teacher model, so that the extracted feature values ​​have a global receptive field, thereby improving the detection performance of the detection model.

[0065] In one embodiment, alternatively, such as Figure 5 and Figure 6 As shown, based on the target prediction results and the initial prediction results corresponding to each student model, the second loss value for each student model in learning the teacher model's relevance to the target detection is determined, including:

[0066] S501. For each student model, based on the confidence level of the target prediction result and the confidence level of the initial prediction result, the target prediction result is processed to obtain the fusion prediction result of the teacher model for each student model.

[0067] To further enable the teacher model to provide more accurate black-box knowledge, the target prediction result can be processed in conjunction with the initial prediction results of the student model. Optionally, the confidence levels of the target prediction result and the initial prediction results (i.e., student candidate boxes) of the student model can be compared. If the confidence level of the target prediction result is greater than or equal to the confidence level of the initial prediction result of the student model, the target prediction result is retained. If the confidence level of the target prediction result is less than the confidence level of the initial prediction result of the student model, the target prediction result is replaced with the initial prediction result of the student model. In this way, although each student model corresponds to the same teacher model, after processing the target prediction result of the teacher model with the initial prediction results of each student model, each student model has a corresponding fused prediction result (i.e., fused teacher candidate boxes).

[0068] Furthermore, non-maximum suppression can be applied to the fusion prediction results (i.e., fusion teacher candidate boxes) to filter out duplicate objects in multiple fusion prediction results.

[0069] S502, Determine the first correlation between the fusion prediction results and the second correlation between the initial prediction results corresponding to each student model.

[0070] Specifically, the following Formula 2 can be used to calculate the first correlation between the fused prediction results of the teacher model (i.e., fused teacher candidate boxes) and the second correlation between the initial prediction results of each student model (i.e., student candidate boxes).

[0071] Formula 2:

[0072] Where Z represents the number of candidate boxes, x represents the features of the candidate boxes, and Ф(x) represents the correlation between candidate boxes.

[0073] Taking detection model 1 as the teacher model and detection models 2 to n as the student models as an example, each of detection models 2 to n has a corresponding fusion prediction result (i.e., fusion teacher candidate box). For detection model 2, the first correlation between the fusion prediction results (i.e., fusion teacher candidate boxes) of detection model 2 is calculated using the above formula 2. The second correlation between the initial prediction results (i.e., student candidate boxes) of detection model 2 is calculated using the above formula 2. Similarly, the first correlation between the fusion prediction results (i.e., fusion teacher candidate boxes) of detection model n is calculated using the above formula 2, and the second correlation between the initial prediction results (i.e., student candidate boxes) of detection model n is calculated using the above formula 2.

[0074] S503. Based on the first relevance and the second relevance, determine the second loss value of each student model in terms of learning the teacher model's relevance to the detection target.

[0075] Specifically, an L1smooth operation can be performed on the first relevance between the fused prediction results of the teacher model (i.e., fused teacher candidate boxes) and the second relevance between the initial prediction results of the student model (i.e., student candidate boxes), thereby obtaining the loss value of each student model in learning the relevance of the teacher model to the detected target. The loss value (LOSS) of each student model in learning the relevance of the teacher model to the detected target can be calculated using the following formula 3. cor .

[0076] Formula 3:

[0077] Where Ф(t) represents the first correlation between the fused teacher candidate boxes in the teacher model, Ф(s) represents the second correlation between the student candidate boxes in the student model, t represents the feature of the fused teacher candidate boxes, and s represents the feature of the student candidate boxes.

[0078] In this embodiment, prior knowledge can be used to filter the initial prediction results (i.e., candidate boxes) of the teacher model to obtain the target prediction results. The initial prediction results of the student model are then used to filter the target prediction results again to obtain fusion prediction results with higher confidence. The correlation between the fusion prediction results of the teacher model is used to guide the student model, making the detection effect of the student model more accurate.

[0079] In one embodiment, alternatively, such as Figure 7 and Figure 8 As shown, based on the target prediction results and the initial prediction results corresponding to each student model, the third loss value for each student model in learning the teacher model in target classification and regression is determined, including:

[0080] S701. For each student model, based on the confidence level of the target prediction result and the confidence level of the initial prediction result, the target prediction result is processed to obtain the fusion prediction result of the teacher model for each student model.

[0081] To further enable the teacher model to provide more accurate black-box knowledge, the target prediction result can be processed in conjunction with the initial prediction results of the student model. Optionally, the confidence levels of the target prediction result and the initial prediction results (i.e., student candidate boxes) of the student model can be compared. If the confidence level of the target prediction result is greater than or equal to the confidence level of the initial prediction result of the student model, the target prediction result is retained. If the confidence level of the target prediction result is less than the confidence level of the initial prediction result of the student model, the target prediction result is replaced with the initial prediction result of the student model. In this way, although each student model corresponds to the same teacher model, after processing the target prediction result of the teacher model with the initial prediction results of each student model, each student model has a corresponding fused prediction result (i.e., fused teacher candidate boxes).

[0082] Furthermore, non-maximum suppression can be applied to the fusion prediction results (i.e., fusion teacher candidate boxes) to filter out duplicate objects in multiple fusion prediction results.

[0083] S702. Use the fusion prediction results as pseudo-label data for the student models. Using the pseudo-label data and the initial prediction results corresponding to each student model, determine the third loss value for each student model in learning the teacher model in terms of target classification and regression.

[0084] Specifically, the loss value (Loss) of each student model in learning the teacher model in terms of regression and classification of the detection target can be calculated using the following formula 4. label_box .

[0085] Formula 4:

[0086] Where α and β represent hyperparameters, which can be set according to requirements. Optionally, they can be set to 0.1, L cls L represents the regression loss function. reg Represents the classification loss function. This indicates the category of the i-th pseudo-label data. Let represent the category of the i-th detected object in the student model. This represents the position information of the i-th pseudo-label data. This represents the location information of the i-th detected target in the student model, and z is the number of pseudo-label data.

[0087] In this embodiment, the initial prediction results of the student model (i.e., student candidate boxes) and the target prediction results of the teacher model (i.e., teacher candidate boxes filtered by prior knowledge) can be fused together. Candidate boxes with higher confidence are selected as pseudo-label data. At the same time, the pseudo-label data is filtered in combination with prior knowledge to make the pseudo-label data more accurate. The detection models are trained based on the more accurate pseudo-label data, thereby improving the detection performance of the detection models.

[0088] Figure 9 This is a schematic diagram of a target detection device provided in an embodiment of this application. Figure 9 As shown, the device may include an acquisition module 901 and a processing module 902.

[0089] Specifically, the acquisition module 901 is used to acquire the image to be detected;

[0090] The processing module 902 is used to input the image to be detected into a pre-trained target detection model to obtain the target recognition result of the image to be detected; wherein, the target detection model is obtained by sequentially setting one of a plurality of detection models as a teacher model and the remaining models as student models, and training the student model based on the teacher model.

[0091] Optionally, based on the above embodiments, the device further includes a training module.

[0092] Specifically, the training module is used to acquire multiple training images and multiple detection models to be trained before acquiring the images to be detected; to set one of the multiple detection models as the teacher model and the rest as student models; to use the teacher model and each student model to identify the target object in each training image, and to obtain the initial prediction results of the teacher model and each student model for each training image; to filter the initial prediction results corresponding to the teacher model according to the preset expected recognition results of each training image, and to obtain the target prediction results; to determine the first loss value of each student model in terms of bottleneck layer, second loss value in terms of target relevance, and third loss value in terms of target classification and regression, based on the target prediction results and the initial prediction results corresponding to each student model; and to set the next detection model in the multiple detection models as the teacher model, and the rest as student models. The process repeats the steps of using the teacher model and each student model to identify target objects in each training image until the last detection model among multiple detection models is set as the teacher model and the remaining models are set as student models. This yields a first loss set for each student model in terms of the bottleneck layer, a second loss set in terms of the relevance of the detected target, and a third loss set in terms of the classification and regression of the detected target. The first loss set, the second loss set, and the third loss set are processed respectively to obtain the corresponding first target loss value, second target loss value, and third target loss value. Based on the first target loss value, the second target loss value, the third target loss value, and the fourth target loss value, each detection model is trained to obtain the corresponding target detection model. The fourth target loss value is determined based on the expected recognition result and the initial prediction result corresponding to each student model.

[0093] Based on the above embodiments, optionally, the training module is specifically used to map the target prediction result to the bottleneck layer to obtain the target image feature vector output by the teacher model at the bottleneck layer; map each initial prediction result to the bottleneck layer to obtain the initial image feature vector output by each student model at the bottleneck layer; map the target image feature vector to each student model to obtain the mapped image feature vector of each student model; and determine the first loss value for each student model learning the teacher model at the bottleneck layer based on each initial image feature vector and the mapped image feature vector.

[0094] Based on the above embodiments, optionally, the training module is specifically used to perform self-attention operations on the target image feature vector and on the initial image feature vector.

[0095] Based on the above embodiments, optionally, the training module is specifically used to process the target prediction results for each student model based on the confidence level of the target prediction result and the confidence level of the initial prediction result, to obtain the fusion prediction result of the teacher model for each student model; determine the first correlation between the fusion prediction results and the second correlation between the initial prediction results corresponding to each student model; determine the second loss value of each student model learning the teacher model in terms of target detection relevance based on the first correlation and the second correlation; use the fusion prediction results as pseudo-label data of the student models, and use the pseudo-label data and the initial prediction results corresponding to each student model to determine the third loss value of each student model learning the teacher model in terms of target detection classification and regression.

[0096] Based on the above embodiments, optionally, the training module is specifically used to compare the confidence levels of the target prediction result and the corresponding initial prediction result for each student model; if the confidence level of the target prediction result is greater than or equal to the confidence level of the initial prediction result, the target prediction result is retained; if the confidence level of the target prediction result is less than the confidence level of the initial prediction result, the target prediction result is replaced with the initial prediction result.

[0097] Based on the above embodiments, optionally, the training module is specifically used to determine the maximum loss value in each first loss value set as the first target loss value for mutual learning among student models in terms of bottleneck layer; calculate the mean of each loss value in each second loss value set, and determine the calculation result as the second target loss value for mutual learning among student models in terms of relevance of detected targets; calculate the mean of each loss value in each second loss value set, and determine the calculation result as the third target loss value for mutual learning among student models in terms of regression and classification of detected targets.

[0098] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 10 As shown, the electronic device may include a processor 1000, a memory 1001, an input device 1002, and an output device 1003; the number of processors 1000 in the electronic device may be one or more. Figure 10 Taking a processor 1000 as an example; the processor 1000, memory 1001, input device 1002, and output device 1003 in the electronic device can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0099] The memory 1001, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the target detection method in the embodiments of this application (e.g., the acquisition module 901 and processing module 902 in the target detection device). The processor 1000 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 1001, thereby realizing the target detection method described above.

[0100] The memory 1001 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 1001 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 1001 may further include memory remotely located relative to the processor 1000, which can be connected to a device / terminal / server via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0101] The input device 1002 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the adaptive cruise controller. The output device 1003 may include a display device such as a display screen.

[0102] This application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a target detection method, the method comprising:

[0103] Acquire the image to be detected;

[0104] The image to be detected is input into a pre-trained target detection model to obtain the target recognition result of the image to be detected; wherein, the target detection model is obtained by sequentially setting one of a plurality of detection models as a teacher model and the remaining models as student models, and training the student model based on the teacher model.

[0105] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the method operations described above, but can also perform related operations in the target detection method provided in any embodiment of this application.

[0106] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this application can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0107] It is worth noting that in the embodiments of the search device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of this application.

[0108] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.

Claims

1. A target detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into a pre-trained target detection model to obtain the target recognition result of the image to be detected; wherein, the target detection model is obtained by sequentially setting one of a plurality of detection models as a teacher model and the remaining models as student models, and training the student model based on the teacher model; Before acquiring the image to be detected, the method further includes: Acquire multiple training images and multiple detection models to be trained; One of the multiple detection models is set as the teacher model, and the rest are set as student models. The teacher model and each student model are used to identify the target object in each training image to obtain the initial prediction results of the teacher model and each student model for each training image. The initial prediction results corresponding to the teacher model are filtered according to the expected recognition results of each training image to obtain the target prediction results; Based on the target prediction results and the initial prediction results corresponding to each student model, determine the first loss value of each student model in terms of learning the teacher model in terms of bottleneck layer, the second loss value in terms of target relevance, and the third loss value in terms of target classification and regression. The next detection model among multiple detection models is set as the teacher model, and the remaining models are set as student models. The step of using the teacher model and each student model to identify the target object in each training image is repeated until the last detection model among multiple detection models is set as the teacher model and the remaining models are set as student models. This results in the first loss set of each student model learning each teacher model in terms of bottleneck layer, the second loss set in terms of target relevance, and the third loss set in terms of target classification and regression. The first set of loss values, the second set of loss values, and the third set of loss values ​​are processed respectively to obtain the corresponding first target loss value, second target loss value, and third target loss value; Based on the first target loss value, the second target loss value, the third target loss value, and the fourth target loss value, each detection model is trained to obtain the corresponding target detection model; wherein, the fourth target loss value is determined based on the expected recognition result and the initial prediction result corresponding to each student model.

2. The method according to claim 1, characterized in that, The step of determining the first loss value of each student model in learning the teacher model at the bottleneck layer based on the target prediction result and the initial prediction result corresponding to each student model includes: The target prediction result is mapped to the bottleneck layer to obtain the target image feature vector output by the teacher model at the bottleneck layer. The initial prediction results are mapped to the bottleneck layer to obtain the initial image feature vectors output by each student model at the bottleneck layer. The target image feature vector is mapped to each student model to obtain the mapped image feature vector of each student model; Based on the initial image feature vectors and the mapped image feature vectors, determine the first loss value for each student model in learning the teacher model at the bottleneck layer.

3. The method according to claim 2, characterized in that, Also includes: A self-attention operation is performed on the target image feature vector, and a self-attention operation is performed on the initial image feature vector.

4. The method according to claim 1, characterized in that, The step of determining the second loss value for each student model in terms of target relevance and the third loss value in terms of target classification and regression, based on the target prediction results and the initial prediction results corresponding to each student model, includes: For each student model, based on the confidence level of the target prediction result and the confidence level of the initial prediction result, the target prediction result is processed to obtain the fusion prediction result of the teacher model for each student model; Determine a first correlation between the fused prediction results and a second correlation between the initial prediction results corresponding to each student model; based on the first correlation and the second correlation, determine a second loss value for each student model in learning the teacher model in terms of target detection relevance; The fusion prediction results are used as pseudo-label data for the student models. Using the pseudo-label data and the initial prediction results corresponding to each student model, the third loss value for each student model in learning the teacher model in terms of target classification and regression is determined.

5. The method according to claim 4, characterized in that, For each student model, based on the confidence level of the target prediction result and the confidence level of the initial prediction result, the target prediction result is processed to obtain the fusion prediction result of the teacher model for each student model, including: For each student model, compare the confidence levels of the target prediction results with the corresponding initial prediction results; If the confidence level of the target prediction result is greater than or equal to the confidence level of the initial prediction result, then the target prediction result is retained. If the confidence level of the target prediction result is lower than the confidence level of the initial prediction result, then the target prediction result is replaced with the initial prediction result.

6. The method according to claim 1, characterized in that, The step of processing the first loss value set, the second loss value set, and the third loss value set respectively to obtain the corresponding first target loss value, second target loss value, and third target loss value includes: The maximum loss value in each first loss value set is determined as the first target loss value for mutual learning among student models at the bottleneck layer. The mean of each loss value in each set of second loss values ​​is calculated, and the calculation result is determined as the second target loss value for mutual learning among student models in terms of the relevance of the detected target; The average value of each loss value in each second loss value set is calculated, and the calculation result is determined as the third target loss value for mutual learning between student models in terms of regression and classification of the detected target.

7. A target detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected; The processing module is used to input the image to be detected into a pre-trained target detection model to obtain the target recognition result of the image to be detected; wherein, the target detection model is obtained by sequentially setting one of a plurality of detection models as a teacher model and the remaining models as student models, and training the student model based on the teacher model; The training module is used to acquire multiple training images and multiple detection models to be trained before the acquisition module acquires the images to be detected; to set one detection model as the teacher model and the rest as student models; to use the teacher model and each student model to identify the target object in each training image, and to obtain the initial prediction results of the teacher model and each student model for each training image; to filter the initial prediction results corresponding to the teacher model according to the preset expected recognition results of each training image, and to obtain the target prediction results; to determine the first loss value of each student model in terms of bottleneck layer, second loss value in terms of target relevance, and third loss value in terms of target classification and regression, based on the target prediction results and the initial prediction results corresponding to each student model; and to set the next detection model in the multiple detection models as the teacher model, and the rest as student models. The process repeats the steps of using the teacher model and each student model to identify target objects in each training image until the last detection model among multiple detection models is set as the teacher model and the remaining models are set as student models. This yields a first loss set for each student model in terms of bottleneck layer, a second loss set in terms of target relevance, and a third loss set in terms of target classification and regression. The first loss set, the second loss set, and the third loss set are processed to obtain corresponding first target loss values, second target loss values, and third target loss values. Based on the first target loss values, the second target loss values, the third target loss values, and the fourth target loss values, each detection model is trained to obtain a corresponding target detection model. The fourth target loss value is determined based on the expected recognition result and the initial prediction result corresponding to each student model.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the processors perform the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images