Multi-source passive field adaptive target detection method and system based on deep learning

Abstract

The present application provides a multi-source passive domain adaptive target detection method and system based on deep learning, which relates to the field of target detection technology. The pre-trained target detection model is obtained by pre-training the basic detection model based on multiple source domain image data; the image data is enhanced and input into the teacher-student pair created for each source domain to obtain the position frame output by the teacher-student pair of each source domain; based on the position frame output by the teacher-student pair, a trained teacher-student pair model is obtained by using a position frame unification module, a multi-source probability fusion module and a memory-based comparative learning module. The trained teacher-student pair model is used to perform target detection on the image data to be processed in the target domain to obtain the target detection result. After training, the model reaches the optimal performance, which can effectively improve the performance of the target detection model under domain adaptation in the real world and the challenges of multi-source passive fields, and improve the generalization performance of the target detection model in the real world environment.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a multi-source passive field adaptive target detection method and system based on deep learning. Background Art

[0002] In the context of deep learning, object detection is defined as the location and classification of objects in media resources such as images, video frames, and live broadcasts. Its main goal is to detect visual instances belonging to predefined categories in multimedia resources. Classic deep learning object detection algorithms are roughly divided into two categories by researchers: two-stage and one-stage object detection.

[0003] Although these target detection models have achieved high results on some data sets and test benchmarks, when applied to real-world application scenarios, typical scenarios in the application scenarios include multi-source and passive domain adaptive target detection. Due to domain offset, that is, the feature distribution of the source domain and the target domain has certain differences, such as lighting, texture, shape, etc., the model pre-trained on the source domain data cannot adapt to the test data composed of target domain data, which will lead to performance degradation and efficiency reduction of the target detection model, and thus lead to its inability to achieve high detection accuracy in multi-source and passive domain scenarios, and its application is limited.

[0004] Based on this, there is an urgent need for an object detection method that can be applied to both passive domain adaptation and multi-source domain adaptation. Summary of the invention

[0005] The present application provides a multi-source passive domain adaptive target detection method and system based on deep learning to solve the problem that the existing domain adaptive target detection technology cannot be applied to passive domain adaptation and multi-source domain adaptation at the same time.

[0006] In a first aspect of an embodiment of the present application, a multi-source passive domain adaptive target detection method based on deep learning is proposed, the method comprising:

[0007] Obtaining pre-trained target detection models of multiple source domains by pre-training a basic detection model based on multiple source domain image data;

[0008] Based on the target domain image data, weak enhancement processing and strong enhancement processing are performed on the target domain image data to obtain weakly enhanced target domain image data and strongly enhanced target domain image data;

[0009] Based on the pre-trained object detection model of each source domain, a teacher-student pair is created for each source domain. A teacher-student pair consists of a teacher model and a student model.

[0010] Input the weakly enhanced target domain image data into the teacher model of each source domain to obtain the position frame output by the RPN detection head of the teacher model of each source domain, and input the strongly enhanced target domain image data into the student model of each source domain to obtain the position frame output by the RPN detection head of the student model of each source domain;

[0011] The position frames output by the RPN detection heads of the teacher models of different source domains are integrated through the position frame unification module to obtain an integrated first position frame, and the position frames output by the RPN detection heads of the student models of different source domains are integrated to obtain an integrated second position frame;

[0012] Obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module;

[0013] Through the memory-based comparative learning module, the feature quality filter is used to filter the feature space of the student model of each source domain and the high-quality instance-level features of each category, and store them in the memory of each source domain for comparative learning of student models of different source domains. The memory of each source domain is updated in a first-in-first-out manner.

[0014] The trained teacher model and the trained student model are used to perform target detection on the image data to be processed in the target domain, and the target detection result is obtained by combining the weights obtained through training.

[0015] In an optional embodiment of the present application, obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module includes:

[0016] The integrated first position frame is processed by the RoI Align component and the RCNN detection head of the teacher model of each source domain to obtain a first classification prediction probability and a first position frame prediction result, and the integrated second position frame is processed by the RoI Align component and the RCNN detection head of the student model of each source domain to obtain a second classification prediction probability and a second position frame prediction result;

[0017] The multi-source probability fusion module processes the first classification prediction probability and the first position box prediction result corresponding to the teacher models of different source domains to obtain the pseudo label.

[0018] In an optional embodiment of the present application, the multi-source probability fusion module processes the first classification prediction probability and the first position box prediction result corresponding to the teacher models of different source domains to obtain the pseudo label, including:

[0019] Screen all the first classification prediction probabilities and first position box prediction results, and obtain the first position box prediction result corresponding to the maximum probability in the teacher models of different source domains based on the first classification prediction probability as the current box;

[0020] Calculate the intersection-and-union ratio of the first position frame prediction result obtained by the teacher model of each source domain and the current frame, and take the first position frame prediction result whose intersection-and-union ratio is greater than a preset threshold as the overlapped position frame;

[0021] Calculating an intersection-and-union ratio between the overlapping position frame and the current frame to obtain a first position frame prediction result corresponding to a maximum intersection-and-union ratio;

[0022] Based on the first position box prediction results and first classification prediction probabilities corresponding to multiple maximum intersection-over-union ratios of the current box and different source domains, a unified position box prediction result and classification prediction probability are obtained;

[0023] The position box prediction result whose classification prediction probability is greater than the preset confidence is added to the prediction set, and the current box and the overlapping position box are deleted. The above screening process is repeated until there is no screening object, and the position box prediction result and classification prediction probability in the prediction set are output as the pseudo label.

[0024] In an optional embodiment of the present application, the method further includes:

[0025] Based on the second classification prediction probability and the second location box prediction result, calculating the RCNN weighted prediction result of the student model, wherein the weight of each source domain is continuously updated during the training process until the training is completed;

[0026] Replacing the integrated first position frame with the integrated second position frame, and after the replacement, processing the integrated second position frame through the RoI Align component and the RCNN detection head of the teacher model of each source domain to obtain a third classification prediction probability and a third position frame prediction result;

[0027] Based on the third classification prediction probability and the third position box prediction result, the RCNN weighted prediction result of the teacher model is calculated.

[0028] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model at least includes a contrast loss, and the contrast loss is obtained according to the following steps:

[0029] Through the feature quality filter, the feature space of the student model of each source domain and the high-quality instance-level features of each category are filtered and stored in the memory bank of each source domain;

[0030] Compute the contrastive loss based on the features in the memory banks of different source domains.

[0031] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model includes at least a consistency loss, and the consistency loss is obtained according to the following steps:

[0032] The KL divergence between the RCNN weighted prediction results of the student model of each source domain and the RCNN weighted prediction results of the teacher model is calculated to obtain the consistency loss.

[0033] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model includes at least information maximization loss, and the information maximization loss is obtained according to the following steps:

[0034] The information maximization loss is calculated using the RCNN weighted prediction results of the student model in each source domain.

[0035] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model at least includes a detection loss, and the detection loss is obtained according to the following steps:

[0036] Based on the pseudo-label of each source domain, the integrated second position box and its corresponding classification probability, the second classification prediction probability and the second position box prediction result, the detection loss is calculated, and the detection loss includes RPN classification loss, RPN regression loss, RCNN classification loss and RCNN regression loss.

[0037] In a second aspect of an embodiment of the present application, a multi-source passive domain adaptive target detection system based on deep learning is proposed, the system comprising:

[0038] A pre-training module, used to obtain pre-trained target detection models of multiple source domains by pre-training a basic detection model based on multiple source domain image data;

[0039] An image enhancement processing module is used to obtain weak enhancement processing and strong enhancement processing on the target domain image data based on the target domain image data to obtain weakly enhanced target domain image data and strongly enhanced target domain image data;

[0040] The teacher-student pair creation module is used to create a teacher-student pair for each source domain based on the pre-trained object detection model of each source domain. A teacher-student pair includes a teacher model and a student model.

[0041] A position frame acquisition module is used to input the weakly enhanced target domain image data into the teacher model of each source domain to obtain the position frame output by the RPN detection head of the teacher model of each source domain, and to input the strongly enhanced target domain image data into the student model of each source domain to obtain the position frame output by the RPN detection head of the student model of each source domain;

[0042] An integrated position frame acquisition module is used to integrate the position frames output by the RPN detection heads of the teacher models of different source domains through the position frame unification module to obtain an integrated first position frame, and to integrate the position frames output by the RPN detection heads of the student models of different source domains to obtain an integrated second position frame;

[0043] A pseudo label acquisition module, used for obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module;

[0044] A contrastive learning module is used to filter the feature space of the student model of each source domain and the high-quality instance-level features of each category through a memory-based contrastive learning module and a feature quality filter, and store them in the memory of each source domain for contrastive learning of student models of different source domains. The memory of each source domain is updated in a first-in-first-out manner.

[0045] The target detection module is used to use the trained teacher model and the trained student model to perform target detection on the image data to be processed in the target domain, and obtain the target detection result by combining the weights obtained through training.

[0046] In a third aspect of an embodiment of the present application, an electronic device is proposed, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the multi-source passive domain adaptive target detection method based on deep learning as described in any one of the first aspects above.

[0047] The present application includes the following advantages: The present application provides a multi-source passive domain adaptive target detection method and system based on deep learning, by obtaining pre-trained target detection models of multiple source domains by pre-training a basic detection model based on multiple source domain image data; obtaining target domain image data, performing weak enhancement processing and strong enhancement processing on the target domain image data, and obtaining weakly enhanced target domain image data and strongly enhanced target domain image data; based on the pre-trained target detection model of each source domain, creating a teacher-student pair for each source domain, a teacher-student pair including a teacher model and a student model; inputting the weakly enhanced target domain image data into the teacher model of each source domain, obtaining the position box output by the RPN detection head of the teacher model of each source domain, and inputting the strongly enhanced target domain image data into the student model of each source domain, obtaining the position box output by the RPN detection head of the student model of each source domain. The method comprises the steps of: placing a frame; integrating the position frames output by the RPN detection heads of the teacher models of different source domains through a position frame unification module to obtain an integrated first position frame, and integrating the position frames output by the RPN detection heads of the student models of different source domains to obtain an integrated second position frame; obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module; filtering the feature space of the student model of each source domain and the high-quality instance-level features of each category through a memory-based comparative learning module using a feature quality filter, and storing them in the memory of each source domain for comparative learning of the student models of different source domains, and updating the memory of each source domain in a first-in-first-out manner; performing target detection on the image data to be processed in the target domain using the trained teacher model and the trained student model, and obtaining the target detection result by combining the weights obtained through training. Through multi-source teacher-student pairs, position box unification module, multi-source probability fusion module, and memory-based comparative learning module, the model achieves optimal performance after training in multi-source domain offset environments simulating a variety of real-world environments. It can effectively improve the performance of the target detection model under domain adaptation in the real world and multi-source passive domain challenges, and improve the generalization performance of the target detection model in real-world environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 It is a flowchart of the steps of a multi-source passive field adaptive target detection method based on deep learning provided by an embodiment of the present application;

[0050] Figure 2 It is a flowchart of a multi-source passive field adaptive target detection method based on deep learning provided in an embodiment of the present application;

[0051] Figure 3 It is a schematic diagram of a position frame unification module of a multi-source passive field adaptive target detection method based on deep learning provided in an embodiment of the present application;

[0052] Figure 4 It is a structural schematic diagram of a multi-source probability fusion module of a multi-source passive field adaptive target detection method based on deep learning provided in an embodiment of the present application;

[0053] Figure 5 It is a structural schematic diagram of a contrastive learning module based on a memory bank of a multi-source passive field adaptive target detection method based on deep learning provided in an embodiment of the present application;

[0054] Figure 6 This is an architecture diagram of a multi-source passive domain adaptive target detection system based on deep learning provided by an embodiment of the present application;

[0055] Figure 7 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] In the context of deep learning, object detection is defined as the location and classification of objects in media resources such as images, video frames, and live broadcasts. Its main goal is to detect visual instances belonging to predefined categories in multimedia resources. Classic deep learning object detection algorithms are roughly divided into two categories by researchers: two-stage and one-stage object detection. Among them, one-stage object detection algorithms include FCOS based on the idea of ​​a fully convolutional network, YOLO based on grid cells, DETR based on an attention mechanism, and DiffusionDet based on a diffusion model. Two-stage object detection algorithms include Fast R-CNN and Faster R-CNN.

[0058] Although these target detection models have achieved high results on some data sets and test benchmarks, when applied to real-world application scenarios, typical scenarios in application scenarios include multi-source and passive domain adaptive target detection. Due to domain offset, that is, the feature distribution of the source domain and the target domain has certain differences, such as lighting, texture, shape, etc., the model pre-trained on the source domain data cannot adapt to the test data composed of target domain data. For example, the target detection model trained on virtual synthetic images often cannot achieve very good detection effects on images collected in the real world, because the lighting conditions, object shape, detection background and other conditions in the real world are often more complex, which will lead to the performance and efficiency of the target detection model. Reduced, which in turn leads to its inability to achieve high detection accuracy and limited application in multi-source and passive domain scenarios.

[0059] In multi-source domain adaptive object detection, the source domain data used for training is collected from two or more source domains. Since traditional domain adaptive object detection models focus on a single source domain, they still cannot play their due role when encountering domain shifts between different source domains and target domains, as well as target shifts between different source domains in multi-source scenarios. In passive domain adaptive object detection, due to limitations such as privacy protection policies and data transmission overhead, source domain data cannot be provided in the adaptive process, and only deep neural network models pre-trained based on source domain data are used instead. However, traditional domain adaptive object detection models often require rich feature distribution and supervision information in the source domain data to align with the target domain, so they are also unable to cope with passive application scenarios.

[0060] For the target detection task in the above-mentioned restricted scenarios, currently no method can cope with both multi-source and passive restricted scenarios at the same time. A multi-source passive domain adaptive target detection method is proposed, which applies the cutting-edge technology of deep learning to solve the problem of domain adaptive target detection in multi-source and passive application scenarios, improves the detection accuracy of the target detection model in multi-source passive domain adaptive scenarios, and enables the target detection model to still perform efficient target detection under the constraints of domain offset and passivity in a variety of complex real-world environments.

[0061] In a first aspect of the embodiments of the present application, a multi-source passive domain adaptive target detection method based on deep learning is disclosed, referring to Figure 1 , Figure 1 A flowchart of a multi-source passive domain adaptive target detection method based on deep learning is provided in an embodiment of the present application, and the method comprises the following steps:

[0062] Step 101: obtaining pre-trained target detection models of multiple source domains by pre-training a basic detection model based on multiple source domain image data;

[0063] Step 102: obtaining target domain image data, performing weak enhancement processing and strong enhancement processing on the target domain image data, and obtaining weakly enhanced target domain image data and strongly enhanced target domain image data;

[0064] Step 103: Based on the pre-trained object detection model of each source domain, create a teacher-student pair for each source domain, where a teacher-student pair includes a teacher model and a student model;

[0065] Step 104: inputting the weakly enhanced target domain image data into the teacher model of each source domain to obtain the position frame output by the RPN detection head of the teacher model of each source domain, and inputting the strongly enhanced target domain image data into the student model of each source domain to obtain the position frame output by the RPN detection head of the student model of each source domain;

[0066] Step 105: integrating the position frames output by the RPN detection heads of the teacher models of different source domains through a position frame unification module to obtain an integrated first position frame, and integrating the position frames output by the RPN detection heads of the student models of different source domains to obtain an integrated second position frame;

[0067] Step 106: obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module;

[0068] Step 107: Using a memory-based comparative learning module and a feature quality filter, the feature space of the student model of each source domain and the high-quality instance-level features of each category are screened and stored in the memory of each source domain for comparative learning of student models of different source domains. The memory of each source domain is updated in a first-in-first-out manner.

[0069] Step 108: Use the trained teacher model and the trained student model to perform target detection on the image data to be processed in the target domain, and combine the weights obtained through training to obtain the target detection result.

[0070] In order to clearly illustrate the multi-source passive domain adaptive target detection method based on deep learning disclosed in the embodiment of the present application, combined with Figure 2 Provide clear instructions, Figure 2 It is a flow chart of a multi-source passive field adaptive target detection method based on deep learning provided in an embodiment of the present application.

[0071] When step 101 is specifically implemented, in transfer learning and domain adaptation, the source domain refers to a data set or data distribution space in which the model is pre-trained or has been learned. For image data, the source domain is a data set containing a large number of images and their corresponding labels (if labels exist). These data are usually used to train the model so that it can learn the features, patterns or regularities in the image. Passive domain adaptation can achieve domain adaptation using only pre-trained source domain models and target domain data without accessing the source domain data. Therefore, before performing multi-source passive domain adaptive target detection, multiple source domain image data are obtained, the basic detection model is pre-trained to obtain pre-trained target detection models for multiple source domains, and the pre-trained target detection model is used for subsequent detection, wherein the basic detection model can be Faster R-CNN.

[0072] When step 102 is specifically implemented, any source domain is selected from multiple source domains as the target domain, and based on the target domain image data, weak enhancement processing and strong enhancement processing are performed on the target domain image data to obtain weakly enhanced target domain image data and strongly enhanced target domain image data. The properties of the image data are changed by weak enhancement processing and strong enhancement processing, and are used to train the pre-trained model to improve the generalization ability and robustness of the model. Among them, weak enhancement processing is a small or mild transformation of the image data. These transformations will not significantly change the overall content or structure of the image data, and can increase the diversity of the image data, including: brightness adjustment, contrast adjustment, color jitter, slight rotation, slight scaling, etc. Strong enhancement processing is a large or drastic transformation of the image data. These transformations may significantly change the overall content or structure of the image data, but help the model learn more robust features, including: large rotation, random cropping, flipping, noise addition, blurring, color transformation, etc.

[0073] When implementing step 103, due to the need to hide the source domain data and the unsupervised target domain data, the traditional method of using pseudo labels instead of supervisory information may introduce too much unnecessary noise, which will eventually cause the model training direction to shift toward unnecessary update directions, seriously affecting the final model performance. Therefore, combined with the characteristics of multiple source domains, based on the pre-trained target detection model of each source domain, a teacher-student pair is created for each source domain, and a teacher-student pair includes a teacher model and a student model; under the multi-source teacher-student pair, the teacher-student pairs belonging to each source domain can effectively learn the feature expression of the target domain in their respective feature spaces.

[0074] When step 104 is specifically implemented, the weakly enhanced target domain image data is input into the teacher model of each source domain to obtain the position box output by the RPN detection head of the teacher model of each source domain, and the strongly enhanced target domain image data is input into the student model of each source domain to obtain the position box output by the RPN detection head of the student model of each source domain.

[0075] When implementing step 105, under multi-source teacher-student pairs, due to cross-model diversity, different models have different location box predictions for the same image, and it is impossible to directly weight the classification predictions under different location boxes. We further propose a location box unification module. For each target domain image, we integrate the region proposals with the highest confidence predicted by different source models based on different source domain knowledge, and mine the region proposals that are most likely to be close to the true target distribution, and then directly perform weighted predictions on the regions in the same location box on the integrated regions. The above-mentioned location box unification module corresponds to Figure 2 For more information, see the Unip module in Figure 3 , Figure 3 It is a schematic diagram of a position frame unification module of a multi-source passive field adaptive target detection method based on deep learning provided by an embodiment of the present application. Specifically, the position frames output by the RPN detection heads of the teacher models of different source domains are integrated through the position frame unification module to obtain an integrated first position frame, and the position frames output by the RPN detection heads of the student models of different source domains are integrated to obtain an integrated second position frame. When integrating the position frames, based on the confidence, non-maximum suppression is used to screen all the position frames output by the RPN detection heads in different target detection models, and the average intersection-union ratio of the screened position frames is calculated, the processed position frames are integrated, and based on the average intersection-union ratio, non-maximum suppression is performed on them, and the processed position frames are output. The position frame unification module not only solves the challenge of cross-model diversity, but also digs out some difficult image data that are only noticed by a few source models as much as possible and introduces them into training, and finally learns richer target domain knowledge.

[0076] When step 106 is specifically implemented, a multi-source probability fusion module (corresponding to Figure 2 The MSPE module in the above example obtains high-quality pseudo labels based on the integrated first location box output by the teacher model. The multi-source probability fusion module gradually selects high-quality location boxes from the set of location box predictions of multiple teachers while eliminating redundant location boxes, and fuses the knowledge of multiple source domains in the probability distribution, so as to achieve the effect of promoting consistent prediction and mining high-quality pseudo labels.

[0077] In an optional embodiment of the present application, a pseudo label is obtained based on the integrated first position box through a multi-source probability fusion module, which specifically includes the following processes: first, the integrated first position box is processed through the RoI Align component and the RCNN detection head of the teacher model of each source domain to obtain the first classification prediction probability and the first position box prediction result, and the integrated second position box is processed through the RoI Align component and the RCNN detection head of the student model of each source domain to obtain the second classification prediction probability and the second position box prediction result; then, the first classification prediction probability and the first position box prediction result corresponding to the teacher models of different source domains are processed through the multi-source probability fusion module to obtain the above-mentioned pseudo label.

[0078] In an optional embodiment of the present application, see Figure 4 , Figure 4 This is a schematic diagram of the structure of a multi-source probability fusion module of a multi-source passive domain adaptive target detection method based on deep learning provided by an embodiment of the present application. Through the multi-source probability fusion module, the first classification prediction probabilities and first position box prediction results corresponding to the teacher models of different source domains are processed to obtain the pseudo label, which specifically includes the following processes: all the first classification prediction probabilities and first position box prediction results are screened, and for all teacher models, the RoI Align component is used to obtain the region of interest features of the integrated first position box (in Figure 4In the example, the borders inside different image data are the first position box results), and the features of the region of interest are input into the RCNN detection head to perform classification prediction of specific categories and regression prediction of the position box. The classification prediction takes a logical number, and the softmax activation function is used to calculate the corresponding classification prediction probability. Then, the integrated first position box is fine-tuned according to the results of the regression prediction. After completion, the first classification prediction probability and its first position box prediction result are input into the multi-source probability fusion module to obtain the first position box prediction result corresponding to the maximum probability in the teacher model of different source domains as the current box; for each source domain first position box prediction result, its intersection-and-union ratio with the current box is calculated, and the first position box prediction result with an intersection-and-union ratio greater than a preset threshold is used as the overlapped position box, and the preset threshold can be set to 0.7; the intersection-and-union ratio of the above-mentioned overlapped position box and the above-mentioned current box is calculated to obtain the first position box prediction result corresponding to the maximum intersection-and-union ratio, and the remaining position box prediction results are used as the overlapped position box. The result is deleted; based on the first position box prediction results and the first classification prediction probabilities corresponding to the multiple maximum intersection-over-union ratios of the current box and different source domains, the first classification prediction probabilities are summed and averaged, and the coordinate values ​​of the first position box prediction results are summed and averaged to obtain a unified position box prediction result and classification prediction probability; the position box prediction results whose classification prediction probabilities are greater than the preset confidence are added to the prediction set, and the current box and the overlapping position boxes are deleted, and the above screening process is repeated until there are no screening objects, and the position box prediction results and classification prediction probabilities in the prediction set are output as pseudo labels, wherein the preset confidence can be set to 0.4.

[0079] In an optional embodiment of the present application, after the student model and the teacher model obtain the integrated position frame respectively, the model will learn a set of weights representing the contribution of each source domain to the domain adaptation according to the domain differences from each source domain to the target domain, so as to weight the predictions of different source domains according to their importance and obtain the target detection result with the highest accuracy. Specifically including: based on the above-mentioned second classification prediction probability and the above-mentioned second position frame prediction result, the RCNN weighted prediction result of the student model is calculated, wherein the weight of each source domain is continuously updated during the training process until the training is completed; the above-mentioned integrated second position frame is used to replace the integrated first position frame, and after the replacement, the above-mentioned integrated second position frame is processed through the RoI Align component and RCNN detection head of the teacher model of each source domain to obtain the third classification prediction probability and the third position frame prediction result; based on the above-mentioned third classification prediction probability and the above-mentioned third position frame prediction result, the RCNN weighted prediction result of the teacher model is calculated.

[0080] When implementing step 107, in order to further improve the detection effect and model performance, a memory-based contrast learning module is used to further learn the features of the target level with high discrimination. Figure 5 , Figure 5 Schematic diagram of the structure of a memory-based contrast learning module of a multi-source passive domain adaptive target detection method based on deep learning provided by the embodiment of the present application. Figure 2 The MBCL module in the source domain uses feature quality filters to filter the feature space of the student model of each source domain and the high-quality instance-level features of each category, and stores them in the memory bank of each source domain for comparative learning by student models of different source domains. The memory bank of each source domain is updated in a first-in-first-out manner.

[0081] Specifically, in a multi-source passive environment, there is no supervision information. The classification prediction based on the pseudo-label information may have a lot of noise in the early stage of training, which will cause the parameter update direction of the model to shift. Therefore, evaluation indicators such as prediction entropy and prediction consistency are used to screen out high-quality category features in each batch to construct contrast loss and improve the quality of contrastive learning. For features of different categories (such as category a and category b), feature quality filters are used in combination with evaluation indicators: category probability, prediction entropy and prediction consensus to filter image data of different categories. Category probability means that the classification prediction probability of the current category is not less than 0.7, prediction entropy means that the classification prediction entropy of the current region of interest is not less than the average of all regions of interest in the current batch, and prediction consensus means that the predictions of all student models are consistent with the weighted predictions, and the weighted predictions are consistent with the pseudo-labels. The filtered different features are stored in the corresponding memory bank. Since the teacher-student model is in a gradual learning process, and there may be some noisy updates under the condition of multi-source passive field adaptation, the memory bank is used to save some features of the previous iteration, and the first-in-first-out update method is adopted to store new features filtered by the filter, making the contrastive learning process more stable. In addition, target learning is performed separately between the feature space of each source domain and the corresponding teacher-student pair, ensuring that multiple average teacher networks can stably improve feature discrimination while avoiding knowledge confusion in different feature spaces.

[0082] When implementing step 108, by introducing the parameterized source domain weights into the process of calculating the final detection result, the model will learn a set of weights representing the contribution of each source domain to the domain adaptation according to the domain differences from each source domain to the target domain, thereby weighting the predictions of different source domains according to their importance to obtain the target detection result with the highest accuracy. Specifically, after the training of the teacher-student model is completed, the trained teacher model and the trained student model are used to perform target detection on the image data to be processed in the target domain, and the target detection result is obtained by combining the weights obtained through training.

[0083] In the initial multi-source domain student model and teacher model, three losses including detection loss, information maximization loss and consistency regularization term are introduced to constrain the learning process of the model to obtain a trained teacher model and a trained student model, and the trained teacher model and the trained student model are used for target detection on the image data to be processed.

[0084] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model includes at least detection loss, which is obtained according to the following steps: based on the pseudo-label of each source domain, the integrated second position box and its corresponding classification probability, the second classification prediction probability and the second position box prediction result, the detection loss is calculated. The RCNN classification loss is weighted according to a set of parameterized weights, and the other two losses are calculated separately in each teacher-student pair. Finally, all losses are integrated to efficiently learn the knowledge of classification and regression, and correctly update the source domain weights. For classification predictions, a set of parameterized weights are used to weight them, and then the detection loss is constructed according to the weighted classification predictions. The detection loss includes RPN classification loss, RPN regression loss, RCNN classification loss and RCNN regression loss. The formula for the above detection loss is as follows:

[0085]

[0086] Where, L det represents the detection loss, They represent RPN network classification loss, RPN network regression loss, RCNN network classification loss, and RCNN network regression loss respectively; represents the i-th student model, Represents the teacher model for the target domain image data X T Generated pseudo-labels; α i represents the weight of the i-th source domain.

[0087] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model includes at least contrast loss, which is obtained according to the following steps: through a feature quality filter, the feature space of the student model of each source domain and the high-quality instance-level features of each category are filtered, and the filtered category features are stored in the memory library of the corresponding category of each source domain; the contrast loss is calculated based on the features stored in the memory libraries of different source domains. The features in the memory library of the current category of the current feature space are regarded as positive keys, and the remaining features are regarded as negative keys, and the InfoNCE loss (Noise Contrastive Estimation of Mutual Information Loss) is calculated. After the contrast loss of each feature space and each category is calculated, it is averaged according to the number of feature spaces and the number of categories. Contrast loss The formula is as follows:

[0088]

[0089] in,

[0090] In the formula, m and k represent the number of source domains and the number of categories respectively. represents the feature memory of the jth category of the i-th source domain, represents the features of the jth class predicted by the i-th student model in the current batch, C i represents the number of all categories predicted in the current batch, τ represents the temperature coefficient, which is used to control the model's discrimination of difficult negative samples, q represents the query, that is, one of the current category features of the current source domain student model in the iteration of the current model, and K + represents a positive key, i.e., one of the memory features in the same category of the same source domain as q; K represents a key, i.e., one of all memory features in all categories of all source domains in the current iteration.

[0091] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model includes at least a consistency loss, which is obtained by following the steps: calculating the KL divergence of the RCNN weighted prediction results of the student model of each source domain and the RCNN weighted prediction results of the teacher model, using the calculated KL divergence as the consistency loss, and using the consistency loss (i.e., the calculated KL divergence) to ensure the consistency of the update between the weighted teacher and the weighted student, that is, using the consistency loss to constrain the consistency between the weighted prediction of the teacher model and the weighted prediction of the student model. KL divergence is commonly used to measure the similarity of two distributions. The more similar the two distributions are, the closer the KL divergence is to 0, and vice versa. The two distributions of the KL divergence used to construct the consistency loss come from the weighted student classification prediction and the weighted teacher classification prediction, respectively. For the weighted teacher classification prediction, the unified position box calculated by the position box unification module of the student model is used to replace all the position boxes of the teacher model, and the weighted prediction of the teacher model after replacing the position box is calculated by forward propagation and activation function according to the parameterized weights, the parameters of the teacher model and the replaced position box.

[0092] In an optional embodiment of the present application, the loss used by the trained teacher model and the trained student model includes at least an information maximization loss, which is obtained by following the steps of: using the RCNN weighted prediction results of the student model of each source domain to calculate the information maximization loss. For the information maximization loss, the weighted classification prediction is used to calculate the information maximization loss, which is used to minimize the conditional entropy and maximize the empirical label distribution entropy. The information maximization loss The formula is as follows:

[0093]

[0094] In the formula, m represents the number of source domains, H represents the entropy function, α represents the source domain weight, and α p represents the source domain weight, θ represents the source domain classifier model, represents the student model of the p-th source domain, σ j represents the activation function of the jth category, N T represents the number of images in the target domain dataset, Represents the target domain image data of the i-th source domain.

[0095] According to the average teacher model algorithm, after all forward propagation and loss calculations are completed, the parameters and weight parameters of all student models are updated by stochastic gradient descent, and the teacher model is updated according to the exponential moving average of the parameter values ​​of the respective paired student models. During testing, in order to better utilize the learned source domain weights and to use a more updated and stable model, the classification predictions of the final teacher model are weighted using weights, where the integrated position boxes of all teacher models are also calculated by the position box unified module algorithm.

[0096] The embodiment of the present application provides a multi-source passive domain adaptive target detection method based on deep learning, which comprises the following steps: obtaining pre-trained target detection models for multiple source domains by pre-training a basic detection model based on multiple source domain image data; obtaining target domain image data, performing weak enhancement processing and strong enhancement processing on the target domain image data, and obtaining weakly enhanced target domain image data and strongly enhanced target domain image data; creating a teacher-student pair for each source domain based on the pre-trained target detection model of each source domain, wherein a teacher-student pair includes a teacher model and a student model; inputting the weakly enhanced target domain image data into the teacher model of each source domain, and obtaining a position box output by the RPN detection head of the teacher model of each source domain, and inputting the strongly enhanced target domain image data into the student model of each source domain, and obtaining a position box output by the RPN detection head of the student model of each source domain; by The location frame unification module integrates the location frames output by the RPN detection heads of the teacher models of different source domains to obtain an integrated first location frame, and integrates the location frames output by the RPN detection heads of the student models of different source domains to obtain an integrated second location frame; a pseudo label is obtained based on the integrated first location frame through a multi-source probability fusion module; a memory-based comparative learning module uses a feature quality filter to screen the feature space of the student model of each source domain and the high-quality instance-level features of each category, and stores them in the memory of each source domain for comparative learning of student models of different source domains, and the memory of each source domain is updated in a first-in-first-out manner; the trained teacher model and the trained student model are used to perform target detection on the image data to be processed in the target domain, and the target detection result is obtained by combining the weights obtained through training. Through multi-source teacher-student pairs, position box unification module, multi-source probability fusion module, and memory-based comparative learning module, the model achieves optimal performance after training in multi-source domain offset environments simulating a variety of real-world environments. It can effectively improve the performance of the target detection model under domain adaptation in the real world and multi-source passive domain challenges, and improve the generalization performance of the target detection model in real-world environments.

[0097] In a second aspect of the embodiment of the present application, a multi-source passive domain adaptive target detection system based on deep learning is disclosed, see Figure 6 , Figure 6 : is an architecture diagram of a multi-source passive domain adaptive target detection system based on deep learning provided by an embodiment of the present application, the system comprising:

[0098] A pre-training module 601 is used to obtain pre-trained target detection models of multiple source domains by pre-training a basic detection model based on multiple source domain image data;

[0099] An image enhancement processing module 602 is used to obtain target domain image data, perform weak enhancement processing and strong enhancement processing on the target domain image data, and obtain weakly enhanced target domain image data and strongly enhanced target domain image data;

[0100] A teacher-student pair creation module 603 is used to create a teacher-student pair for each source domain based on the pre-trained object detection model of each source domain, where a teacher-student pair includes a teacher model and a student model;

[0101] The position frame acquisition module 604 is used to input the weakly enhanced target domain image data into the teacher model of each source domain to obtain the position frame output by the RPN detection head of the teacher model of each source domain, and input the strongly enhanced target domain image data into the student model of each source domain to obtain the position frame output by the RPN detection head of the student model of each source domain;

[0102] The integrated position frame acquisition module 605 is used to integrate the position frames output by the RPN detection heads of the teacher models of different source domains through the position frame unification module to obtain an integrated first position frame, and to integrate the position frames output by the RPN detection heads of the student models of different source domains to obtain an integrated second position frame;

[0103] A pseudo label acquisition module 606, configured to obtain a pseudo label based on the integrated first position frame through a multi-source probability fusion module;

[0104] A contrastive learning module 607 is used to filter the feature space of the student model of each source domain and the high-quality instance-level features of each category through a memory-based contrastive learning module and a feature quality filter, and store them in the memory of each source domain for contrastive learning of student models of different source domains. The memory of each source domain is updated in a first-in-first-out manner.

[0105] The target detection module 608 is used to use the trained teacher model and the trained student model to perform target detection on the image data to be processed in the target domain, and obtain the target detection result by combining the weights obtained through training.

[0106] Wherein, the pseudo label acquisition module includes:

[0107] A second classification result acquisition submodule is used to process the integrated first position frame through the RoI Align component and the RCNN detection head of the teacher model of each source domain to obtain a first classification prediction probability and a first position frame prediction result, and to process the integrated second position frame through the RoI Align component and the RCNN detection head of the student model of each source domain to obtain a second classification prediction probability and a second position frame prediction result;

[0108] The pseudo label acquisition submodule is used to process the first classification prediction probability and the first position box prediction result corresponding to the teacher models of different source domains through the multi-source probability fusion module to obtain the pseudo label.

[0109] Wherein, the pseudo label acquisition submodule includes:

[0110] A current frame acquisition unit is used to screen all first classification prediction probabilities and first position frame prediction results, and obtain the first position frame prediction result corresponding to the maximum probability in the teacher models of different source domains as the current frame based on the first classification prediction probability;

[0111] The overlapped position frame acquisition unit is used to calculate the intersection-and-union ratio of the first position frame prediction result obtained by the teacher model of each source domain and the current frame, and take the first position frame prediction result whose intersection-and-union ratio is greater than a preset threshold as the overlapped position frame;

[0112] A maximum intersection-and-union ratio calculation unit, used to calculate the intersection-and-union ratio of the overlapped position frame and the current frame, and obtain a first position frame prediction result corresponding to the maximum intersection-and-union ratio;

[0113] A unified result acquisition unit, configured to obtain a unified position box prediction result and classification prediction probability based on the first position box prediction results and first classification prediction probabilities corresponding to multiple maximum intersection-over-union ratios of the current box and different source domains;

[0114] A repeating unit is used to add the position box prediction result whose classification prediction probability is greater than a preset confidence to the prediction set, and delete the current box and the overlapping position box, repeat the above screening process until there is no screening object, and output the position box prediction result and classification prediction probability in the prediction set as the pseudo label.

[0115] Wherein, the system further comprises:

[0116] A first weighted prediction result acquisition module, used to calculate the RCNN weighted prediction result of the student model based on the second classification prediction probability and the second location box prediction result, wherein the weight of each source domain is continuously updated during the training process until the training is completed;

[0117] A third classification result acquisition module is used to replace the integrated first position frame with the integrated second position frame, and after the replacement, the integrated second position frame is processed by the RoI Align component and the RCNN detection head of the teacher model of each source domain to obtain a third classification prediction probability and a third position frame prediction result;

[0118] The second weighted prediction result acquisition module is used to calculate the RCNN weighted prediction result of the teacher model based on the third classification prediction probability and the third position box prediction result.

[0119] The losses used by the trained teacher model and the trained student model include at least contrast loss, and the system further includes a contrast loss acquisition module, which includes:

[0120] A filtering and storage submodule is used to filter the feature space of the student model of each source domain and the high-quality instance-level features of each category through a feature quality filter, and store them in a memory bank of each source domain;

[0121] The contrast loss calculation submodule is used to calculate the contrast loss based on the features in the memory banks of different source domains.

[0122] Among them, the losses used by the trained teacher model and the trained student model include at least consistency loss, and the system also includes a consistency loss acquisition module; the consistency loss module is used to calculate the KL divergence of the RCNN weighted prediction results of the student model of each source domain and the RCNN weighted prediction results of the teacher model to obtain the consistency loss.

[0123] Among them, the losses used by the trained teacher model and the trained student model include at least information maximization loss, and the system also includes an information maximization loss acquisition module, which is used to use the RCNN weighted prediction results of the student model of each source domain to calculate the information maximization loss.

[0124] Among them, the losses used by the trained teacher model and the trained student model include at least detection loss. The system also includes a detection loss acquisition module, which is used to calculate the detection loss based on the pseudo-label of each source domain, the integrated second position box and its corresponding classification probability, the second classification prediction probability and the second position box prediction result. The detection loss includes RPN classification loss, RPN regression loss, RCNN classification loss and RCNN regression loss.

[0125] Based on the same application concept, the embodiment of the present application discloses an electronic device in a third aspect. Figure 7 A schematic diagram of an electronic device disclosed in an embodiment of the present application is shown. Figure 7 As shown, the electronic device 100 includes: a memory 110 and a processor 120, the memory of the electronic device is not less than 12G, the main frequency of the processor is not less than 2.4GHz, the memory 110 and the processor 120 are connected through a bus communication, and a computer program is stored in the memory 110. The computer program can be run on the processor 120 to implement a multi-source passive field adaptive target detection method based on deep learning disclosed in an embodiment of the present application.

[0126] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0127] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, systems, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0130] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0131] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0132] The above is a detailed introduction to a multi-source passive field adaptive target detection method and system based on deep learning provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A multi-source passive domain adaptive target detection method based on deep learning, characterized in that: The method comprises: Obtaining pre-trained target detection models of multiple source domains by pre-training a basic detection model based on multiple source domain image data; Based on the target domain image data, weak enhancement processing and strong enhancement processing are performed on the target domain image data to obtain weakly enhanced target domain image data and strongly enhanced target domain image data; Based on the pre-trained object detection model of each source domain, a teacher-student pair is created for each source domain. A teacher-student pair consists of a teacher model and a student model. Input the weakly enhanced target domain image data into the teacher model of each source domain to obtain the position frame output by the RPN detection head of the teacher model of each source domain, and input the strongly enhanced target domain image data into the student model of each source domain to obtain the position frame output by the RPN detection head of the student model of each source domain; The position frames output by the RPN detection heads of the teacher models of different source domains are integrated through the position frame unification module to obtain an integrated first position frame, and the position frames output by the RPN detection heads of the student models of different source domains are integrated to obtain an integrated second position frame; Obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module; Through the memory-based comparative learning module, the feature quality filter is used to filter the feature space of the student model of each source domain and the high-quality instance-level features of each category, and store them in the memory of each source domain for comparative learning of student models of different source domains. The memory of each source domain is updated in a first-in-first-out manner. The trained teacher model and the trained student model are used to perform target detection on the image data to be processed in the target domain, and the target detection result is obtained by combining the weights obtained through training.

2. The multi-source passive domain adaptive target detection method based on deep learning according to claim 1 is characterized in that: The method of obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module includes: The integrated first position frame is processed by the RoIAlign component and the RCNN detection head of the teacher model of each source domain to obtain a first classification prediction probability and a first position frame prediction result, and the integrated second position frame is processed by the RoI Align component and the RCNN detection head of the student model of each source domain to obtain a second classification prediction probability and a second position frame prediction result; The multi-source probability fusion module processes the first classification prediction probability and the first position box prediction result corresponding to the teacher models of different source domains to obtain the pseudo label.

3. The multi-source passive domain adaptive target detection method based on deep learning according to claim 2 is characterized in that: The multi-source probability fusion module processes the first classification prediction probability and the first position box prediction result corresponding to the teacher models of different source domains to obtain the pseudo label, including: Screen all the first classification prediction probabilities and first position box prediction results, and obtain the first position box prediction result corresponding to the maximum probability in the teacher models of different source domains based on the first classification prediction probability as the current box; Calculate the intersection-and-union ratio of the first position frame prediction result obtained by the teacher model of each source domain and the current frame, and take the first position frame prediction result whose intersection-and-union ratio is greater than a preset threshold as the overlapped position frame; Calculating an intersection-and-union ratio between the overlapping position frame and the current frame to obtain a first position frame prediction result corresponding to a maximum intersection-and-union ratio; Based on the first position box prediction results and first classification prediction probabilities corresponding to multiple maximum intersection-over-union ratios of the current box and different source domains, a unified position box prediction result and classification prediction probability are obtained; The position box prediction result whose classification prediction probability is greater than the preset confidence is added to the prediction set, and the current box and the overlapping position box are deleted. The above screening process is repeated until there is no screening object, and the position box prediction result and classification prediction probability in the prediction set are output as the pseudo label.

4. The multi-source passive domain adaptive target detection method based on deep learning according to claim 2 is characterized in that: The method further comprises: Based on the second classification prediction probability and the second location box prediction result, calculating the RCNN weighted prediction result of the student model, wherein the weight of each source domain is continuously updated during the training process until the training is completed; Replacing the integrated first position frame with the integrated second position frame, and after the replacement, processing the integrated second position frame through the RoI Align component and the RCNN detection head of the teacher model of each source domain to obtain a third classification prediction probability and a third position frame prediction result; Based on the third classification prediction probability and the third position box prediction result, the RCNN weighted prediction result of the teacher model is calculated.

5. The multi-source passive domain adaptive target detection method based on deep learning according to claim 1 is characterized in that: The losses used by the trained teacher model and the trained student model include at least a contrast loss, which is obtained by following the steps below: Through the feature quality filter, the feature space of the student model of each source domain and the high-quality instance-level features of each category are filtered and stored in the memory bank of each source domain; Compute the contrastive loss based on the features in the memory banks of different source domains.

6. The multi-source passive domain adaptive target detection method based on deep learning according to claim 1 is characterized in that: The losses used by the trained teacher model and the trained student model include at least a consistency loss, which is obtained by following the steps below: The KL divergence between the RCNN weighted prediction results of the student model of each source domain and the RCNN weighted prediction results of the teacher model is calculated to obtain the consistency loss.

7. The multi-source passive domain adaptive target detection method based on deep learning according to claim 1 is characterized in that: The losses used by the trained teacher model and the trained student model include at least an information maximization loss, which is obtained according to the following steps: The information maximization loss is calculated using the RCNN weighted prediction results of the student model in each source domain.

8. The multi-source passive domain adaptive target detection method based on deep learning according to claim 2 is characterized in that: The losses used by the trained teacher model and the trained student model include at least a detection loss, which is obtained by following the steps below: Based on the pseudo-label of each source domain, the integrated second position box and its corresponding classification probability, the second classification prediction probability and the second position box prediction result, the detection loss is calculated, and the detection loss includes RPN classification loss, RPN regression loss, RCNN classification loss and RCNN regression loss.

9. A multi-source passive domain adaptive target detection system based on deep learning, characterized in that: The system comprises: A pre-training module, used to obtain pre-trained target detection models of multiple source domains by pre-training a basic detection model based on multiple source domain image data; An image enhancement processing module is used to obtain weak enhancement processing and strong enhancement processing on the target domain image data based on the target domain image data to obtain weakly enhanced target domain image data and strongly enhanced target domain image data; The teacher-student pair creation module is used to create a teacher-student pair for each source domain based on the pre-trained object detection model of each source domain. A teacher-student pair includes a teacher model and a student model. A position frame acquisition module is used to input the weakly enhanced target domain image data into the teacher model of each source domain to obtain the position frame output by the RPN detection head of the teacher model of each source domain, and to input the strongly enhanced target domain image data into the student model of each source domain to obtain the position frame output by the RPN detection head of the student model of each source domain; An integrated position frame acquisition module is used to integrate the position frames output by the RPN detection heads of the teacher models of different source domains through the position frame unification module to obtain an integrated first position frame, and to integrate the position frames output by the RPN detection heads of the student models of different source domains to obtain an integrated second position frame; A pseudo label acquisition module, used for obtaining a pseudo label based on the integrated first position frame through a multi-source probability fusion module; A contrastive learning module is used to filter the feature space of the student model of each source domain and the high-quality instance-level features of each category through a memory-based contrastive learning module and a feature quality filter, and store them in the memory of each source domain for contrastive learning of student models of different source domains. The memory of each source domain is updated in a first-in-first-out manner. The target detection module is used to use the trained teacher model and the trained student model to perform target detection on the image data to be processed in the target domain, and obtain the target detection result by combining the weights obtained through training.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the multi-source passive domain adaptive target detection method based on deep learning according to any one of claims 1-8.

Images