Passive field adaptive target identification method

Abstract

The invention belongs to the technical field of target recognition, and particularly discloses a passive domain target adaptive recognition method, which comprises the following steps of: training a source domain model by using source domain data to obtain a model classifier by using the characteristic that the source domain data and target domain data have attributes of the same category set, and calculating a source domain category relationship by using the source domain classifier; and on the basis, training and utilizing a teacher-student model in knowledge distillation to realize passive field adaptive target identification, specifically, adopting the teacher model to learn category information of a target domain, and introducing a source domain category relationship to fuse with the category information of the target domain, so as to obtain a category relationship which better conforms to an actual data style of the target domain. Meanwhile, by extracting the feature representation of each category in training, the similarity between the categories is reduced, and the discrimination of the categories is improved, so that the boundary line between different categories is clearer. According to the invention, the classification accuracy of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a passive domain adaptive target recognition method. Background Art

[0002] Currently, deep learning has achieved remarkable success in the field of image recognition, and this achievement relies on a large amount of labeled training data. However, models trained based on such data can often only achieve optimal performance on datasets similar to the training data. Once applied to a new dataset, its performance will drop significantly due to the existence of domain shift. Domain adaptation provides a promising solution, which enables trained models to maintain good performance in different domains through transfer learning strategies. Traditional unsupervised domain adaptation methods require the acquisition of source domain data during the adaptation process. Due to constraints such as data privacy and storage restrictions, source domain data is often difficult to obtain in practical applications. For example:

[0003] Medical imaging: In medical scenarios, once a source domain model trained on raw X-ray image datasets is available, the source domain knowledge embedded in the model can be used to classify new unlabeled CT image datasets, improving work efficiency and reducing annotation costs. However, due to data privacy requirements, only the source domain model can be accessed, not the patient case image datasets.

[0004] Autonomous driving: In autonomous driving technology, large amounts of labeled image data from simulated scenarios can help optimize autonomous driving performance in real-world scenarios. Eliminating this reliance on simulated data and utilizing models trained solely on simulated datasets can significantly improve algorithm deployment efficiency.

[0005] Terminal device field: On some terminal devices (such as RK3399 embedded devices), it is difficult to load additional source domain data due to extremely limited storage resources.

[0006] These scenarios pose significant challenges to traditional domain adaptation methods, which struggle to effectively address these complex application constraints. In contrast, passive domain adaptation effectively addresses this challenge by adapting the model to the target domain without relying on source domain data. Using only a pre-trained source domain model and unlabeled target domain data, it can achieve good performance in the target domain.

[0007] In the study of closed-set passive domain adaptation, the source and target domains share the same set of categories. This property provides valuable insights for solving the problem, especially when the category relationships between the two domains exhibit high correlation. For example, the similarity between computer and television in terms of feature attributes is significantly higher than that between computer and bicycle. This inter-category similarity or difference can serve as a bridge between the source and target domains, thereby optimizing the training process of the target model. Specifically, the category relationships in the source domain can be easily obtained through the source classifier, providing effective guidance for model transfer training to the target domain. However, current research on category relationships in passive domain adaptation remains relatively under-researched, and existing methods often assume that the category relationships between the source and target domains are completely identical. In reality, due to factors such as lighting conditions, shooting angles, and camera parameters, domain shift causes the category relationships between the two domains to be merely correlated, not completely consistent. Therefore, to fully leverage prior knowledge of category relationships in the source domain to advance passive domain adaptation research, it is necessary to overcome the adverse effects of domain shift.

[0008] Furthermore, the degree of discernibility between categories directly affects classifier performance: when category discernibility is higher, the classification boundaries constructed by the classifier for the target domain data are clearer, and the classification accuracy of the model is also improved. Therefore, it is urgent to explore a method to effectively improve category discernibility. Summary of the Invention

[0009] The purpose of the present invention is to provide a passive domain adaptive object recognition method to solve the problems of inconsistent category relationships and blurred boundaries between categories due to domain offset between the source domain and the target domain, so as to match the category relationship with the target domain data style and improve the category discriminability.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] A passive domain adaptive target recognition method, characterized by comprising the following steps:

[0012] S1, obtain labeled source domain data, train the source domain model based on the source domain data, and obtain the feature extractor and source domain classifier of the source domain model;

[0013] S2, uses the source domain classifier to calculate the source domain category relationship;

[0014] S3, based on the source domain model and the source domain category relationship, uses the teacher-student model in knowledge distillation to achieve passive domain adaptive object recognition; the specific steps include the following:

[0015] S3.1. Training the teacher-student model in knowledge distillation. The training process includes two steps: training the teacher model to obtain optimized category relations and training the student model to obtain a target model. The teacher model training process uses target domain data as input and introduces source domain category relations for optimization, obtaining optimized category relations as output. The student model training process uses target domain data as input and optimizes the optimized category relations as the learning objective to obtain the target model.

[0016] S3.2, update the teacher-student model in knowledge distillation, use the total loss function to update the student model parameters through gradient backpropagation; update the teacher model parameters from the student model through the exponential moving average method;

[0017] S3.3, passive domain adaptive object recognition is completed using the teacher-student model in the updated knowledge distillation.

[0018] The present invention provides a passive domain adaptive object recognition method. This method utilizes the characteristic that source domain data and target domain data share the same set of attributes. The source domain data is used to train a source domain model to obtain a model classifier, which is then used to calculate source domain category relationships. Furthermore, passive domain adaptive object recognition is achieved by training and utilizing a teacher-student model in knowledge distillation. Specifically, during the teacher model training process, the source domain category relationships are combined with the target domain category relationships obtained by the teacher model to construct an optimized category relationship. This not only preserves the source domain category relationships but also fully reflects the data style and actual category relationships of the target domain. During the student model training process, the optimized category relationship is used as the learning goal for the student model training. Consistency constraints are imposed on the two, enabling the student model to obtain more reasonable category relationships and overcoming the adverse effects of domain shift in the prior art. Furthermore, to make the boundaries between different categories clearer, the present invention extracts feature representations of each category in the target domain data during the student model training process to reduce the similarity between categories and improve category discrimination.

[0019] Compared with the prior art, the classification accuracy of the present invention is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Comparison between the prior art and the embodiment; (a) shows the relationship between the source domain category and the target domain category in the prior art method, (b) shows the actual relationship between the source domain category and the target domain category, and (c) shows the relationship between the source domain category and the target domain category in the embodiment method.

[0021] Figure 2This is a schematic diagram of the optimized category relationship of the passive domain adaptive target recognition method of the present invention; it includes the optimized category relationship construction process and the training loss of the student model.

[0022] Figure 3 This is a schematic diagram of the principle of category independence of the passive domain adaptive target recognition method of the present invention; specifically, it is a process of class independence of the target domain data by the student model. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0024] This embodiment provides a passive domain adaptive target recognition method, including the following steps:

[0025] S1: Obtain labeled source domain data, train a source domain model based on the source domain data, and obtain the source domain model's feature extractor and source domain classifier. The implementation process is as follows:

[0026] The source domain model is optimized using the cross entropy loss function with labeled source domain data as input, and supervised training is performed using the labels of the source domain data to obtain the feature extractor and source domain classifier of the source domain model, where the dimension of the source domain classifier is a K×d matrix, where K represents the number of categories in the source domain and d represents the dimension of the feature extractor output. The source domain data obtained in this embodiment can be a large amount of annotated image data in a simulated scene in autonomous driving technology, or a CT image dataset in the field of medical imaging. This embodiment uses a CT image dataset as the labeled source domain data.

[0027] S2 uses the source domain classifier to calculate the source domain category relationship; its implementation process is as follows:

[0028] First, based on the source domain classifier (dimension is K×d), we obtain K category information corresponding to the source domain data; then, using this category information, we use the cosine similarity algorithm to calculate the similarity between categories and express it as a K×K dimensional category similarity matrix to obtain the source domain category relationship. The calculation formula is:

[0029] R S =sim(w i ,w j )∈R K×K ;

[0030] Among them, R S represents the source domain category relationship, i represents the i-th dimension of the classifier, j represents the j-th dimension of the classifier, and w i represents the weight of classifier dimension i, w j represents the weight of classifier dimension j, and sim(·,·) represents the cosine similarity.

[0031] S3, based on the source domain model and source domain category relationship, uses the teacher-student model in knowledge distillation to achieve passive domain adaptive object recognition. The implementation process includes the following steps:

[0032] The source domain category relationship is of great reference value to the target domain as a priori category relationship. However, due to the domain offset between the source domain and the target domain, in order to better match the data of the target domain, it is necessary to fuse the information of the target domain to match the actual data style of the target domain. The teacher-student model in knowledge distillation is a dual-model structure that includes a student model and a teacher model. The structural characteristics of this model meet the requirements of optimized category relationships. Because its teacher model updates parameters through the exponential moving average method, it can learn new knowledge in the target domain while ensuring the stability of the model, that is, it has both stability and foresight. Therefore, this embodiment selects the teacher-student model in knowledge distillation as the core model, and its process of realizing passive domain adaptive target recognition includes the following steps:

[0033] S3.1, training the teacher-student model in knowledge distillation. The training process of the teacher-student model in knowledge distillation includes two parts: training the teacher model to obtain optimized category relationships and training the student model to obtain the target model.

[0034] like Figure 2 The training process of the teacher model shown in the figure takes the target domain data as input and introduces the source domain category relationship for optimization to obtain the optimized category relationship output. The specific steps include:

[0035] The teacher model is trained for multiple rounds with the target domain data as input to obtain K target domain category representations.

[0036] The average feature of each category of a batch of data is used as the corresponding category. Assume that the average feature F of each category tc ∈R K×d It is calculated based on the characteristics of a batch of data output by the teacher model, so the characteristics of the teacher model for category k are expressed as:

[0037]

[0038] in, represents the feature representation of the teacher model for category k, N b represents the amount of data in a batch, i represents the i-th data in a batch, I(·) is the indicator function, argmax(·) represents the index corresponding to the maximum value in the vector in the brackets, is the prediction result of the teacher model for sample i, is the feature output by the teacher model for sample i, and d represents the feature dimension.

[0039] Summarize the K target domain category representations and use the cosine similarity algorithm to calculate the category relationship between the teacher model and the target domain data; the specific calculation formula is as follows:

[0040]

[0041] Among them, R tc represents the category relationship of the teacher model, sim(·,·) is the cosine similarity function, and F tc is the aggregated target domain category representation matrix, It's F tc The transposed matrix of .

[0042] As training progresses, the teacher model gradually acquires more knowledge about category relationships in the target domain, while the student model increasingly learns prior category relationships from the source domain. Therefore, the teacher model should gradually assume a more important role in the category relationship fusion process, while the influence of the source domain category relationship should gradually weaken to ensure that the final category relationship can better match the target domain data style. Therefore, a time factor t∈(0,1] is introduced here to balance the two. The optimized category relationship is expressed as follows:

[0043] R o =t·R tc +(1-t)·R S ;

[0044]

[0045] Among them, R O represents the optimized category relationship, R S ∈R K×K It is the source domain category relationship obtained by the source domain classifier, Iter is the current training iteration number, and Max_iter is the maximum training iteration number.

[0046] The student model training takes the target domain data as input and optimizes the optimized category relationship as the learning goal to obtain the target model; the specific steps include:

[0047] Assume that the average feature F of each category st ∈R K×d is a batch of outputs from the student model (N b ) The characteristics of the data are calculated, and the characteristics of the student model for category k are expressed as:

[0048]

[0049] in, represents the feature representation of the student model for category k, i represents the i-th data in a batch, I(·) is the indicator function, argmax(·) represents the index corresponding to the maximum value in the vector in the brackets, Represents the prediction result of the student model for sample i, It represents the features output by the student model for sample i, and d represents the feature dimension.

[0050] Assume R st represents the category relationship between the student model and the target domain data, and its calculation formula is:

[0051]

[0052] Among them, sim(·,·) is the cosine similarity function, F st ∈R K×d Represents the category representation of the student model for K categories It's F st The transposed matrix of .

[0053] In the knowledge distillation of this embodiment, the teacher-student model in the initial training will use the source domain model to initialize the model parameters. In order to make the boundary lines between different categories clearer and maximize the mutual exclusivity between different categories, in each iteration of training, during the teacher-student model training process in the knowledge distillation, the input target domain data is subjected to category independence processing, such as Figure 3 As shown, the features of each category in the target domain data are represented according to the student model to reduce the similarity between categories and enhance the discriminability between categories.

[0054] S3.2, update the teacher-student model in knowledge distillation, use the total loss function to update the student model parameters through gradient backpropagation; update the teacher model parameters from the student model through the exponential moving average method.

[0055] The total loss function includes the loss of the baseline method, the optimized category relationship loss, and the category independence loss. Its formula is as follows:

[0056] L=L baseline +αL OCR +βL CI ;

[0057] L OCR =1-sim(R O ,R st );

[0058]

[0059] Among them, L represents the total loss; L baselinerepresents the baseline method loss. This embodiment uses the SHOT or AaD algorithm to calculate the baseline method loss. L OCR represents the category relationship loss optimized by the student model; L CI represents the class independence loss of the student model, and α and β are trade-off hyperparameters.

[0060] At the end of each iteration, the student model updates its parameters by minimizing the loss L and performing gradient backpropagation.

[0061] The teacher model obtains new parameters from the student model through the exponential moving average method, and updates the teacher model parameters by weighted summing the parameters of the historical teacher model and the parameters of the latest student model. Specifically:

[0062]

[0063] Among them, θ tc and θ st are the parameters of the teacher model and the student model respectively, Iter is the training time, and σ is the hyperparameter set to 0.99.

[0064] S3.3, input the passive domain data to be identified, that is, input new unlabeled CT image data, and use the updated teacher-student model in knowledge distillation to complete passive domain adaptive target recognition.

[0065] Figure 1 The figure shows the comparison between the existing technology and the embodiment; (a) shows the relationship between the source domain category and the target domain category in the existing method, (b) shows the relationship between the source domain category and the target domain category in the actual method, and (c) shows the relationship between the source domain category and the target domain category in the method of this embodiment. Figure 1 As shown in (a)-(c), in existing methods, the source domain category relationship Rs and the target domain category relationship Rt are identical. However, in real applications, due to factors such as data style, shooting angle, and lighting, there is domain offset between the source and target domains, resulting in related but not identical category relationships. This embodiment, however, integrates the target domain knowledge learned during training with the prior source domain category relationship. The resulting optimized category relationship serves as the learning target for the target domain model, effectively improving recognition accuracy.

Claims

1. A passive domain adaptive target recognition method, characterized in that: The following steps are involved: S1, obtain labeled source domain data, train the source domain model based on the source domain data, and obtain the feature extractor and source domain classifier of the source domain model; S2, uses the source domain classifier to calculate the source domain category relationship; S3, based on the source domain model and the source domain category relationship, uses the teacher-student model in knowledge distillation to achieve passive domain adaptive object recognition; the specific steps include the following: S3.1, training the teacher-student model used in knowledge distillation. The training process involves training the teacher model to obtain optimized category relations and training the student model to obtain the target model. The teacher model training process uses target domain data as input and simultaneously introduces source domain category relations for optimization, resulting in optimized category relations output. The student model training takes the target domain data as input and optimizes the optimized category relationship as the learning goal to obtain the target model; S3.2, update the teacher-student model in knowledge distillation, use the total loss function to update the student model parameters through gradient backpropagation; update the teacher model parameters from the student model through the exponential moving average method; S3.3, passive domain adaptive object recognition is completed using the teacher-student model in the updated knowledge distillation.

2. The passive domain adaptive target recognition method according to claim 1, characterized in that: The implementation process of step S1 of training the source domain model according to the source domain data to obtain the feature extractor and source domain classifier of the source domain model includes: The source domain model is optimized using the cross-entropy loss function with labeled source domain data as input, and the labels of the source domain data are used for supervised training to obtain the feature extractor and source domain classifier of the source domain model; the dimension of the source domain classifier is a K×d matrix, where K represents the number of categories in the source domain and d represents the dimension of the feature extractor output.

3. The passive domain adaptive target recognition method according to claim 1, characterized in that: The implementation process of calculating the source domain category relationship using the source domain classifier in step S2 includes: S2.1, based on the dimension K×d of the source domain classifier, calculate the category information of the corresponding source domain data; S2.2: Using the category information obtained in step S2.1, the cosine similarity algorithm is used to calculate the similarity between K categories and express it as a K×K-dimensional category similarity matrix to obtain the source domain category relationship. The calculation formula is: R S =sim(w i ,w j )∈R K×K ; Among them, R S Represents the source domain category relationship, i represents the i-th dimension of the classifier, j represents the j-th dimension of the classifier, and w i represents the weight of classifier dimension i, w j represents the weight of classifier dimension j, and sim(·,·) represents the cosine similarity.

4. The passive domain adaptive target recognition method according to claim 1, characterized in that: The step 3.1 also includes, during the teacher-student model training process in knowledge distillation, performing category-independent processing on the input target domain data, extracting feature representations of each category in the target domain data, so as to reduce the similarity between categories and enhance the discriminability between categories.

5. A passive domain adaptive target recognition method according to claim 4, characterized in that: The training process of the teacher model in S3.1 takes the target domain data as input and introduces the source domain category relationship for optimization. The implementation process of obtaining the optimized category relationship includes: Using the target domain data as input, the teacher model is trained for multiple rounds, and the representation of each category of each batch of data in training is calculated; Summarize the K category representations of the target domain data, and use the cosine similarity algorithm to calculate the category relationship of the teacher model for the target domain data; in this process, introduce the time factor t∈(0,1] to fuse the source domain category relationship and obtain the optimized category relationship.

6. A passive domain adaptive target recognition method according to claim 5, characterized in that: The total loss function in step S3.2 includes three parts: the loss of the baseline method, the optimized category relationship loss, and the category independence loss. Its formula is as follows: L=L vaseline +αL OCR +βL CI ; L OCR =1-sim(R O ,R st ); Among them, L represents the total loss, L baseline represents the baseline method loss. This embodiment uses the SHOT or AaD algorithm to calculate the baseline method loss. L OCR represents the category relationship loss optimized by the student, L CI represents the student category independence loss, and α and β are trade-off hyperparameters.

Images