A two-way discriminative feature alignment based hierarchical knowledge fusion method and device

Abstract

The application discloses a bidirectional discriminative feature alignment hierarchical knowledge fusion method, comprising the following steps: inputting samples into a teacher model to obtain a teacher soft prediction result set, and inputting unlabeled image data into an initial student model to obtain a student model prediction result; extracting the last layer feature and inputting the last layer feature into a common feature extractor to obtain a common feature; through a discriminative centroid clustering strategy, different class centers are far away from each other, and each teacher common feature is close to the same class center; the fuzziness of the teacher soft prediction result is measured by entropy impurity, reliable source domain features and target domain features are constructed, pseudo labels are respectively subjected to Kronecker product with the common features of the source domain and the common features of the target domain for discriminative mapping, and feature alignment is performed through a maximum average difference method; a total loss function is constructed, and the initial student model is trained through the total loss function to obtain a comprehensive student model capable of accurate classification.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence, and particularly relates to a bidirectional discriminative feature alignment hierarchical knowledge fusion method and device. BACKGROUND

[0002] In recent years, deep neural networks (DNNs) have achieved remarkable success in many artificial intelligence tasks, such as computer vision and natural language processing. However, the success of widely used DNNs relies on expensive computational costs and storage, as well as a large amount of human annotation. To alleviate the duplication of work, many researchers have released their well-trained models online, which has prompted us to reuse them in a plug-and-play manner.

[0003] As a model reuse strategy, knowledge distillation (KA) algorithms have achieved remarkable performance in various applications. They study how to effectively utilize multiple pre-trained teacher networks to train a comprehensive small student model to handle all the tasks of the teachers on unlabeled data. The student in these traditional KA methods is usually trained to imitate the teacher's output (called classification score learning) and / or intermediate layers (called feature learning) corresponding to the unlabeled data.

[0004] However, publicly available trained models usually have different architectures. Therefore, a more realistic scenario is heterogeneous knowledge distillation (HKA). In this case, the student cannot directly learn from the features introduced between each network layer of the teacher as usual. Therefore, they can only resort to classification score learning to achieve the purpose, such as Data-free KA and SKA.

[0005] Chinese Patent Publication No. CN111160409A discloses a heterogeneous neural network knowledge distillation method based on common feature learning, which includes: obtaining a plurality of pre-trained neural network models, referred to as teacher models; using the features output by the teacher models and the prediction results output by the teacher models, a common feature learning and soft target distillation method is used to guide the training of the student model; in this common feature learning process, the features of multiple heterogeneous networks are projected into a common feature interval, and the student model integrates the knowledge of the numerous teacher models, and the soft target distillation method makes the prediction results of the student model consistent with the prediction results of the teacher models, thereby obtaining a more powerful student model with the task processing capabilities of all teacher models. The above-mentioned disclosed patent is applicable to the knowledge distillation of neural network models, especially the knowledge distillation of heterogeneous image classification task models.

[0006] However, it is rude to blindly align the student features and the teacher features, and the students trained without distinguishing feature alignment are likely to align with irrelevant class features or be interfered by irrelevant class features to reduce the classification performance. Therefore, the students are difficult to learn the real data distribution from the teachers in this case, thereby leading to generally low performance and poor generalization of heterogeneous knowledge fusion. SUMMARY

[0007] The application provides a bidirectional discriminative feature alignment hierarchical knowledge fusion method, which can obtain a student model capable of accurately judging the class of unlabeled image data through less training.

[0008] A bidirectional discriminative feature alignment hierarchical knowledge fusion method comprises:

[0009] (1) obtaining an unlabeled image data set, a teacher model, constructing an initial student model, inputting the unlabeled image data as a sample into the teacher model to obtain a teacher soft prediction result set, inputting the spliced teacher soft prediction result into an activation function to obtain a pseudo label, and inputting the unlabeled image data into the initial student model to obtain a student model prediction result;

[0010] (2) extracting the last layer features in the teacher model and the initial student model respectively, and inputting the last layer features into a common feature extractor to obtain a teacher common feature set and a student common feature respectively; determining the class center based on the pseudo label corresponding to the class identifier using an incremental learning strategy, and performing distance penalty on different class centers in the teacher model through a discriminative centroid clustering strategy to make the different class centers away from each other, and at the same time, each teacher common feature is close to the same class center to obtain a clustering common feature set;

[0011] (3) splicing the teacher soft prediction result, inputting the spliced result into an activation function to obtain a pseudo label; inputting the teacher soft prediction result into an entropy impurity formula to measure the fuzziness of the teacher soft prediction result, then comparing the normalized result with the constraint boundary to screen confident teacher models meeting the requirements, mixing the clustering common feature set and the student common feature corresponding to the screened confident teacher models to obtain a mixed domain common feature set, randomly screening part of the common feature set from the mixed domain common feature set as a source domain feature, and using the Kronecker product to bind the common feature and the corresponding pseudo label, so that the common features in the source domain and the target domain are distinguished and mapped, the common features of the same class in the source domain and the target domain are mapped to the same subspace, and finally the mapped common features in the source domain feature and the target domain feature are aligned through a maximum mean difference method;

[0012] (4) constructing a total loss function, training the initial student model to obtain the final student model through the total loss function, the total loss function including a discriminant centroid clustering strategy loss function, a reliable joint combination loss function, a reconstruction loss function and a classification score loss function;

[0013] Wherein, the reconstruction loss function is constructed based on the last layer feature and the reconstruction feature in the teacher model, the reconstruction feature of the teacher model is obtained based on the common feature of the teacher model using a multi-layer convolutional neural network, the discriminant centroid clustering strategy loss function is constructed based on the plurality of class centers and each teacher common feature; the reliable joint combination loss function is constructed based on the maximum average difference loss using the difference between the Kronecker product results of the source domain feature and the target domain feature and the corresponding pseudo label; the classification score loss function is constructed based on the teacher soft prediction result set and the student model prediction result through the cross entropy loss;

[0014] (5) when applied, inputting the unlabeled image data into the final student model to obtain the class of the unlabeled image data.

[0015] and inputting the last layer feature into the common feature extractor to obtain the teacher common feature set and the student common feature, including:

[0016] First, input the last layer feature into the separately parameterized adaptive layer to align the feature dimension to obtain a plurality of adaptive layer features, and convert the plurality of adaptive layer features to the homogeneous public space through the shared extractor to obtain the teacher common feature set and the student common feature.

[0017] Based on the class label corresponding to the pseudo label, the class center of the kth class label of the batch sample number τ of the nth teacher model is determined by using the incremental learning strategy is:

[0018]

[0019] Wherein, τ is the index of the batch sample number, t n is the nth teacher model, k is the index of the class label, and m is the momentum accumulation hyperparameter.

[0020] Input the sum result into the activation function to obtain the pseudo label y as:

[0021] y = argmax (softmax (c))

[0022]

[0023] Wherein, N is the number of teacher models, is the nth teacher soft prediction result.

[0024] The fuzziness of the soft prediction of the n-th teacher model to the i-th unlabeled image data class is measured by entropy impurity is:

[0025]

[0026] wherein K is the number of class labels, is the soft prediction of the n-th teacher model to the i-th unlabeled image data class.

[0027] The number L of teacher models satisfying the comparison requirement is obtained by comparing the fuzziness after normalization with the marginal constraint:

[0028]

[0029] wherein, is a normalization operator, and η is a marginal constraint value, is the n-th teacher model satisfying the comparison requirement.

[0030] Total loss function is:

[0031]

[0032]

[0033]

[0034]

[0035]

[0036] wherein, are the reconstruction feature and the last layer feature of the i-th sample of the n-th teacher model respectively, B is the batch quantity, N is the number of teachers, and λ C is a classification score loss weight, λ J is a joint group alignment loss weight, λ DR is the weight of the reconstruction loss and the class center loss in the total loss, is a classification score loss function, is a reliable joint combination loss function, is a discriminant centroid clustering strategy loss function, is a reconstruction loss function, and B is the number of unlabeled image data, is a cross loss function, is the prediction result of the i-th unlabeled image data student model, is the soft prediction result of the n-th teacher model to the i-th unlabeled image data, T is the number of mixed common feature domain arrangement combinations of groups, P is the number of common features in the source domain, Q is the number of common features in the target domain, is a logic function, is the Kronecker product of the pseudo-label of the i-th unlabeled image data and the p-th common feature of the r-th group source domain, is the Kronecker product of the pseudo-label of the j-th unlabeled image data and the q-th common feature of the r-th target domain, so that each teacher common feature is close to the same class center, so that different class centers are distance penalized, and different class centers are far away from each other, alpha is a weighting parameter, and N is the number of teacher models, is the class center of the y i -th class in the n-th teacher model, k1 and k2 are both indexes of class identifiers, but k1 and k2 are not equal, and v is a constraint margin that controls the distance between different class centers in the teacher model.

[0037] the class center of the y i -th class in the n-th teacher model is:

[0038]

[0039]

[0040] wherein tau is an index of the number of batch samples, y i is the pseudo-label of the i-th unlabeled image data.

[0041] A hierarchical knowledge fusion device for bidirectional discriminative feature alignment, comprising a computer memory, a computer processor and a computer program stored in the computer memory and executable on the computer processor, characterized in that the computer memory adopts a final student model;

[0042] When the computer processor executes the computer program, the following steps are implemented: inputting unlabeled image data into the final student model to obtain the category of the unlabeled image data.

[0043] Compared with the prior art, the beneficial effects of the present application are:

[0044] 1. The knowledge fusion method for bidirectional discriminative feature alignment of the present application first constructs a double discriminative feature learning process in heterogeneous knowledge fusion (HKA), which not only ensures the discriminativeness of the teacher features before alignment, but also promotes the student to learn each teacher comprehensively and discriminatively.

[0045] 2、Hierarchical feature alignment, including class-level, group-level and global-level feature alignment, which maps features orthogonally to their semantic subspaces, which hinders the cumbersome interference knowledge from being transferred to the student, improving the accuracy of the model.

[0046] 3、The hierarchical feature alignment method of the application anchors the differences to each class. Joint group feature alignment (JGFA) decouples the relationships and differences in complex multi-teacher knowledge fusion layer by layer by fully mining the relationships in the local group composed of multiple domains under each class, making it easier to capture and model, so as to promote the student to fully integrate the complementary knowledge of different domain teachers, and improve the generalization ability of the student model.

[0047] 4、The centroid clustering strategy (DCCS) module of the application alleviates the loss of discriminative information in the common feature space, and complements the joint group feature alignment (JGFA), only transferring the most discriminative knowledge to the student, alleviating the storage pressure of the small model, making KA easier to deploy on small edge devices. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 The structure diagram of the bidirectional discriminative knowledge fusion system is provided for the specific embodiment;

[0049] Figure 2 The common space feature extractor network structure diagram is provided for the specific embodiment;

[0050] Figure 3 The hierarchical knowledge fusion system block diagram based on bidirectional discriminative feature alignment is provided for the specific embodiment;

[0051] Figure 4 The flowchart of the bidirectional discriminative feature alignment method is provided for the specific embodiment;

[0052] Figure 5 The working principle diagram of the bidirectional discriminative feature alignment hierarchical knowledge fusion is provided for the specific embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical scheme and technical effect of the application more clear, the application is further described in detail below in combination with the drawings of the specification.

[0054] As shown in Figure 1 A bidirectional discriminative feature alignment hierarchical knowledge fusion (DDFA) system, comprising:

[0055] A common space feature extractor module for eliminating the dimensional differences between the features output by heterogeneous networks and converting the features of all teachers and student networks to a homogeneous public space, the network structure being based on an adaptive layer of each network and a shared extraction layer.

[0056] Discriminative Centroid Clustering Strategy (DCCS) module ensures that the transformed features can be mapped back to the original space to guarantee accuracy while ensuring the distinguishability of the features in the common space. This module performs statistical analysis on the teacher features in the common feature space and uses an incremental learning strategy to simulate class centers, clustering the same class features to the class centers to make them as close as possible to share the same distribution as in the original space of the teacher. At the same time, the introduction of the constraint margin is to punish the distance between different classes, so that they are far away from each other, thereby achieving the effect of controlling the distance of different classes of teacher samples, thereby prompting the feature fusion to keep the intra-class aggregation and inter-class distinguishability;

[0057] Joint Group Feature Alignment (JGFA) module, the motivation of joint group feature alignment (JGFA) is to first increase the difficulty of imitating irrelevant class features, while making students more easily imitate the features of the same class teacher, thereby avoiding the inconsistency of score learning. Second, all teachers and students are regarded as a mixed domain, and any local block is traversed and aligned, so that the relationship within the domain can be fully utilized to promote the consensus of all domains. Once all local parts composed of multiple domains can be aligned into a whole, the student can fuse the features of all teachers, rather than their compromised representation;

[0058] Adaptive Teacher Selection Learning module uses the information entropy index to effectively measure the ambiguity of teacher prediction to filter the teachers that can produce predictions with smaller prediction ambiguity for learning. Thus, the quality of knowledge fusion is improved.

[0059] As shown in Figure 3 , a hierarchical knowledge fusion method of bidirectional discriminative feature alignment, the specific steps are as follows:

[0060] S1: Given an unlabeled data set χ under a small batch image recognition task, a class label k, a plurality of well-trained teacher models {t1,…,t N} engaged in different classification tasks, and an untrained student model s, wherein N is the number of teacher models.

[0061] S2: The unlabeled image data is taken as a sample, and the sample is input into the teacher model to obtain a teacher soft prediction result set, denoted as Where n is the index of the teacher model, the unlabeled image data is input into the initial student model to obtain the student model prediction result c s ; The teacher soft prediction result set is distilled as a target, and a classification score loss function is constructed based on the teacher soft prediction result set and the student model prediction result through cross-entropy loss. The output of the student model is driven to be consistent with the teacher model through the classification score loss function, as shown in Figure 5 (a), that is, the classification score learning process, the classification score loss function is:

[0062]

[0063] where B is the number of unlabeled image data, is the cross loss function, is the student model prediction result for the i-th unlabeled image data, is the soft prediction result of the n-th teacher model for the i-th unlabeled image data.

[0064] S3: As shown in Figure 2 , the last layer features in the teacher model and the initial student model are extracted respectively, i.e. the last layer features of the teacher model are the last layer features of the student model are F S , which are input into a separately parameterized adaptation layer with 1x1 kernel convolution to align the feature dimensions of all heterogeneous teachers and students, thereby generating adaptation layer features, after which these features are input into a small and learnable subnetwork consisting of a small convolutional network (1x1 stride) composed of three residuals. Since the parameters are shared between teachers and students, it is called a shared extractor, which converts these adaptation layer features into a common feature space (CFS), generating public features and h s , respectively, where f = C x H x W and H, W, C represent height, width and the number of channels of common features, respectively. The loss in the process of converting the last layer features of the teacher model into common features is reduced to within the loss threshold range by the reconstruction loss function, which is is:

[0065]

[0066] where, are the reconstruction features and the last layer features of the i-th sample of the n-th teacher model, respectively, B is the batch size, N is the number of teachers, and α is the weighting parameter.

[0067] S4: A discriminative centroid clustering strategy (DCCS) module is additionally introduced in the common feature space (CFS), and its loss function is The common feature space (CFS) is discriminatively corrected, and the specific steps are as follows:

[0068] Based on the class label corresponding to the pseudo label, an incremental learning strategy is used to determine the class center, and the class center of the k-th class label of the batch sample number τ of the n-th teacher model is is:

[0069]

[0070] Where τ is the index of the batch sample number, τ max ∈(500,800), t n Let k be the index of the class identifier, m be the momentum accumulation hyperparameter, and 0 ≤ m ≤ 1.

[0071] By employing the Discriminant Centroid Clustering (DCCS) strategy, the discriminant centroid clustering strategy loss function is used. Distance penalties are applied to different cluster centers in the teacher model to keep them further apart, while common features of each teacher are aligned with cluster centers of the same class to obtain a set of common cluster features. The discriminant centroid clustering strategy loss function is used in this process. for:

[0072]

[0073] in, This allows the common characteristics of each teacher to converge towards the center of the same class. This causes a distance penalty to be imposed on centers of different classes, making them farther apart, such as... Figure 5 As shown in (b), α is the trade-off parameter, and N is the number of teacher models. For the nth teacher model, the yth... i The class centers of each class, k1 and k2 are both indices of the class identifier, but k1 and k2 are not equal, and v is a constraint margin that controls the distance between different class centers in the teacher model.

[0074] The y-th teacher model in the nth teacher model i Class center of each class for:

[0075]

[0076]

[0077] Where τ is the index of the batch sample number, y i Let be the pseudo-label of the i-th unlabeled graph data.

[0078] S5: Joint Group Feature Alignment (JGFA) first increases the difficulty of mimicking features from irrelevant classes, while simultaneously making it easier for student models to mimic features from similar teacher models, thus avoiding inconsistent score learning. Joint Group Feature Alignment (JGFA) introduces a simple yet effective cross-covariance operator. The joint semantic feature alignment is realized. By manipulating the multiplicative interaction of multiple random variables, the common features of the same kind are novelly mapped to the same feature subspace, thereby facilitating the transferability of the common features in the marginal alignment; a mixed group feature alignment module is introduced to traverse and align all local blocks of the domains, so that the intra-domain relationship can be fully utilized to maximize the domain consensus to fully facilitate knowledge fusion; the specific steps are as follows:

[0079] The teacher soft prediction result set is added, and the added result is input into an activation function to obtain a pseudo label y:

[0080] y=argmax(softmax(c))

[0081]

[0082] Wherein, N is the number of teacher models, is the n-th teacher soft prediction result.

[0083] Simply assigning the same weight to all teacher models and being indifferent to the advantages and disadvantages of various teacher models may even reduce the performance of multi-view learning in KA. Therefore, a reliable JGFA (rJGFA) is proposed to learn more 'confident' prediction corresponding features from teachers. Here, we use entropy impurity to measure the ambiguity of each teacher output, and the specific steps are as follows:

[0084] The ambiguity of the teacher soft prediction result is measured by entropy impurity, and after normalization, the ambiguity is compared with the marginal constraint to screen the teacher models that meet the comparison requirement, and the specific steps are as follows:

[0085] The ambiguity of each teacher prediction can be defined as:

[0086]

[0087] The smaller the value is, the higher the confidence of the prediction is, wherein K is the number of class labels, is the n-th teacher model for the soft prediction of the i-th unlabeled image data class. If the i-th image belongs to k, Otherwise

[0088] According to this ambiguity, the learning from the teacher can be selected, first, L teacher models that are confident on sample i are screened, i.e. the teacher models that meet the comparison requirement, which satisfy Wherein, wherein, is a normalization operator, and η is a marginal constraint value, is the n-th confident teacher model that meets the comparison requirement;

[0089] Constructing mixed domain group-level feature alignment to further explore the relationship in each local block composed of multiple domains under each class, such as Figure 5 (c) instead of learning each teacher model separately, all teachers and students are regarded as a mixed domain, and any local block thereof is traversed and aligned, so that the intra-domain relationship can be fully utilized to promote consensus of all domains, and once all local parts composed of multiple domains can be aligned as a whole, the student can fuse the features of all teachers instead of their compromised representation, which is easier to capture and more detailed than directly modeling the overall difference of all domains, and is helpful to model the difference of all domains, and the specific steps are as follows:

[0090] Self-confident teacher model corresponding to cluster common feature set and student common feature Construction of the rth group of domains and its features L+1 domains, randomly select P domains from L+1 domains to connect as source domain features Splice the features in the remaining Q=L-P+1 domains as target domain features The number of permutations and combinations

[0091] JGFA introduces a simple and effective cross-covariance operator where denotes the Kronecker product, which can manipulate the multiplicative interaction of multiple random variables. Novel mapping of common features h using its one-hot semantic information y to map features of the same class to the same subspace to facilitate the transferability of features of the same class in edge alignment;

[0092] Kronecker product of pseudo-labels with source domain common features and target domain common features enables features of the same class in source and target domains to be mapped to the same subspace, as shown in Figure 5 (c) to obtain source domain cross-covariance and target domain cross-covariance, the Kronecker product is SJFO, and the source domain features and target domain features based on the results of Kronecker product with the corresponding pseudo-labels are aligned by the maximum mean difference method (MMD), i.e. by training the reliable joint combination loss function.

[0093] where the reliable joint combination loss function is:

[0094]

[0095] where T is the number of permutations and combinations of multiple mixed common feature domains, P is the number of domains in the source domain, and Q is the number of domains in the target domain, is a logical function, is a Kronecker product of pseudo-labels of the i-th unlabeled image data and p common features of the r-th group source domain, is a Kronecker product of pseudo-labels of the j-th unlabeled image data and q common features of the r-th target domain. JGFA forces students to learn features of all teachers under each class. For example, the features that students learn also include features extracted from area A and area B, which are features that expert teachers (only focus on area C to extract features) cannot capture, such as Figure 5 (d).

[0096] S7: Finally, integrate all the losses above: total loss is defined as:

[0097]

[0098] wherein, wherein, λ C is a classification score loss weight, λ J is a joint group alignment loss weight, λ DR is a weight of the reconstruction loss and the class center loss in the total loss.

[0099] The final student model is obtained by training the initial student model through the total loss function. When applied, the unlabeled image data is input into the final student model to obtain the class of the unlabeled image data.

[0100] In summary, the method provided by the embodiment makes the common features more discriminative for the student to imitate, and the hierarchical feature learning method proposed also drives the student to learn from the teachers differently, which makes the student not only more easily integrate the knowledge of all teachers in each class, but also naturally establish semantic coherence between feature learning and classification score learning. Therefore, not only the accuracy of knowledge integration can be effectively improved, but also the complementarity information in the teachers can be fully integrated to improve the generalization.

Claims

1. A hierarchical knowledge fusion method based on bidirectional discriminative feature alignment, characterized in that, include: (1) Obtain the unlabeled image dataset and teacher model, construct the initial student model, use the unlabeled image data as samples, input the samples into the teacher model to obtain the teacher soft prediction result, and input the stitched teacher soft prediction result into the activation function to obtain pseudo label, and input the unlabeled image data into the initial student model to obtain the student model prediction result; (2) Extract the last layer features from the teacher model and the initial student model respectively, and input the last layer features into the common feature extractor to obtain the teacher common feature set and the student common feature set respectively; Based on the class identifiers corresponding to the pseudo-labels, an incremental learning strategy is adopted to determine the class centers. By using a discriminative centroid clustering strategy, distance penalties are applied to different class centers in the teacher model to make different class centers move away from each other. At the same time, the common features of each teacher are brought closer to the same class centers to obtain a set of common cluster features. (3) Input each teacher soft prediction result into the entropy impurity formula to measure the fuzziness of the teacher soft prediction result. Then compare the fuzziness normalized result with the constraint boundary and select the confident teacher model that meets the requirements. Mix the cluster common feature set corresponding to the selected confident teacher model with the student common feature to obtain the mixed domain common feature set. Randomly select a part of the common feature set from the mixed domain common feature set as the source domain feature. Then use the remaining common feature set as the target domain feature. Use Kronecker product to bind the common features and the corresponding pseudo label so that the common features of the source domain and the target domain can be discriminatively mapped, so as to achieve the purpose of mapping the same type of features in the source domain and the target domain to the same subspace. Finally, align the mapped common features in the source domain features and the target domain features using the maximum average difference method. (4) Construct the total loss function, and train the initial student model to obtain the final student model through the total loss function. The total loss function includes the discriminant centroid clustering strategy loss function, the reliable joint combination loss function, the reconstruction loss function, and the classification score loss function. Specifically, a reconstruction loss function is constructed based on the last layer features and reconstructed features in the teacher model; reconstructed features of the teacher model are obtained using a multi-layer convolutional neural network based on the common features of the teacher model; a discriminative centroid clustering strategy loss function is constructed based on multiple class centers and the common features of each teacher; a reliable joint combination loss function is constructed using the maximum average difference loss based on the Kronecker product of the source domain features and target domain features with their corresponding pseudo-labels; and a classification score loss function is constructed using cross-entropy loss based on the set of teacher soft prediction results and student model prediction results. (5) When applying, input the unlabeled image data into the final student model to obtain the category of the unlabeled image data.

2. The hierarchical knowledge fusion method with bidirectional discriminative feature alignment according to claim 1, characterized in that, The final layer of features is then input into a common feature extractor to obtain the teacher common feature set and the student common feature set, including: First, the features of the last layer are input into the separately parameterized adaptation layer to align the feature dimensions and obtain multiple adaptation layer features. Then, the multiple adaptation layer features are transformed into a homogeneous common space through a shared extractor to obtain the set of common features of teachers and common features of students.

3. The hierarchical knowledge fusion method with bidirectional discriminative feature alignment according to claim 1, characterized in that, Based on the class identifiers corresponding to the pseudo-labels, an incremental learning strategy is used to determine the class center of the k-th class identifier with a batch sample size of τ for the nth teacher model. for: Where τ is the index of the batch sample number, t n Let k be the index of the class identifier and m be the momentum accumulation hyperparameter for the nth teacher model.

4. The hierarchical knowledge fusion method with bidirectional discriminative feature alignment according to claim 1, characterized in that, The summation result is input into the activation function to obtain the pseudo-label y: y = argmax(softmax(c)) Where N is the number of teacher models. This represents the soft prediction result for the nth teacher.

5. The hierarchical knowledge fusion method with bidirectional discriminative feature alignment according to claim 1, characterized in that, Entropy impurity is used to measure the ambiguity of the nth teacher model's soft prediction of the i-th unlabeled image data category. for: Where K is the number of class identifiers, This represents the soft prediction of the nth teacher model for the i-th unlabeled image data category.

6. The hierarchical knowledge fusion method with bidirectional discriminative feature alignment according to claim 1, characterized in that, After fuzziness normalization, the number of teacher models L that meet the comparison requirements is obtained by comparing them with the marginal constraints: in, Here, η is the normalization operator, and η is the marginal constraint value. The nth teacher model to meet the comparison requirements.

7. The reliable bidirectional discriminative feature-aligned hierarchical knowledge fusion method according to claim 1, characterized in that, Total loss function for: in, Let B and N be the reconstructed features of the i-th sample in the n-th teacher model and the features of the last layer, respectively. Let B be the batch size, N be the number of teachers, and λ be the number of samples. C For the classification score loss weights, λ J For the joint group alignment loss weights, λ DR To reconstruct the weights of the loss and class center loss in the total loss, The classification score loss function is... For reliable joint combination loss function, To determine the loss function of the centroid clustering strategy, Here, B represents the reconstruction loss function, and B is the number of unlabeled image data points. The crossover loss function, For the student model prediction results for the i-th unlabeled image data, For the nth teacher model soft prediction result of the i-th unlabeled image data, T represents the number of permutations and combinations of multiple mixed common feature domains, P represents the number of common features in the source domain, and Q represents the number of common features in the target domain. For logical functions, Perform a Kronecker product between the pseudo-label of the i-th unlabeled image data and the p common features of the r-th group source domain. To perform the Kronecker product on the q common features of the pseudo-label of the j-th unlabeled image data and the r-th target domain, This allows the common characteristics of each teacher to converge towards the center of the same class. This involves applying a distance penalty to centers of different classes, causing them to move further apart. α is a tradeoff parameter, and N is the number of teacher models. For the nth teacher model, the yth... i The class centers of each class, k1 and k2 are both indices of the class identifier, but k1 and k2 are not equal, and v is a constraint margin that controls the distance between different class centers in the teacher model.

8. The hierarchical knowledge fusion method with bidirectional discriminative feature alignment according to claim 7, characterized in that, The y-th teacher model in the nth teacher model i Class center of each class for: Where τ is the index of the batch sample number, y i Let be the pseudo-label of the i-th unlabeled graph data.

9. A hierarchical knowledge fusion apparatus with bidirectional discriminative feature alignment, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, characterized in that, The computer memory uses the final student model as described in any one of claims 1 to 8; When the computer processor executes the computer program, it performs the following steps: inputting unlabeled image data into the final student model to obtain the category of the unlabeled image data.

Images