Multi-view generalized zero sample classification auxiliary method and device based on PU learning

Abstract

The invention discloses a multi-view generalized zero sample classification auxiliary method and device based on PU learning, and belongs to the technical field of picture classification, and the method comprises the steps: obtaining pictures, and extracting the multi-view visual features of the pictures in a training set and a test set; constructing a reference model; constructing a multi-view PU classification model; inputting the multi-view visual features corresponding to the pictures in the training set and the test set into a multi-view PU classification model, and training the multi-view PU classification model by using an alternating direction multiplier method; resetting a semantic prototype of each unseen class based on the first semantic projection; resetting a semantic prototype of each seen class based on the second semantic projection; and classifying to-be-identified pictures by using the reference model. According to the method, the unseen class samples are separated from the to-be-recognized test samples by utilizing the PU learning framework and are used for class semantic calibration, so that the deviation of the model to the seen class samples can be effectively relieved, and the generalization performance of the generalized zero sample classifier is remarkably improved.

Description

Technical Field

[0001] The present invention relates to a zero-shot classification assistance method, specifically a multi-view generalized zero-shot classification assistance method and device based on PU learning, belonging to the technical field of image classification. Background Art

[0002] In recent years, the application of multi-view visual feature fusion methods in zero-shot learning (ZSL) has made remarkable progress. In practical application scenarios, by using different feature extractors, visual representations of images can be obtained from multiple perspectives. Due to the heterogeneity between different perspective features, visual data based on multi-view feature representations can provide more comprehensive visual information of images. Relevant research results show that using appropriate methods to fuse multi-view visual information can significantly improve the performance of ZSL learning. However, multi-view fusion may further exacerbate the domain shift problem, resulting in poor performance of the model in the generalized zero-shot learning (GZSL) task. In view of this, most existing methods balance the decision boundary of the model between seen classes and unseen classes by introducing calibration biases. Although the domain shift problem is alleviated to a certain extent, the improvement effect is still relatively limited. If the seen-class samples and unseen-class samples can be identified in the samples to be predicted, the above problems can be solved. Therefore, there is an urgent need for a new calibration strategy to identify the seen-class samples and unseen-class samples in the samples to be predicted to improve the generalization performance of the generalized zero-shot classifier. Summary of the Invention

[0003] Object of the Invention: Aiming at the above problems, the object of the present invention is to provide a multi-view generalized zero-shot classification assistance method and device based on PU learning, which uses the multi-view visual features of picture samples to train a classifier, improves the generalization performance of the classifier, and thus realizes more accurate recognition of images.

[0004] Technical Solution: One aspect of the present invention provides a multi-view generalized zero-shot classification assistance method based on PU learning, which is characterized by including the following steps: Obtain pictures, construct a training set using seen-class pictures, construct a test set using seen-class pictures and unseen-class pictures, and extract the multi-view visual features of the pictures in the training set and the test set; Construct a baseline model; Construct a multi-view PU classification model; Input the multi-view visual features corresponding to the pictures in the training set and the test set into the multi-view PU classification model, train the multi-view PU classification model using the alternating direction method of multipliers, and use the trained multi-view PU classification model to identify the test samples, and screen out the pseudo-unseen-class pictures with high confidence; Input the screened pseudo-unseen class images with high confidence into the baseline model, output the corresponding first semantic projection, and reset the semantic prototype of each unseen class based on the first semantic projection; Input the images in the training set into the baseline model to obtain the corresponding second semantic projection, and reset the semantic prototype of each seen class based on the second semantic projection; Use the baseline model to classify the images to be recognized.

[0005] Furthermore, represent the multi-view PU classification model as problem P, and the expression is: Problem P: , In the formula, represents the view number, m represents the total number of views, , and are variables to be optimized, represents the parameters of the multi-view PU classification model under the th view, represents the corrected negative mean of the test set features under the th view, is the slack variable, represents the visual feature matrix of the training samples on the th view, represents the th training sample's feature vector under the th view, represents a real matrix space, where is the number of rows and n is the number of columns, represents the sum of the visual feature vectors of the training samples on the th view, , represents the transpose; represents the visual feature matrix of the test set samples on the th view, represents the th test image's feature vector under the th view; is the hyperparameter, represents the F norm; , is the prior probability of the training samples, represents the number of test samples; is the proportion of the seen class samples in the test set among all seen class samples; Construct the constraint conditions of this multi-view PU classification model, which are expressed as: , , , Among them, the first constraint is used to ensure that the classification results between perspectives satisfy complementarity, and the second constraint is used to ensure the robustness of the multi-perspective PU classification model; In the formula, is the test set, represents the estimated value obtained through the grouped mean median estimator of, represents the th feature vector of the test image under the th perspective; represents the covariance matrix of the images in the test set on the th perspective feature, and the formula is:

[0006] . Furthermore, the steps to obtain the estimated value of through the grouped mean median estimator include: Dividing the test set into sample sets of the same size, denoted as ; Calculating the mean vector of each group of samples, where ; Then calculating the median of each group of samples, and the formula is: where is the Euclidean distance from the th group to the th group, and is the median, denotes the median; .

[0007] Furthermore, the steps to train the multi-perspective PU classification model using the alternating direction method of multipliers include: Step 41, initializing , setting the maximum number of iterations k max , letting the iteration number , determining the convergence threshold , ; Step 42, constructing the augmented Lagrangian function of the multi-perspective PU classification model, and the expression is: , Construct constraint conditions: , , where, is the Lagrange multiplier matrix, is the penalty parameter; Step 43, fix the parameters and , and only update , then the multi-view PU classification model is represented as sub-problem p1, and the expression is: Sub-problem p1: , Let the gradient in sub-problem p1 be 0, solve sub-problem p1, and obtain the update formula for as: , In the formula, I represents the identity matrix; Step 44, fix the parameters and , and only update , then the multi-view PU classification model is represented as sub-problem p2, and the expression is: Sub-problem p2: , , Let the gradient in sub-problem p2 be 0, solve sub-problem p2, and obtain the update formula for as: , Step 45, fix the parameters and , and only update , then the multi-view PU classification model is represented as sub-problem p3, and the expression is: Sub-problem p3: , Constraint conditions: , According to the duality theory, by constructing the Lagrangian function and solving the corresponding dual problem, the update formula for is obtained as: , Step 46, update through the following formula: , Step 47: Determine whether the convergence condition is satisfied. Stop the iteration until the convergence condition is satisfied or the maximum number of iterations is reached; otherwise, update the number of iterations , and return to Step 43; where the convergence condition is that the difference between the changes in two adjacent iterations is less than a set threshold, expressed as: , , , .

[0008] Furthermore, the steps of screening out pseudo unseen-class images with high confidence include: Calculate the confidence of the test set images , and the calculation formula is: , Arrange all test set images in ascending order according to the confidence value, and select the samples with confidence values less than the 50th percentile from the sorted image samples as high-confidence pseudo unseen-class images.

[0009] Furthermore, the steps of resetting the semantic prototype of each unseen class based on the first semantic projection include: Step 51: Set the maximum number of iterations to , and initialize the number of iterations to 1; Step 52: Calculate the predicted probability vector at the current iteration , and the formula is: , wherein, is the unseen-class semantic prototype matrix; is the semantic projection of some unseen-class test samples with high confidence under the baseline model; and are both element-wise operations; Return the row vector of the maximum element in each column; Return the row vector of the sum of each column; is a hyperparameter greater than 0; Step 53: Calculate the pseudo labels of the unseen-class test samples at the current iteration , and the formula is: , wherein, represents the vector of the indices of the maximum element in each column; is the pseudo label vector of the predicted test set; Step 54: Calculate the probability threshold at the current iteration , the formula is: , Step 55, update the prototype according to the threshold , the formula is: , where, represents the column vector of the average value returned for each row; Step 56, perform regularization on , the formula is: , Step 57, repeat Step 52 to Step 56 until the maximum number of iterations is reached and stop the iteration to obtain the calibrated pseudo-unseen class semantics.

[0010] Furthermore, the steps of using the baseline model to classify the image to be recognized include: Input the image to be recognized into the baseline model, and calculate the predicted label vector through the following formula: , where, represents the calibrated semantic matrix of the image to be recognized, is the semantic projection of the total sample set under the baseline model; returns the vector of the indices of the maximum elements in each column; is the pseudo-label vector of the image to be recognized predicted.

[0011] On the other hand, the multi-view generalized zero-shot classification assistance system based on PU learning provided by the present invention includes: A data preparation module for extracting multi-view visual features of training pictures and test pictures; A model learning module for feeding the extracted multi-view visual features into a multi-view PU classification model and training using the alternating direction multiplier method; using the established multi-view PU classification model to identify test samples and screening out pseudo-unseen class pictures with high confidence; A calibration module for inputting the pseudo-unseen class pictures with high confidence screened out into the baseline model and resetting the semantic prototype of each unseen class using the corresponding semantic projection output; then inputting the pictures of the training set into the baseline model and resetting the semantic prototype of each seen class using the corresponding semantic projection output; An image recognition module for classifying the image to be recognized using the baseline model.

[0012] Beneficial effects: Compared with the prior art, the present invention has the remarkable advantages that: The present invention constructs a binary classification model for seen classes and unseen classes applicable to multi-view visual features; based on the established benchmark model, the semantic prototypes of unseen classes are calibrated using test samples with relatively high confidence in some unseen classes, and the semantic prototypes of seen classes are calibrated using samples from the training set; the present invention uses the PU learning framework to separate unseen class samples from the test samples to be recognized and uses them for class semantic calibration, which can effectively alleviate the bias of the model towards seen class samples, thereby significantly improving the generalization performance of the generalized zero-shot classifier. Description of the Drawings

[0013] Figure 1 It is a flowchart of an auxiliary method for multi-view generalized zero-shot classification based on PU learning; Figure 2 It is a flowchart block diagram of an auxiliary method for multi-view generalized zero-shot classification based on PU learning. Detailed Embodiments

[0014] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0015] Embodiment 1 PU learning (Positive and Unlabeled Learning, abbreviated as PU learning), as a binary classification method in a weakly labeled scenario, can be trained only with positive samples and unlabeled samples in the absence of negative samples. For generalized zero-shot learning, if seen class samples are regarded as positive examples and unseen class samples are regarded as negative examples, then the samples available for training only contain positive examples and unlabeled samples, which is very compatible with the modeling paradigm of PU learning, where positive examples represent seen class samples with known labels, and unlabeled samples are samples to be predicted, including both seen class and unseen class samples. Therefore, in this example, a new auxiliary method for multi-view generalized zero-shot classification based on PU learning is proposed to improve the generalization performance of the generalized zero-shot classifier.

[0016] Combined with Figure 1 and Figure 2 , the auxiliary method for multi-view generalized zero-shot classification based on PU learning described in this embodiment includes the following steps: Step 1, obtain pictures, construct a training set using seen class pictures, construct a test set using seen class pictures and unseen class pictures, and extract multi-view visual features of the pictures in the training set and the test set.

[0017] Obtain the image to be detected, which includes seen-class images and unseen-class images. Use the seen-class images to construct a training set, and use the seen-class images and unseen-class images to construct a test set. Use the pre-trained deep convolutional neural network model GoogLeNet on the ImageNet database to extract multi-visual features of the images in the training set, such as Figure 2 In which, View 1, View 2, and View n represent n different visual features, and two different scaling ratios of 0.1 and 1 are used respectively to represent the local perspective A and the global perspective B. The 1024-dimensional vector activated by the top hidden unit of GoogLeNet is used as the visual feature.

[0018] Step 2, construct a baseline model.

[0019] In this example, use the multi-view visual-semantic mapping model in a case-based multi-view visual fusion transductive zero-shot classification method disclosed in Patent CN117541882B, and use it as the baseline model after training.

[0020] Step 3, construct a multi-view PU classification model.

[0021] Furthermore, represent the multi-view PU classification model as problem P, and the expression is:[[]] Problem P: , In the formula, represents the view serial number, m represents the total number of views, , and are variables to be optimized, represents the parameters of the multi-view PU classification model under the th view, represents the negative mean of the corrected test set features under the th view, is a slack variable, represents the visual feature matrix of the training samples on the th view, represents the feature vector of the th training sample under the th view, represents a real matrix space, where is the number of rows and n is the number of columns, represents the sum of the visual feature vectors of the training samples on the th view, , represents the transpose; represents the test set samples on the Visual feature matrix from a perspective denotes the th test image's feature vector at the th perspective; is a hyperparameter, denotes the F-norm; , which is the prior probability of the training samples, denotes the number of training samples, which is the same as the above number of columns, denotes the number of test samples; is the proportion of the seen-class samples in the test set among all seen-class samples; The constraints for constructing this multi-view PU classification model are expressed as: , , , Among them, the first constraint is used to ensure that the classification results between perspectives satisfy complementarity, and the second constraint is used to ensure the robustness of the multi-view PU classification model; In the formula, is the test set, denotes the estimated value obtained through the grouped mean median estimator , denotes the th test image's feature vector at the th perspective; denotes the covariance matrix of the images in the test set on the th perspective feature, and the formula is: .

[0022] Furthermore, the steps for obtaining the estimated value of through the grouped mean median estimator include: Divide the test set into sample sets of the same group size, denoted as ; Calculate the mean vector of each group of samples, where ; Then calculate the median of each group of samples, and the formula is: , where is the Euclidean distance from the i-th group to the j-th group 's median, denotes the median; Find the index of the group with the smallest median distance, then the The mean vector of the group is the desired estimated value .

[0023] Step 4: Input the multi-view visual features corresponding to the pictures in the training set and the test set into the multi-view PU classification model. Use the alternating direction method of multipliers to train the multi-view PU classification model, and use the trained multi-view PU classification model to identify the test samples, and screen out the pseudo-unseen class pictures with high confidence.

[0024] Furthermore, the steps of training the multi-view PU classification model by using the alternating direction method of multipliers include: Step 41: Initialize , set the maximum number of iterations k max , let the number of iterations , determine the convergence threshold , ; Step 42: Construct the augmented Lagrangian function of the multi-view PU classification model, and the expression is: , Construct the constraint conditions: , , where is the Lagrange multiplier matrix, is the penalty parameter; Step 43: Fix the parameters and , only update , then the multi-view PU classification model is expressed as sub-problem p1, and the expression is: Sub-problem p1: , Let the gradient in sub-problem p1 be 0, solve sub-problem p1, and obtain the update formula of as: , In the formula, I represents the identity matrix; Step 44: Fix the parameters and , only update , then the multi-view PU classification model is expressed as sub-problem p2, and the expression is: Sub-problem p2: , , Set the gradient in sub-problem p2 to 0 and solve sub-problem p2 to obtain The update formula for is: , Step 45, fix the parameters and , and only update , then represent the multi-view PU classification model as sub-problem p3, and the expression is: Sub-problem p3: , Constraint condition: , According to the duality theory, by constructing the Lagrangian function and solving the corresponding dual problem, obtain The update formula is: , Step 46, update through the following formula: , Step 47, determine whether the convergence condition is satisfied, and stop the iteration until the convergence condition is satisfied or the maximum number of iterations is reached; otherwise, update the number of iterations , and return to Step 43; where the convergence condition is that the difference between two adjacent iterations is less than the set threshold, which is expressed as: , , , .

[0025] Furthermore, the steps for screening out pseudo-unseen class pictures with high confidence include: Calculate the confidence of the test set pictures , and the calculation formula is: , Arrange all test set pictures in ascending order according to the confidence value, and select the samples with confidence values less than the 50% quantile from the sorted picture samples as the pseudo-unseen class pictures with high confidence.

[0026] Step 5, input the screened pseudo-unseen class pictures with high confidence into the benchmark model, output the corresponding first semantic projection, and reset the semantic prototype of each unseen class based on the first semantic projection.

[0027] Furthermore, the steps for resetting the semantic prototype of each unseen class based on the first semantic projection include: Step 51, set the maximum number of iterations to , and initialize the iteration number to 1; Step 52, calculate the predicted probability vector at the current iteration number , and the formula is: , wherein, is the semantic prototype matrix of unseen classes; is the semantic projection of some unseen class test samples with high partial confidence under the baseline model; and are both element-wise operations; returns the row vector of the maximum element in each column; returns the row vector of the sum of each column; is a hyperparameter greater than 0; Step 53, calculate the pseudo-labels of the unseen class test samples at the current iteration number , and the formula is: , wherein, represents the vector that returns the index of the maximum element in each column; is the pseudo-label vector of the predicted unseen class samples; Step 54, calculate the probability threshold at the current iteration number , and the formula is: , Step 55, update the prototype according to the threshold , and the formula is: , wherein, represents the column vector that returns the average value of each row; Step 56, perform regularization on , and the formula is: , Step 57, repeat Step 52 to Step 56 until the iteration stops when the maximum number of iterations is reached, and obtain the matrix of well-calibrated pseudo-unseen class semantics.

[0028] Step 6, input the pictures in the training set into the baseline model to obtain the corresponding second semantic projection, and reset the semantic prototype of each seen class based on the second semantic projection.

[0029] Furthermore, the step of resetting the semantic prototype of each seen class based on the second semantic projection includes: Step 61, set the maximum number of iterations to , and initialize the iteration number is 1; Step 62, calculate the predicted probability vector at the current iteration , the formula is: , wherein, is the semantic prototype matrix of the seen classes, is the semantic projection of the seen-class samples under the baseline model; Step 63, calculate the pseudo-labels of the seen-class test samples at the current iteration , the formula is: , wherein, is the pseudo-label vector of the predicted seen-class samples; Step 64, let the probability threshold ; Step 65, update the prototypes of the seen classes according to the threshold , the formula is: , Step 66, perform regularization on , the formula is: , Step 67, repeat Step 62 to Step 66 until the maximum number of iterations is reached and the iteration stops.

[0030] Step 7, use the baseline model to classify the picture to be recognized.

[0031] Furthermore, the steps of using the baseline model to classify the picture to be recognized include: Input the image to be recognized into the baseline model, and calculate the predicted label vector through the following formula: , wherein, represents the calibrated semantic matrix of the image to be recognized, including the calibrated pseudo-unseen-class semantic prototype , and the calibrated seen-class semantic prototype , is the semantic projection of the total sample set under the baseline model; is the pseudo-label vector of the predicted image to be recognized.

[0032] In order to verify the effectiveness of the method proposed by the present invention, a comparative experiment was carried out. In the experiment, three datasets, AWA, CUB, and SUN, were used. The standard split (SS) method was adopted to divide the dataset, and the core evaluation indexes were the accuracy rate S of recognizing the seen-class samples, the accuracy rate U of recognizing the unseen-class samples, and the harmonic mean , and the results are shown in Table 1.

[0033] Table 1

[0034] It can be seen from the experimental results that compared with the traditional generalized zero-shot classification method, the method proposed by the present invention has significantly improved the recognition accuracy, especially in the performance of unseen classes. Specifically, on the AWA2 dataset, most methods show significant differences between seen classes and unseen classes. For example, the accuracy of the DTN method on seen classes is close to 90%, but only 54.8% on unseen classes, while the method described in the present invention enables the recognition accuracy of unseen classes to reach 96.8%, which is close to the accuracy of 97.8% for seen classes, achieving a good balance between seen classes and unseen classes and effectively alleviating the domain shift problem.

[0035] Example Two The multi-view generalized zero-shot classification assistance system based on PU learning described in this example includes: A data preparation module for extracting multi-view visual features of training pictures and test pictures; A model learning module for sending the extracted multi-view visual features into a multi-view PU classification model and training using the alternating direction method of multipliers; using the established multi-view PU classification model to identify test samples and screening out pseudo-unseen class pictures with high confidence; A calibration module for inputting the pseudo-unseen class pictures with high confidence into a benchmark model and resetting the semantic prototypes of each unseen class using the corresponding semantic projections output; then inputting the pictures of the training set into the benchmark model and resetting the semantic prototypes of each seen class using the corresponding semantic projections output; A picture recognition module for classifying the image to be recognized using the benchmark model.

Claims

1. A multi-view generalized zero-shot classification assistance method based on PU learning, characterized in that It includes the following steps: Obtain pictures, construct a training set using pictures of seen classes, construct a test set using pictures of seen classes and pictures of unseen classes, and extract multi-view visual features of the pictures in the training set and the test set; Construct a baseline model; Construct a multi-view PU classification model; Input the multi-view visual features corresponding to the pictures in the training set and the test set into the multi-view PU classification model, train the multi-view PU classification model using the alternating direction method of multipliers, and use the trained multi-view PU classification model to identify test samples, and screen out pseudo-unseen class pictures with high confidence; Input the screened pseudo-unseen class pictures with high confidence into the baseline model, output the corresponding first semantic projection, and reset the semantic prototype of each unseen class based on the first semantic projection; Input the pictures in the training set into the baseline model, obtain the corresponding second semantic projection, and reset the semantic prototype of each seen class based on the second semantic projection; Use the baseline model to classify the pictures to be recognized.

2. The multi-view generalized zero-shot classification assistance method based on PU learning according to claim 1, characterized in that Express the multi-view PU classification model as problem P, and the expression is: Problem P: ， In the formula, represents the view serial number, m represents the total number of views, , and are variables to be optimized, represents the parameters of the multi-view PU classification model under the -th view, represents the negative mean value of the corrected test set features under the -th view, is a slack variable, represents the visual feature matrix of the training samples on the -th view, represents the -th training sample's feature vector under the -th view, represents a real matrix space, where is the number of rows, n is the number of columns, represents the sum of the visual feature vectors of the training samples on the -th view, , represents the transpose; represents the visual feature matrix of the test set samples on the -th view, represents the -th test image's feature vector under the -th view; is a hyperparameter, represents the F norm; , which is the prior probability of the training samples, represents the number of test samples; is the proportion of the seen class samples in the test set among all seen class samples; Construct the constraint conditions of this multi-view PU classification model, which are expressed as: ， ， ， Among them, the first constraint is used to ensure that the classification results between perspectives satisfy complementarity, and the second constraint is used to ensure the robustness of the multi-view PU classification model; Wherein, is the test set, represents the estimated value obtained by the grouped mean median estimator ; represents the -th test image's feature vector at the -th perspective; represents the covariance matrix of the images in the test set on the -th perspective feature, and the formula is: 。 3. The multi-view generalized zero-shot classification assistance method based on PU learning according to claim 2, characterized in that, Obtained by the grouped mean median estimator The steps for the estimated value include: Divide the test set into sample sets of the same group size, denoted as ; Calculate the mean vector of each group of samples , where ; Then calculate the median of each group of samples, and the formula is: , where , which is the Euclidean distance from the i-th group to the j-th group is the median, denotes the median; Find the index of the group with the smallest median distance , then the mean vector of the th group is the desired estimate .

4. The multi-view generalized zero-shot classification assistance method based on PU learning according to claim 3, characterized in that The steps of training the multi-view PU classification model using the alternating direction method of multipliers include: Step 41, initialization , set the maximum number of iterations k max , let the number of iterations , determine the convergence threshold , ; Step 42, construct the augmented Lagrangian function of the multi-view PU classification model, and the expression is: ， Construct constraint conditions: ， ， Among them, is the Lagrange multiplier matrix, is the penalty parameter; Step 43, fix the parameters and , only update , then the multi-view PU classification model is represented as sub-problem p1, and the expression is: Sub-problem p1: ， Set the gradient in sub-problem p1 to 0, solve sub-problem p1, and obtain The update formula is as follows: ， In the formula, I represents the identity matrix; Step 44, fix the parameters and , only update , then the multi-view PU classification model is represented as sub-problem p2, and the expression is: Sub-problem p2: ， ， Set the gradient in sub-problem p2 to 0 and solve sub-problem p2 to obtain The update formula is as follows: ， Step 45, fix parameters and , only update , then the multi-view PU classification model is represented as sub-problem p3, and the expression is: Sub-problem p3: ， Constraint conditions: ， According to the duality theory, by constructing the Lagrangian function and solving the corresponding dual problem, we obtain The update formula is as follows: ， Step 46, update by the following formula :[[]]END]] ， Step 47: Determine whether the convergence condition is satisfied. Stop the iteration until the convergence condition is satisfied or the maximum number of iterations is reached; otherwise, update the number of iterations , and return to Step 43; Among them, the convergence condition is that the difference between two adjacent iterations is less than the set threshold, which is expressed as: ， ， ， 。 5. The multi-view generalized zero-shot classification assistance method based on PU learning according to claim 4, characterized in that The steps of screening out pseudo-unseen class pictures with high confidence include: Calculate the confidence of the test set images , and the calculation formula is as follows: ， Arrange all test set pictures in ascending order according to the confidence value, and screen out the samples with confidence values less than the 50th percentile from the sorted picture samples as pseudo-unseen class pictures with high confidence.

6. The multi-view generalized zero-shot classification assistance method based on PU learning according to claim 5, characterized in that And the steps of resetting the semantic prototype of each unseen class based on the first semantic projection include: Step 51, set the maximum number of iterations to , and initialize the iteration count to 1; Step 52, calculate the predicted probability vector at the current iteration number , and the formula is: ， In the formula, is the semantic prototype matrix of unseen classes; is the semantic projection of some unseen class test samples with high partial confidence under the baseline model; and are both element-wise operations; returns the row vector of the maximum elements of each column; returns the row vector of the sum of each column; is a hyperparameter greater than 0; Step 53, calculate the pseudo-labels of the unseen class test samples at the current iteration number , and the formula is: ， Among them, represents a vector that returns the indices of the maximum elements in each column; is the pseudo-label vector of the predicted test set; Step 54, calculate the probability threshold at the current iteration number , and the formula is: ， Step 55, update the prototype according to the threshold , the formula is: ， Among them, represents the column vector of the average value returned for each row; Step 56, for perform regularization, and the formula is: ， Step 57, repeat steps 52 to 56 until the maximum number of iterations is reached and the iteration stops, and obtain the calibrated pseudo-unseen class semantics.

7. The multi-view generalized zero-shot classification assistance method based on PU learning according to claim 6, characterized in that The steps of using the baseline model to classify the pictures to be recognized include: Input the image to be recognized into the baseline model, and calculate the predicted label vector through the following formula: ， Among them, represents the calibrated semantic matrix of the image to be recognized, is the semantic projection of the total sample set under the benchmark model; returns a vector of the indices of the maximum elements in each column; is the pseudo-label vector of the image to be recognized predicted.

8. Multi-view generalized zero-shot classification assistance system based on PU learning, characterized in that Include: A data preparation module for extracting multi-view visual features of training pictures and test pictures; A model learning module for sending the extracted multi-view visual features into the multi-view PU classification model and training using the alternating direction method of multipliers; Use the established multi-view PU classification model to identify test samples and screen out pseudo-unseen class pictures with high confidence; A calibration module for inputting the screened pseudo-unseen class pictures with high confidence into the baseline model, and resetting the semantic prototype of each unseen class using the corresponding semantic projection output; then input the pictures in the training set into the baseline model, and reset the semantic prototype of each seen class using the corresponding semantic projection output; The image recognition module is used to classify the image to be recognized by using the benchmark model.

Images