Multi-view generalized zero-shot classification auxiliary method and device based on pu learning

Abstract

The application discloses a multi-view generalized zero-shot classification auxiliary method and device based on PU learning, and belongs to the technical field of picture classification. The method comprises the following steps: acquiring pictures, extracting multi-view visual features of the pictures in a training set and a test set; constructing a benchmark model; constructing a multi-view PU classification model; inputting the multi-view visual features corresponding to the pictures in the training set and the test set into the multi-view PU classification model, and training the multi-view PU classification model by using an alternating direction multiplier method; resetting semantic prototypes of each unseen class based on a first semantic projection; resetting semantic prototypes of each seen class based on a second semantic projection; and classifying pictures to be identified by using the benchmark model. The application separates unseen class samples from the test samples to be identified by using a PU learning framework, and uses the unseen class samples for class semantic calibration, so that the bias of the model to the seen class samples can be effectively relieved, and thus the generalization performance of the generalized zero-shot classifier is significantly improved.

Description

Technical Field

[0001] This invention relates to a zero-shot classification aid method, specifically a multi-view generalized zero-shot classification aid method and device based on PU learning, belonging to the field of image classification technology. Background Technology

[0002] In recent years, multi-view visual feature fusion methods have made significant progress in zero-shot learning (ZSL). In practical applications, visual representations of images from multiple perspectives can be obtained by using different feature extractors. Due to the heterogeneity between features from different perspectives, visual data based on multi-view feature representations can provide more comprehensive visual information about the image. Related research results show that fusing multi-view visual information using appropriate methods can significantly improve the performance of ZSL learning. However, multi-view fusion may further exacerbate the domain drift problem, resulting in poor model performance in generalized zero-shot learning (GZSL) tasks. Therefore, most existing methods introduce calibration biases to balance the decision boundaries between seen and unseen classes, which alleviates the domain drift problem to some extent, but the improvement effect is still relatively limited. If seen and unseen samples can be identified in the samples to be predicted, the above problem can be easily solved. Therefore, a new calibration strategy is urgently needed to identify seen and unseen samples in the samples to be predicted in order to improve the generalization performance of generalized zero-shot classifiers. Summary of the Invention

[0003] Purpose of the invention: To address the above problems, the purpose of this invention is to provide a multi-view generalized zero-shot classification auxiliary method and device based on PU learning. This method uses multi-view visual features of image samples to train the classifier, thereby improving the generalization performance of the classifier and achieving more accurate image recognition.

[0004] Technical solution: One aspect of the present invention provides a multi-view generalized zero-shot classification auxiliary method based on PU learning, characterized by comprising the following steps:

[0005] Acquire images, construct a training set using seen images, construct a test set using both seen and unseen images, and extract multi-view visual features from images in the training and test sets.

[0006] Build a benchmark model;

[0007] Construct a multi-view PU classification model;

[0008] The multi-view visual features corresponding to the images in the training set and the test set are input into the multi-view PU classification model. The multi-view PU classification model is trained using the alternating direction multiplier method. The trained multi-view PU classification model is then used to identify the test samples and filter out pseudo-unseen images with high confidence.

[0009] The selected high-confidence pseudo-unseen class images are input into the baseline model, the corresponding first semantic projection is output, and the semantic prototype of each unseen class is reset based on the first semantic projection.

[0010] The images in the training set are input into the baseline model to obtain the corresponding second semantic projection, and the semantic prototype of each seen class is reset based on the second semantic projection.

[0011] The baseline model is used to classify the images to be identified.

[0012] Furthermore, the multi-view PU classification model is represented as problem P, with the expression:

[0013] Question P:

[0014] ,

[0015] In the formula, This indicates the viewpoint number, and m represents the total number of viewpoints. , and For the variable to be optimized, Indicates the first Parameters of a multi-view PU classification model from multiple perspectives Indicates the first The negative mean of the corrected test set features from each perspective. As slack variables, Indicates the training sample at the th Visual feature matrix from each viewpoint Indicates the first The training sample at the th ... Feature vectors from various perspectives Represent a The space of real matrices, where n is the number of rows, and n is the number of columns. Indicates the training sample at the th The sum of visual feature vectors from each viewpoint , Indicates transpose; Indicates the test set sample at the th Visual feature matrix from each viewpoint Indicates the first The test image is in the first... Feature vectors from various perspectives; For hyperparameters, Denotes the F-norm; , is the prior probability of the training sample. Indicates the number of test samples; It represents the proportion of seen class samples in the test set out of all seen class samples;

[0016] The constraints for constructing this multi-view PU classification model are expressed as follows:

[0017] ,

[0018] ,

[0019] ,

[0020] The first constraint is used to ensure that the classification results between perspectives are complementary, and the second constraint is used to ensure the robustness of the multi-view PU classification model.

[0021] In the formula, For the test set, This indicates that the mean and median are obtained through a grouping mean and median estimator. The estimated value, Indicates the first The test image is in the first... Feature vectors from various perspectives; This indicates that the image in the test set is in the [number]th [position]. The covariance matrix for each viewpoint feature is given by the formula:

[0022] .

[0023] Furthermore, the group mean-median estimator is used to obtain... The steps for estimating the value include:

[0024] test set Divided into A set of samples of the same size is denoted as ;

[0025] Calculate the mean vector for each sample group. ,in ;

[0026] Next, calculate the median of each sample group using the following formula:

[0027] ,in That is, the Euclidean distance from group i to group j. the median of This represents the median;

[0028] Find the index of the group with the minimum median distance Then the first The mean vector of the group is what we are looking for. The estimated value .

[0029] Furthermore, the steps for training the multi-view PU classification model using the alternating direction multiplier method include:

[0030] Step 41, Initialization Set the maximum number of iterations k max Let the number of iterations be... Determine the convergence threshold , ;

[0031] Step 42, construct the augmented Lagrangian function of the multi-view PU classification model, the expression of which is:

[0032] ,

[0033] Construct constraints:

[0034] ,

[0035] ,

[0036] in, It is a Lagrange multiplier matrix. It is a penalty parameter;

[0037] Step 43, fix parameters and Update only The multi-view PU classification model is then represented as a subproblem p1, expressed as:

[0038] Subproblem p1:

[0039] ,

[0040] Setting the gradient of subproblem p1 to 0, we solve subproblem p1 and obtain... The update formula is:

[0041] ,

[0042] In the formula, I Represents the identity matrix;

[0043] Step 44, fix parameters and Update only The multi-view PU classification model can then be represented as a subproblem p2, expressed as:

[0044] Subproblem p2:

[0045] ,

[0046] ,

[0047] Setting the gradient of subproblem p2 to 0, and solving subproblem p2, we get... The update formula is:

[0048] ,

[0049] Step 45, fix parameters and Update only The multi-view PU classification model is then represented as subproblem p3, expressed as:

[0050] Subproblem p3:

[0051] ,

[0052] Constraints:

[0053] ,

[0054] According to duality theory, by constructing the Lagrangian function and solving the corresponding dual problem, we obtain... The updated formula is:

[0055] ,

[0056] Step 46, update using the following formula :

[0057] ,

[0058] Step 47: Determine if the convergence condition is met, and stop iterating until the convergence condition is met or the maximum number of iterations is reached; otherwise, update the number of iterations. and return to step 43;

[0059] The convergence condition is that the difference between the changes in two adjacent iterations is less than a set threshold, expressed as:

[0060] ,

[0061] ,

[0062] ,

[0063] .

[0064] Furthermore, the steps for filtering out high-confidence pseudo-unseen images include:

[0065] Calculate the confidence score of the test set images. The calculation formula is:

[0066] ,

[0067] All test set images are sorted in ascending order based on confidence scores. From the sorted image samples, samples with confidence scores less than the 50th percentile are selected as high-confidence pseudo-unseen images.

[0068] Furthermore, the step of resetting the semantic prototype of each unseen class based on the first semantic projection includes:

[0069] Step 51, set the maximum number of iterations to... Initialize the number of iterations =1;

[0070] Step 52: Calculate the prediction probability vector for the current iteration number. The formula is:

[0071] ,

[0072] In the formula, It is an unseen class semantic prototype matrix; It is the semantic projection of some high-confidence unseen class test samples onto the benchmark model; and All are element-wise operations; Returns the row vector of the largest element in each column; Returns the row vector of the sum of each column; Hyperparameters that are greater than 0;

[0073] Step 53: Calculate the pseudo-labels of the unseen class test samples at the current iteration number. The formula is:

[0074] ,

[0075] in, This represents a vector that returns the index of the largest element in each column; It is the predicted pseudo-label vector of the test set;

[0076] Step 54: Calculate the probability threshold for the current iteration number. The formula is:

[0077] ,

[0078] Step 55: Update the prototype based on the threshold. The formula is:

[0079] ,

[0080] in, This represents a column vector that returns the average value of each row.

[0081] Step 56, for To perform regularization, the formula is:

[0082] ,

[0083] Step 57: Repeat steps 52 to 56 until the maximum number of iterations is reached, then stop iterating to obtain the calibrated pseudo-unseen class semantics.

[0084] Furthermore, the steps for classifying the image to be identified using the baseline model include:

[0085] The image to be identified is input into the baseline model, and the predicted label vector is calculated using the following formula:

[0086] ,

[0087] in, This represents the calibrated semantic matrix of the image to be recognized. This represents the semantic projection of the total sample set onto the baseline model. Returns a vector of the indices of the largest element in each column; It is the predicted pseudo-label vector of the image to be identified.

[0088] Another aspect of the present invention provides a multi-view generalized zero-shot classification auxiliary system based on PU learning, comprising:

[0089] The data preparation module is used to extract multi-view visual features from training and testing images;

[0090] The model learning module is used to feed the extracted multi-view visual features into the multi-view PU classification model and train it using the alternating direction multiplier method. The established multi-view PU classification model is used to identify test samples and filter out pseudo-unseen images with high confidence.

[0091] The calibration module is used to input the pseudo-unseen class images with high confidence into the baseline model, and reset the semantic prototype of each unseen class using the corresponding semantic projection of the output; then, the images of the training set are input into the baseline model, and the semantic prototype of each seen class is reset using the corresponding semantic projection of the output.

[0092] The image recognition module is used to classify the images to be recognized using a benchmark model.

[0093] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0094] This invention constructs a binary classification model for seen and unseen classes applicable to multi-view visual features. Based on the established benchmark model, the semantic prototypes of unseen classes are calibrated using test samples with high confidence in some unseen classes, and the semantic prototypes of seen classes are calibrated using samples from the training set. This invention utilizes the PU learning framework to separate unseen class samples from the test samples to be identified and uses them for category semantic calibration, which can effectively alleviate the bias of the model towards seen class samples, thereby significantly improving the generalization performance of the generalized zero-shot classifier. Attached Figure Description

[0095] Figure 1 A flowchart of a multi-view generalized zero-shot classification auxiliary method based on PU learning;

[0096] Figure 2 This is a flowchart of a multi-view generalized zero-shot classification auxiliary method based on PU learning. Detailed Implementation

[0097] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0098] Example 1

[0099] Positive and Unlabeled Learning (PU) is a binary classification method for weakly labeled scenarios, capable of training using only positive and unlabeled samples when negative samples are unavailable. For generalized zero-shot learning, if seen-class samples are considered positive and unseen-class samples are considered negative, the training samples consist only of positive and unlabeled samples, perfectly aligning with the PU modeling paradigm. Here, positive samples represent seen-class samples with known labels, and unlabeled samples are samples to be predicted, encompassing both seen and unseen classes. Therefore, this example proposes a novel multi-view generalized zero-shot classification auxiliary method based on PU learning to improve the generalization performance of the generalized zero-shot classifier.

[0100] Combination Figure 1 and Figure 2 The multi-view generalized zero-shot classification auxiliary method based on PU learning described in this embodiment includes the following steps:

[0101] Step 1: Obtain images, construct a training set using seen images, construct a test set using both seen and unseen images, and extract multi-view visual features from the images in the training and test sets.

[0102] Obtain images to be detected, including seen and unseen images. Construct a training set using the seen images and a test set using the seen and unseen images. Use the GoogLeNet deep convolutional neural network model, pre-trained on the ImageNet database, to extract multi-visual features from the images in the training set, such as... Figure 2 In this study, View 1, View 2, and View n represent n different visual features, and two different scaling ratios, 0.1 and 1, are used to represent local A and global view B, respectively. The 1024-dimensional vector of the top-level hidden unit of GoogLeNet is used as the visual feature.

[0103] Step 2, construct the baseline model.

[0104] In this example, the multi-view visual-semantic mapping model in an instance-based multi-view visual fusion transduction zero-shot classification method disclosed in patent CN117541882B is used as the benchmark model after being trained.

[0105] Step 3: Construct a multi-view PU classification model.

[0106] Furthermore, the multi-view PU classification model is represented as problem P, with the expression:

[0107] Question P:

[0108] ,

[0109] In the formula, This indicates the viewpoint number, and m represents the total number of viewpoints. , and For the variable to be optimized, Indicates the first Parameters of a multi-view PU classification model from multiple perspectives Indicates the first The negative mean of the corrected test set features from each perspective. As slack variables, Indicates the training sample at the th Visual feature matrix from each viewpoint Indicates the first The training sample at the th ... Feature vectors from various perspectives Represent a The space of real matrices, where n is the number of rows, and n is the number of columns. Indicates the training sample at the th The sum of visual feature vectors from each viewpoint , Indicates transpose; Indicates the test set sample at the th Visual feature matrix from each viewpoint Indicates the first The test image is in the first... Feature vectors from various perspectives; For hyperparameters, Denotes the F-norm; , is the prior probability of the training sample. This indicates the number of training samples, the same as the number of columns mentioned above. Indicates the number of test samples; It represents the proportion of seen class samples in the test set out of all seen class samples;

[0110] The constraints for constructing this multi-view PU classification model are expressed as follows:

[0111] ,

[0112] ,

[0113] ,

[0114] The first constraint is used to ensure that the classification results between perspectives are complementary, and the second constraint is used to ensure the robustness of the multi-view PU classification model.

[0115] In the formula, For the test set, This indicates that the mean and median are obtained through a grouping mean and median estimator. The estimated value, Indicates the first The test image is in the first... Feature vectors from various perspectives; This indicates that the image in the test set is in the [number]th [position]. The covariance matrix for each viewpoint feature is given by the formula:

[0116] .

[0117] Furthermore, the group mean-median estimator is used to obtain... The steps for estimating the value include:

[0118] test set Divided into A set of samples of the same size is denoted as ;

[0119] Calculate the mean vector for each sample group. ,in ;

[0120] Next, calculate the median of each sample group using the following formula:

[0121] ,in That is, the Euclidean distance from group i to group j. the median of This represents the median;

[0122] Find the index of the group with the minimum median distance Then the first The mean vector of the group is what we are looking for. The estimated value .

[0123] Step 4: Input the multi-view visual features corresponding to the images in the training set and the test set into the multi-view PU classification model, train the multi-view PU classification model using the alternating direction multiplier method, and use the trained multi-view PU classification model to identify the test samples and filter out pseudo-unseen images with high confidence.

[0124] Furthermore, the steps for training the multi-view PU classification model using the alternating direction multiplier method include:

[0125] Step 41, Initialization Set the maximum number of iterations k max Let the number of iterations be... Determine the convergence threshold , ;

[0126] Step 42, construct the augmented Lagrangian function of the multi-view PU classification model, the expression of which is:

[0127] ,

[0128] Construct constraints:

[0129] ,

[0130] ,

[0131] in, It is a Lagrange multiplier matrix. It is a penalty parameter;

[0132] Step 43, fix parameters and Update only The multi-view PU classification model is then represented as a subproblem p1, expressed as:

[0133] Subproblem p1:

[0134] ,

[0135] Setting the gradient of subproblem p1 to 0, we solve subproblem p1 and obtain... The update formula is:

[0136] ,

[0137] In the formula, I Represents the identity matrix;

[0138] Step 44, fix parameters and Update only The multi-view PU classification model can then be represented as a subproblem p2, expressed as:

[0139] Subproblem p2:

[0140] ,

[0141] ,

[0142] Setting the gradient of subproblem p2 to 0, and solving subproblem p2, we get... The update formula is:

[0143] ,

[0144] Step 45, fix parameters and Update only The multi-view PU classification model is then represented as subproblem p3, expressed as:

[0145] Subproblem p3:

[0146] ,

[0147] Constraints:

[0148] ,

[0149] According to duality theory, by constructing the Lagrangian function and solving the corresponding dual problem, we obtain... The updated formula is:

[0150] ,

[0151] Step 46, update using the following formula :

[0152] ,

[0153] Step 47: Determine if the convergence condition is met, and stop iterating until the convergence condition is met or the maximum number of iterations is reached; otherwise, update the number of iterations. and return to step 43;

[0154] The convergence condition is that the difference between the changes in two adjacent iterations is less than a set threshold, expressed as:

[0155] ,

[0156] ,

[0157] ,

[0158] .

[0159] Furthermore, the steps for filtering out high-confidence pseudo-unseen images include:

[0160] Calculate the confidence score of the test set images. The calculation formula is:

[0161] ,

[0162] All test set images are sorted in ascending order based on confidence scores. From the sorted image samples, samples with confidence scores less than the 50th percentile are selected as high-confidence pseudo-unseen images.

[0163] Step 5: Input the selected high-confidence pseudo-unseen class images into the baseline model, output the corresponding first semantic projection, and reset the semantic prototype of each unseen class based on the first semantic projection.

[0164] Furthermore, the step of resetting the semantic prototype of each unseen class based on the first semantic projection includes:

[0165] Step 51, set the maximum number of iterations to... Initialize the number of iterations =1;

[0166] Step 52: Calculate the prediction probability vector for the current iteration number. The formula is:

[0167] ,

[0168] In the formula, It is an unseen class semantic prototype matrix; It is the semantic projection of some high-confidence unseen class test samples onto the benchmark model; and All are element-wise operations; Returns the row vector of the largest element in each column; Returns the row vector of the sum of each column; Hyperparameters that are greater than 0;

[0169] Step 53: Calculate the pseudo-labels of the unseen class test samples at the current iteration number. The formula is:

[0170] ,

[0171] in, This represents a vector that returns the index of the largest element in each column; It is the predicted pseudo-label vector of unseen class samples;

[0172] Step 54: Calculate the probability threshold for the current iteration number. The formula is:

[0173] ,

[0174] Step 55: Update the prototype based on the threshold. The formula is:

[0175] ,

[0176] in, This represents a column vector that returns the average value of each row.

[0177] Step 56, for To perform regularization, the formula is:

[0178] ,

[0179] Step 57: Repeat steps 52 to 56 until the maximum number of iterations is reached, then stop iterating to obtain the matrix of pseudo-unseen semantic classes after good calibration.

[0180] Step 6: Input the images in the training set into the baseline model to obtain the corresponding second semantic projection, and reset the semantic prototype of each seen class based on the second semantic projection.

[0181] Furthermore, the step of resetting the semantic prototype of each seen class based on the second semantic projection includes:

[0182] Step 61, set the maximum number of iterations to Initialize the number of iterations =1;

[0183] Step 62, calculate the prediction probability vector for the current iteration number. The formula is:

[0184] ,

[0185] in, It is the semantic prototype matrix of the known classes. It is the semantic projection of seen class samples under the benchmark model;

[0186] Step 63: Calculate the pseudo-labels of the seen class test samples at the current iteration number. The formula is:

[0187] ,

[0188] in, It is the predicted pseudo-label vector of the seen class samples;

[0189] Step 64, set the probability threshold. ;

[0190] Step 65: Update the prototype of the seen classes according to the threshold. The formula is:

[0191] ,

[0192] Step 66, for To perform regularization, the formula is: ,

[0193] Step 67: Repeat steps 62 to 66 until the maximum number of iterations is reached and then stop iterating.

[0194] Step 7: Classify the images to be identified using a baseline model.

[0195] Furthermore, the steps for classifying the image to be identified using the baseline model include:

[0196] The image to be identified is input into the baseline model, and the predicted label vector is calculated using the following formula:

[0197] ,

[0198] in, This represents the calibrated semantic matrix of the image to be identified, including the calibrated pseudo-unseen class semantic prototype. and the calibrated known semantic prototype , This represents the semantic projection of the total sample set onto the baseline model. It is the predicted pseudo-label vector of the image to be identified.

[0199] To verify the effectiveness of the proposed method, comparative experiments were conducted. The experiments used three datasets: AWA, CUB, and SUN. The datasets were partitioned using the standard split (SS) method. The core evaluation metrics were the accuracy S for identifying seen class samples, the accuracy U for identifying unseen class samples, and the harmonic mean. The results are shown in Table 1.

[0200] Table 1

[0201]

[0202] Experimental results show that, compared with traditional generalized zero-shot classification methods, the method proposed in this invention significantly improves recognition accuracy, especially in unseen categories. Specifically, on the AWA2 dataset, most methods exhibit significant differences between seen and unseen categories. For example, the DTN method achieves an accuracy of nearly 90% for seen categories but only 54.8% for unseen categories, while the method described in this invention achieves a recognition accuracy of 96.8% for unseen categories, close to the 97.8% accuracy for seen categories, achieving a good balance between seen and unseen categories and effectively mitigating the neighborhood bias problem.

[0203] Example 2

[0204] The multi-view generalized zero-shot classification auxiliary system based on PU learning described in this embodiment includes:

[0205] The data preparation module is used to extract multi-view visual features from training and testing images;

[0206] The model learning module is used to feed the extracted multi-view visual features into the multi-view PU classification model and train it using the alternating direction multiplier method. The established multi-view PU classification model is used to identify test samples and filter out pseudo-unseen images with high confidence.

[0207] The calibration module is used to input the pseudo-unseen class images with high confidence into the baseline model, and reset the semantic prototype of each unseen class using the corresponding semantic projection of the output; then, the images of the training set are input into the baseline model, and the semantic prototype of each seen class is reset using the corresponding semantic projection of the output.

[0208] The image recognition module is used to classify the images to be recognized using a benchmark model.

Claims

1. A multi-view generalized zero-shot classification auxiliary method based on PU learning, characterized in that, Includes the following steps: Images are acquired, a training set is constructed using seen images, and a test set is constructed using both seen and unseen images. Multi-view visual features are extracted from images in the training and test sets. Among these, the GoogLeNet deep convolutional neural network model, pre-trained on the ImageNet database, is used to extract multi-view visual features from images in the training set. A baseline model is constructed, which is obtained by training a multi-view visual-semantic mapping model in an instance-based multi-view visual fusion transduction zero-shot classification method. Construct a multi-view PU classification model; The multi-view visual features corresponding to the images in the training set and the test set are input into the multi-view PU classification model. The multi-view PU classification model is trained using the alternating direction multiplier method. The trained multi-view PU classification model is then used to identify the test samples and filter out pseudo-unseen images with high confidence. The selected high-confidence pseudo-unseen class images are input into the baseline model, the corresponding first semantic projection is output, and the semantic prototype of each unseen class is reset based on the first semantic projection. The images in the training set are input into the baseline model to obtain the corresponding second semantic projection, and the semantic prototype of each seen class is reset based on the second semantic projection. The baseline model is used to classify the images to be identified; The multi-view PU classification model is represented as problem P, and its expression is: Question P: ， In the formula, This indicates the viewpoint number, and m represents the total number of viewpoints. , and For the variable to be optimized, Indicates the first Parameters of a multi-view PU classification model from multiple perspectives Indicates the first The negative mean of the corrected test set features from each perspective. As slack variables, Indicates the training sample at the th Visual feature matrix from each viewpoint Indicates the first The training sample at the th ... Feature vectors from various perspectives Represent a The space of real matrices, where n is the number of rows, and n is the number of columns. Indicates the training sample at the th The sum of visual feature vectors from each viewpoint , Indicates transpose; Indicates the test set sample at the th Visual feature matrix from each viewpoint Indicates the first The test image is in the first... Feature vectors from various perspectives; For hyperparameters, Denotes the F-norm; , is the prior probability of the training sample. This indicates the number of training samples, the same as the column numbers mentioned above. Indicates the number of test samples; It represents the proportion of seen class samples in the test set out of all seen class samples; The constraints for constructing this multi-view PU classification model are expressed as follows: ， ， ， The first constraint is used to ensure that the classification results between perspectives are complementary, and the second constraint is used to ensure the robustness of the multi-view PU classification model. In the formula, For hyperparameters, For the test set, This indicates that the mean and median are obtained through a grouping mean and median estimator. The estimated value; This indicates that the image in the test set is in the [number]th [position]. The covariance matrix for each viewpoint feature is given by the formula: ； The steps for training a multi-view PU classification model using the alternating direction multiplier method include: Step 41, Initialization Set the maximum number of iterations k max Let the number of iterations be... Determine the convergence threshold , ; Step 42, construct the augmented Lagrangian function of the multi-view PU classification model, the expression of which is: ， Construct constraints: ， ， in, It is a Lagrange multiplier matrix. It is a penalty parameter; Step 43, fix parameters and Update only The multi-view PU classification model is then represented as a subproblem p1, expressed as: Subproblem p1: ， Setting the gradient of subproblem p1 to 0, we solve subproblem p1 and obtain... The update formula is: ， In the formula, I represents the identity matrix; Step 44, fix parameters and Update only The multi-view PU classification model can then be represented as a subproblem p2, expressed as: Subproblem p2: ， ， Setting the gradient of subproblem p2 to 0, and solving subproblem p2, we get... The update formula is: ， Step 45, fix parameters and Update only The multi-view PU classification model is then represented as subproblem p3, expressed as: Subproblem p3: ， Constraints: ， According to duality theory, by constructing the Lagrangian function and solving the corresponding dual problem, we obtain... The updated formula is: ， Step 46, update using the following formula : ， Step 47: Determine if the convergence condition is met, and stop iterating until the convergence condition is met or the maximum number of iterations is reached; otherwise, update the number of iterations. and return to step 43; The convergence condition is that the difference between the changes in two adjacent iterations is less than a set threshold, expressed as: ， ， ， 。 2. The multi-view generalized zero-shot classification auxiliary method based on PU learning according to claim 1, characterized in that, Obtained through the grouped mean and median estimator The steps for estimating the value include: test set Divided into A set of samples of the same size is denoted as ; Calculate the mean vector for each sample group. ; Next, calculate the median of each sample group using the following formula: ,in , That is, the Euclidean distance from group i to group j. the median of This represents the median; Find the index of the group with the minimum median distance Then the first The mean vector of the group is what we are looking for. The estimated value .

3. The multi-view generalized zero-shot classification auxiliary method based on PU learning according to claim 2, characterized in that, The steps to filter out high-confidence pseudo-unseen images include: Calculate the confidence score of the test set images. The calculation formula is: ， All test set images are sorted in ascending order based on confidence scores. From the sorted image samples, samples with confidence scores less than the 50th percentile are selected as high-confidence pseudo-unseen images.

4. The multi-view generalized zero-shot classification auxiliary method based on PU learning according to claim 3, characterized in that, The steps of resetting the semantic prototype of each unseen class based on the first semantic projection include: Step 51, set the maximum number of iterations to... Initialize the number of iterations =1; Step 52, calculate the prediction probability vector for the current iteration number. The formula is: ， In the formula, It is an unseen class semantic prototype matrix; It is the semantic projection of some high-confidence unseen class test samples onto the benchmark model; and All are element-wise operations; Returns the row vector of the largest element in each column; Returns the row vector of the sum of each column; Step 53: Calculate the pseudo-labels of the unseen class test samples at the current iteration number. The formula is: ， in, This represents a vector that returns the index of the largest element in each column; It is the predicted pseudo-label vector of the test set; Step 54: Calculate the probability threshold for the current iteration number. The formula is: ， Step 55: Update the prototype based on the threshold. The formula is: ， in, This represents a column vector that returns the average value of each row. Step 56, for To perform regularization, the formula is: ， Step 57: Repeat steps 52 to 56 until the maximum number of iterations is reached, then stop iterating to obtain the calibrated pseudo-unseen class semantics.

5. The multi-view generalized zero-shot classification auxiliary method based on PU learning according to claim 4, characterized in that, The steps for classifying the image to be identified using a baseline model include: The image to be identified is input into the baseline model, and the predicted label vector is calculated using the following formula: ， in, This represents the calibrated semantic matrix of the image to be recognized. This represents the semantic projection of the total sample set onto the baseline model. Returns a vector of the indices of the largest element in each column; It is the predicted pseudo-label vector of the image to be identified.

6. A multi-view generalized zero-shot classification auxiliary system based on PU learning, applied to the multi-view generalized zero-shot classification auxiliary method based on PU learning as described in any one of claims 1 to 5, characterized in that, include: The data preparation module is used to extract multi-view visual features from training and testing images; The model learning module is used to feed the extracted multi-view visual features into the multi-view PU classification model and train it using the alternating direction multiplier method. The established multi-view PU classification model was used to identify the test samples and filter out pseudo-unseen class images with high confidence. The calibration module is used to input the pseudo-unseen class images with high confidence into the baseline model, and reset the semantic prototype of each unseen class using the corresponding semantic projection of the output; then, the images of the training set are input into the baseline model, and the semantic prototype of each seen class is reset using the corresponding semantic projection of the output. The image recognition module is used to classify the images to be recognized using a benchmark model.

Images