A multi-view image classification method based on a progressive semantic binding mechanism

Abstract

The application belongs to the technical field of image classification, and particularly relates to a multi-view image classification method based on a progressive semantic binding mechanism; comprising: acquiring a multi-view image and a category label; pre-processing the multi-view image to generate a high-dimensional feature vector; processing the high-dimensional feature vector by using an encoder to obtain a latent feature representation; constructing a progressive semantic binding loss and a learning reinforcement discriminant loss according to the latent feature representation; inputting the latent feature representation into a decoder to generate a reconstructed high-dimensional feature vector and calculate a reconstruction loss; inputting the latent feature representation into a classifier to obtain a classification result of the multi-view image; calculating a cross-entropy loss according to the classification result and the category label; fusing each loss to obtain a total loss of the model; adjusting model parameters according to the total loss of the model to obtain a trained model; the application guarantees the semantic consistency of multi-view data at different levels, effectively enhances the difference between features of different categories, and significantly improves the precision and robustness of multi-view data alignment and fusion.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology, specifically relating to a multi-view image classification method based on a progressive semantic binding mechanism. Background Technology

[0002] Multi-view learning typically enhances a model's ability to represent and discriminate complex data distributions by incorporating information from different modalities or perspectives. This technique is widely used in various artificial intelligence fields such as image recognition, video analysis, semantic segmentation, and cross-modal retrieval, and has received considerable attention in recent years.

[0003] Existing multi-view learning methods primarily integrate information from various views into a shared space through feature alignment and fusion strategies. However, multi-view data often suffers from modal differences, semantic inconsistencies, and statistical distribution shifts, making it difficult to achieve complete structural alignment of features across different views. This leads to uncertainty in the fused representation and a decrease in discriminative power. Furthermore, most methods lack discriminative constraints during feature fusion, making it difficult to simultaneously ensure intra-class consistency and inter-class separability, thus affecting the final model's discriminative performance.

[0004] Specifically, existing feature alignment mechanisms often suffer from excessively strong constraints on the consistency of features of similar samples, leading to over-aggregation of multiple view features in a shared space and causing a "mode collapse" phenomenon. This problem manifests as insufficient diversity, a tendency for sample features to cluster together, loss of original semantic differences, and consequently weakened model capabilities in characterizing fine-grained semantic structures. This collapse is particularly pronounced when using contrastive learning, center loss, or similarity maximization methods, especially in scenarios with blurred positive-negative sample boundaries or class imbalance, severely impacting model generalization ability and stability.

[0005] Furthermore, multi-view feature fusion strategies often rely on static or one-time mapping operations, failing to achieve layer-by-layer optimization of the feature space and gradual enhancement of semantic alignment. They lack structural adaptability and discriminative driving capabilities, easily leading to problems such as redundant fusion feature dimensions, unclear inter-class boundaries, and semantic drift. Simultaneously, when modeling features of different categories, existing technologies largely fail to effectively utilize the structural relationship between intra-class compactness and inter-class separability, lacking fine-grained modeling of class centers or neighborhood structures, thus limiting the full mining and utilization of multi-view information in complex task scenarios.

[0006] In summary, existing multi-view learning methods still have the following key problems in the feature alignment and fusion process:

[0007] (1) The lack of joint modeling of intra-class consistency and inter-class separation leads to insufficient discrimination ability;

[0008] (2) Pattern collapse is prone to occur during feature alignment, which affects the generalization performance and semantic preservation ability of the model.

[0009] (3) The integration mechanism is static and simple, lacking progressive and structure-aware dynamic optimization strategies;

[0010] (4) The modeling of inter-class relationships is crude and lacks an adaptive boundary enhancement mechanism driven by the class center.

[0011] Therefore, there is an urgent need for a discriminative learning-driven progressive semantic binding mechanism to alleviate the above-mentioned problems in current multi-view learning. By gradually optimizing the feature alignment process, designing structure-aware discriminative constraints, and implementing an adaptive class boundary enhancement strategy, the discriminativeness and robustness of multi-view feature fusion can be fundamentally improved, thereby achieving a more efficient, stable, and generalizable multi-view representation learning framework. Summary of the Invention

[0012] To address the shortcomings of existing technologies, this invention proposes a multi-view image classification method based on a progressive semantic binding mechanism. The method includes: acquiring multi-view images and inputting them into a trained multi-view image classification model to obtain the classification results of the multi-view images.

[0013] The training process for a multi-view image classification model includes:

[0014] S1: Obtain multi-view images and category labels; preprocess the multi-view images to generate high-dimensional feature vectors;

[0015] S2: The encoder is used to process the high-dimensional feature vector to obtain the latent feature representation of the image;

[0016] S3: Construct progressive semantic binding loss and learn reinforcement discrimination loss based on latent feature representation; wherein the progressive semantic binding loss includes category semantic contrast binding loss, sample semantic contrast binding loss and feature semantic contrast binding loss;

[0017] S4: Input the latent feature representation into the decoder to generate a reconstructed high-dimensional feature vector; calculate the reconstruction loss based on the high-dimensional feature vector and the reconstructed high-dimensional feature vector.

[0018] S5: Input the latent feature representation into the classifier to obtain the classification result of the multi-view image; calculate the cross-entropy loss based on the classification result of the multi-view image and the category label;

[0019] S6: Calculate the total model loss based on progressive semantic binding loss, learning reinforcement discrimination loss, reconstruction loss, and cross-entropy loss; adjust the model parameters based on the total model loss to obtain the trained multi-view image classification model.

[0020] Preferably, the process of constructing the progressive semantic binding loss includes:

[0021] Calculate the sample center features for each category; construct the category semantic contrast binding loss based on the similarity between the latent feature representation of the image and the sample center features of each category;

[0022] The sample semantic contrast binding loss is constructed based on the ratio between the similarity of images of the same type and the similarity of images of different types.

[0023] Generate a multi-view centered feature matrix based on the latent feature representations of images of the same category; calculate the local variance loss and local covariance loss of the multi-view centered feature matrix in a unified feature space;

[0024] Combine the multi-view centralized feature matrices of all categories to generate a global centralized feature matrix; calculate the overall variance loss of the global centralized feature matrix in a unified feature space;

[0025] By fusing local variance loss, local covariance loss, and global variance loss, we obtain the feature semantic comparison binding loss.

[0026] Furthermore, the category semantic contrast binding loss is expressed as:

[0027]

[0028] in, This represents the category semantic contrast binding loss, where V represents the number of views, C represents the number of categories, and n represents the number of categories. i The number of image samples for category i is represented by τ, and the temperature parameter is represented by τ. p represents the latent feature representation of the j-th image in the i-th category under the v-th view. i p represents the sample center feature of the i-th class. k This represents the sample center feature of the k-th class. express With p i Cosine similarity between them, ∈=10 -6 This indicates a small value that is prevented from being divided by zero.

[0029] Furthermore, the sample semantic contrast binding loss is expressed as:

[0030]

[0031] in, Let V represent the semantic contrast binding loss of the samples, C represent the number of views, and n represent the number of categories. i This represents the number of image samples of category i. This represents the latent feature representation of the j-th image of the i-th category under view v. This represents the latent feature representation of the j-th image of the i-th category under view u. This represents the latent feature representation of the j-th image of the l-th category under view m, where τ represents the temperature parameter, ∈ = 10. -6 This indicates a small value that is prevented from being divided by zero.

[0032] Furthermore, the fusion of local variance loss, local covariance loss, and global variance loss is expressed as follows:

[0033]

[0034] in, This represents the feature semantic comparison binding loss. This represents the local variance loss. This represents the local covariance loss. Represents the overall variance loss, ∈ = 10 -6 This indicates a small value that is prevented from being divided by zero.

[0035] Preferably, the process of constructing a learning reinforcement discriminative loss includes:

[0036] Calculate the view mean features of the image based on its latent feature representation; construct the intra-class aggregation loss based on the sum of the mean Euclidean distances between the view mean features of images of the same class; construct the inter-class separation loss based on the sum of the mean Euclidean distances between the view mean features of images of different classes; calculate the semantic boundary enhancement loss based on the intra-class aggregation loss and the inter-class separation loss.

[0037] The class boundary enhancement loss is constructed by minimizing the Euclidean distance between a single image and the center of its class, and maximizing the Euclidean distance between the image and the center of its nearest out-of-class class.

[0038] The semantic boundary reinforcement loss and the category boundary reinforcement loss are weighted and summed to obtain the learning reinforcement discrimination loss.

[0039] Furthermore, the intra-class aggregation loss and inter-class separation loss are expressed as follows:

[0040]

[0041] The semantic boundary enhancement loss is represented as:

[0042]

[0043] in, Indicates intra-class aggregation loss. Indicates the inter-class separation loss. Represents semantic boundary reinforcement loss, ∈=10 -6 This indicates a small value to prevent division by zero. This represents the normalized feature distance matrix, which consists of the Euclidean distances between the mean features of each view. This indicates the indicator matrix, which determines whether the view mean features of two image samples belong to the same category. If they do, the value of the corresponding element is 1; otherwise, it is 0. p represents the sample center feature of the i-th class. i and the sample center feature p of the kth class k Cosine similarity between them.

[0044] Furthermore, the category boundary reinforcement loss is expressed as:

[0045]

[0046] in, This represents the category boundary reinforcement loss. This represents the latent feature representation of the j-th image of the i-th category under view v. V represents the view mean feature of the j-th image in the i-th category, where V represents the number of views. It is the normalized view mean feature of the j-th image in the i-th category, μ i μ represents the view center feature of the i-th category. k Let C represent the view center feature of the k-th category, and let C represent the number of categories, ∈ = 10. -6 To prevent division by zero, n represents a small value. i This represents the number of image samples of category i.

[0047] Preferably, the formula for calculating the total loss of the model is:

[0048]

[0049] in, This represents the total loss of the model. Represents cross-entropy loss, Indicates the reconstruction loss. This represents the category semantic contrast binding loss. This represents the sample semantic comparison binding loss. This represents the feature semantic comparison binding loss. This indicates that learning reinforces the discrimination of loss.

[0050] The beneficial effects of this invention are as follows:

[0051] This invention addresses the common problems in existing multi-view learning methods, such as insufficient feature alignment, inaccurate semantic fusion, and susceptibility to pattern collapse. It proposes a discriminative learning-driven progressive semantic binding mechanism. By introducing category semantic comparison binding, sample semantic comparison binding, feature semantic comparison binding, and learned reinforcement discriminative methods, it significantly improves the consistency of intra-class features and the discriminative power of inter-class features while achieving multi-view feature alignment and fusion. This fundamentally alleviates problems such as pattern collapse and improves the accuracy of output results. This invention is not only applicable to the semantic alignment needs of various heterogeneous view data but also possesses good adaptability and scalability, providing crucial support for the in-depth application of multi-view learning in image recognition, text classification, and cross-modal retrieval, demonstrating promising application prospects. Attached Figure Description

[0052] Figure 1 This is a diagram illustrating the process of progressive contrastive semantic binding and learning-based reinforcement discrimination in this invention.

[0053] Figure 2 This is a comparison chart showing the performance of the present invention and existing mainstream multi-view alignment and fusion models ETMC, UMDL, PDMF, IPMVSC, MV-HFMD, and RCML on the datasets HandWritten, Scene15, 100Leaves, and Hdigit, with Acc, Precision, Recall, and MacroF1 as the metrics. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention proposes a multi-view image classification method based on a progressive semantic binding mechanism, such as... Figure 1 As shown, the method includes the following:

[0056] The multi-view image is acquired and input into a trained multi-view image classification model to obtain the classification result of the multi-view image.

[0057] The training process for a multi-view image classification model includes:

[0058] S1: Obtain multi-view images and category labels; preprocess the multi-view images to generate high-dimensional feature vectors.

[0059] Obtain the multi-view images and their corresponding category labels, preprocess the input multi-view images, and generate the corresponding high-dimensional feature vectors X. (v) (the element in the i-th row and j-th column of the matrix) (This refers to the high-dimensional feature vector of the j-th image of the i-th category under view v), which is the original multi-view data.

[0060] S2: The encoder is used to process the high-dimensional feature vector to obtain the latent feature representation of the image.

[0061] X (v) In the input encoder, the data is projected onto a unified low-dimensional embedding space to obtain the latent feature matrix H of the multi-view data. v (the element in the i-th row and j-th column of the matrix) (The feature of a single sample is the latent feature representation of the j-th image of the i-th category under view v); preferably, the encoder can be a multi-layer perceptron (MLP).

[0062] S3: Construct progressive semantic binding loss and learn reinforcement discrimination loss based on latent feature representation; the progressive semantic binding loss includes category semantic contrast binding loss, sample semantic contrast binding loss and feature semantic contrast binding loss.

[0063] The process of constructing progressive semantic binding loss includes:

[0064] Calculate the sample center features for each category:

[0065]

[0066] Where, p i Let V represent the sample center feature of the i-th category, C represent the number of views, and n represent the number of categories. i This represents the number of samples of category i (each sample has features across all views, so the number of samples of category i is n in any view). i ).

[0067] A category semantic contrast binding loss is constructed based on the similarity between the latent feature representation of an image and the sample center features of each category. Specifically, the similarity between the sample feature (i.e., the latent feature representation) and the sample center features of its respective category is used as the numerator, and the similarity with the sample center features of all categories is used as the denominator. By maximizing the ratio between the two, the consistency alignment between multi-view features and category semantic centers is enhanced.

[0068]

[0069] in, Let InfoNCE represent the category semantic contrast binding loss, τ represent the temperature parameter, and p represent the category semantic contrast binding loss. i p represents the sample center feature of the i-th class. k This represents the sample center feature of the k-th class. express With p i Cosine similarity between them, ∈=10 -6 This indicates a small value that is prevented from being divided by zero.

[0070] This loss function enables category-semantic-level contrastive binding of multi-view feature representations.

[0071] The sample semantic contrast binding loss is constructed based on the ratio between the similarity of similar images of the same class and the similarity of dissimilar images. Specifically, a sample-level semantic contrast binding loss function, ContLoss, is constructed using multi-view feature representations, where the numerator is the similarity index of feature representations between samples of the same class, and the denominator is the similarity index of feature representations between samples of different classes. By maximizing the ratio between the similarity of similar samples of the same class and the similarity of dissimilar samples, semantic aggregation and class differentiation at the sample level are achieved. After processing, multi-view feature representations of sample binding are obtained. The corresponding loss expression is:

[0072]

[0073] in, This represents the sample semantic comparison binding loss. This represents the latent feature representation of the j-th image of the i-th category under view u. This represents the latent feature representation of the j-th image of the l-th category under view m, where τ represents the temperature parameter, ∈ = 10. -6 This indicates a small value that is prevented from being divided by zero.

[0074] This loss function enables sample-level binding of multi-view feature representations.

[0075] Multi-view centered feature matrices are generated based on the latent feature representations of images of the same category. The local variance loss and local covariance loss of the multi-view centered feature matrices in a unified feature space are calculated. Specifically, for samples within the same category, their multi-view centered feature matrices are constructed using the multi-view feature representations.

[0076]

[0077] Calculate the local variance loss VarLoss and local covariance loss CovLoss of this matrix in a unified feature space. Minimize the local variance loss to enhance intra-class semantic consistency, and maximize the local covariance loss to enhance the expressive differences between feature dimensions. The corresponding expressions are:

[0078]

[0079] Where d represents the number of dimensions. This represents the standard deviation of the feature matrix of the i-th class of samples in view v on the j-th feature dimension. This represents the feature matrix formed by the features of the i-th class of samples in the v-th view after centering. This represents the covariance matrix of the i-th class of samples under the v-th view. Represents the covariance matrix The (j,k)th element, ∈=10 -6 This indicates a small value that is prevented from being divided by zero.

[0080] Combine the multi-view centralized feature matrices of all categories to generate a global centralized feature matrix; calculate the overall variance loss of the global centralized feature matrix in a unified feature space, specifically:

[0081] Construct a globally centered feature matrix Z based on the multi-view feature representations of all samples. (v) This involves combining multi-view centered feature matrices from all categories and maximizing the overall variance loss of these matrices in the feature space. This improves the semantic discriminative power and ability to differentiate between sample features, resulting in a feature-bound multi-view feature representation. The corresponding strategy is described as follows:

[0082]

[0083] By fusing local variance loss, local covariance loss, and global variance loss, we obtain the feature semantic comparison binding loss:

[0084]

[0085] This loss function implements feature-level semantic binding for multi-view feature representations.

[0086] The process of constructing a learning reinforcement discriminative loss includes:

[0087] Calculate the view mean features of the image based on its latent feature representation; construct the intra-class aggregation loss based on the sum of the Euclidean distances between the view mean features of images of the same class; construct the inter-class separation loss based on the sum of the Euclidean distances between the view mean features of images of different classes; specifically:

[0088] We construct a joint optimization strategy based on Euclidean distance for intra-class aggregation and inter-class separation using multi-view feature representation. First, we calculate the view mean feature of the sample across all views:

[0089]

[0090] Next, the features are normalized:

[0091]

[0092] Furthermore, a feature distance matrix is ​​constructed based on the Euclidean distance between the mean features of each view after normalization.

[0093]

[0094] in, It is the normalized view mean feature of the j-th image in the i-th category. It is the normalized k-th category and the View mean characteristics of each image.

[0095] The category indicator matrix is ​​then calculated. This involves determining whether the view mean features of two image samples belong to the same category. If they do, the corresponding element's value is 1; otherwise, it's 0. The corresponding calculation formula is:

[0096]

[0097] Where the matrix elements in express and Do they belong to the same category? If they belong to the same category... otherwise

[0098] Further, cosine similarity between category centers is introduced. As a weighting coefficient, the Euclidean distance is adjusted for weighting, and the corresponding calculation formula is:

[0099]

[0100] Where p i p is the sample center feature of the i-th class. k This represents the central feature of the sample in the k-th category.

[0101] Intra-class feature convergence is achieved by minimizing the sum of the mean Euclidean distances between the mean features of views within the same class; simultaneously, inter-class discriminative power is enhanced by maximizing the sum of the mean Euclidean distances between the mean features of views from different classes. This is achieved by introducing cosine similarity between class centers. As a weighting coefficient, the Euclidean distance is adjusted to enhance the adaptive learning of semantic boundaries. The corresponding expression is:

[0102]

[0103] Where, ∈=10-6 This indicates a small value that is prevented from being divided by zero.

[0104] Calculate the semantic boundary enhancement loss based on the intra-class aggregation loss and inter-class separation loss:

[0105]

[0106] in, Indicates intra-class aggregation loss. This represents the inter-class separation loss.

[0107] The class boundary enhancement loss is constructed by minimizing the Euclidean distance between a single sample and the center of its class, and maximizing the Euclidean distance between that sample and the center of its nearest out-of-class class. Specifically:

[0108] By minimizing the Euclidean distance between a single image and the center of its class, and simultaneously maximizing the Euclidean distance between that image and the center of its nearest out-of-class class, explicit enhancement of class boundaries and further improvement of discriminative performance are achieved. The corresponding expression is:

[0109]

[0110] in, This represents the category boundary reinforcement loss. μ represents the view mean feature of the j-th image in the i-th category. i μ represents the view center feature of the i-th category. k This represents the view center feature of the k-th category.

[0111] The weighted sum of the semantic boundary reinforcement loss and the category boundary reinforcement loss yields the learning reinforcement discrimination loss:

[0112]

[0113] Where γ is the weighting coefficient.

[0114] Ultimately, the output multi-view feature representation not only achieves spatial alignment but also possesses stronger intra-class consistency and inter-class discriminability, while the corresponding category center is the fused representation of multi-view data in the semantic space.

[0115] S4: Input the latent feature representation into the decoder to generate the reconstructed high-dimensional feature vector; calculate the reconstruction loss based on the high-dimensional feature vector and the reconstructed high-dimensional feature vector.

[0116] Representing latent features The decoder should be used to restore the original multi-view data as much as possible. The reconstruction loss is calculated using the high-dimensional feature vector and the reconstructed high-dimensional feature vector, expressed as follows:

[0117]

[0118] S5: Input the latent feature representation into the classifier to obtain the classification result of the multi-view image; calculate the cross-entropy loss based on the classification result of the multi-view image and the category label.

[0119] The cross-entropy loss function first applies softmax normalization to the model output, then calculates the log probability of the predicted probability for the true class, takes a negative sign, and averages it over all samples. A smaller value indicates that the model's prediction is closer to the true label. The corresponding formula is:

[0120]

[0121] Where, p i,k This represents the probability that the i-th sample is predicted to be of the k-th class. This represents the probability that the i-th sample is correctly predicted as the i-th class.

[0122] S6: Calculate the total model loss based on progressive semantic binding loss, learning reinforcement discrimination loss, reconstruction loss, and cross-entropy loss; adjust the model parameters based on the total model loss to obtain the trained multi-view image classification model.

[0123] The total loss of the model is expressed as:

[0124]

[0125] in, This represents the total loss of the model. Represents cross-entropy loss, Indicates the reconstruction loss. This represents the category semantic contrast binding loss. This represents the sample semantic comparison binding loss. This represents the feature semantic comparison binding loss. This indicates that learning reinforces the discrimination of loss.

[0126] The encoder parameters are iterated in a direction that reduces the overall loss function until the difference between the loss values ​​of two consecutive iterations is less than 10. -4 Alternatively, training may stop after a predetermined number of iterations, such as 400, while preserving the model parameters, resulting in a trained multi-view image classification model. The latent feature representation of the multi-view data obtained using the trained encoder is the multi-view feature enhanced with consistency alignment and class discriminativeness. Using this feature for subsequent tasks yields more accurate results.

[0127] For example, by acquiring multi-view images and inputting them into a trained multi-view image classification model, accurate multi-view image classification results can be obtained.

[0128] Multi-view features enhanced with consistency alignment and category discriminative power The input is fed into the classifier to obtain the probability of each sample being assigned to each class. The class with the highest probability is taken as the final class of the sample. The corresponding expression is as follows:

[0129]

[0130] Among them, f c Represents the classifier, ω v This represents the learnable weight of the v-th view. This represents the classification category of the i-th sample.

[0131] The performance of the present invention and the comparison method was compared through simulation on various datasets, such as... Figure 2 As shown in the figures, the present invention has superior performance.

[0132] In summary, this invention proposes a progressive contrastive semantic binding strategy to ensure semantic consistency of multi-view data at different levels. At the same time, it adopts a learning-based reinforcement discriminative architecture to effectively enhance the differences in features between different categories, significantly improving the accuracy and robustness of multi-view data alignment and fusion.

[0133] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-view image classification method based on a progressive semantic binding mechanism, characterized in that, include: Acquire multi-view images and input them into a trained multi-view image classification model to obtain the classification results of the multi-view images; The training process for a multi-view image classification model includes: S1: Obtain multi-view images and category labels; preprocess the multi-view images to generate high-dimensional feature vectors; S2: The encoder is used to process the high-dimensional feature vector to obtain the latent feature representation of the image; S3: Construct a progressive semantic binding loss and learn a reinforcement discrimination loss based on the latent feature representation; the progressive semantic binding loss includes category semantic contrast binding loss, sample semantic contrast binding loss, and feature semantic contrast binding loss; the process of constructing the progressive semantic binding loss includes: Calculate the sample center features for each category; construct the category semantic contrast binding loss based on the similarity between the latent feature representation of the image and the sample center features of each category; The sample semantic contrast binding loss is constructed based on the ratio between the similarity of images of the same type and the similarity of images of different types. Generate a multi-view centered feature matrix based on the latent feature representations of images of the same category; calculate the local variance loss and local covariance loss of the multi-view centered feature matrix in a unified feature space; Combine the multi-view centralized feature matrices of all categories to generate a global centralized feature matrix; calculate the overall variance loss of the global centralized feature matrix in a unified feature space; By fusing local variance loss, local covariance loss, and global variance loss, we obtain the feature semantic comparison binding loss. S4: Input the latent feature representation into the decoder to generate a reconstructed high-dimensional feature vector; calculate the reconstruction loss based on the high-dimensional feature vector and the reconstructed high-dimensional feature vector. S5: Input the latent feature representation into the classifier to obtain the classification result of the multi-view image; calculate the cross-entropy loss based on the classification result of the multi-view image and the category label; S6: Calculate the total model loss based on progressive semantic binding loss, learning reinforcement discrimination loss, reconstruction loss, and cross-entropy loss; adjust the model parameters based on the total model loss to obtain the trained multi-view image classification model.

2. The multi-view image classification method based on a progressive semantic binding mechanism according to claim 1, characterized in that, The category semantic contrast binding loss is expressed as: ； in, This represents the category semantic contrast binding loss. Indicates the number of views. Indicates the number of categories. This represents the number of image samples of category i. Indicates temperature parameter, This represents the latent feature representation of the j-th image in the i-th category under the v-th view. Represents the sample center feature of the i-th category. This represents the sample center feature of the k-th class. express and Cosine similarity between them This indicates a small value that is prevented from being divided by zero.

3. The multi-view image classification method based on a progressive semantic binding mechanism according to claim 1, characterized in that, The sample semantic comparison binding loss is expressed as: ； in, This represents the sample semantic comparison binding loss. Indicates the number of views. Indicates the number of categories. This represents the number of image samples of category i. This represents the latent feature representation of the j-th image of the i-th category under view v. This represents the latent feature representation of the j-th image of the i-th category under view u. This indicates that under view m, the first... The latent feature representation of the j-th image in each category Indicates temperature parameter, This indicates a small value that is prevented from being divided by zero.

4. The multi-view image classification method based on a progressive semantic binding mechanism according to claim 1, characterized in that, The combined local variance loss, local covariance loss, and global variance loss are expressed as follows: ； in, This represents the feature semantic comparison binding loss. This represents the local variance loss. This represents the local covariance loss. This represents the overall variance loss. This indicates a small value that is prevented from being divided by zero.

5. A multi-view image classification method based on a progressive semantic binding mechanism according to claim 1, characterized in that, The process of constructing a learning reinforcement discriminative loss includes: Calculate the view mean features of the image based on its latent feature representation; construct the intra-class aggregation loss based on the sum of the mean Euclidean distances between the view mean features of images of the same class; construct the inter-class separation loss based on the sum of the mean Euclidean distances between the view mean features of images of different classes; calculate the semantic boundary enhancement loss based on the intra-class aggregation loss and the inter-class separation loss. The class boundary enhancement loss is constructed by minimizing the Euclidean distance between a single image and the center of its class, and maximizing the Euclidean distance between the image and the center of its nearest out-of-class class. The semantic boundary reinforcement loss and the category boundary reinforcement loss are weighted and summed to obtain the learning reinforcement discrimination loss.

6. The multi-view image classification method based on a progressive semantic binding mechanism according to claim 5, characterized in that, Intra-class aggregation loss and inter-class separation loss are expressed as follows: ； ； The semantic boundary enhancement loss is represented as: ； in, Indicates intra-class aggregation loss. Indicates the inter-class separation loss. This represents semantic boundary reinforcement loss. This indicates a small value to prevent division by zero. This represents the normalized feature distance matrix, which consists of the Euclidean distances between the mean features of each view. This indicates the indicator matrix, which determines whether the view mean features of two image samples belong to the same category. If they do, the value of the corresponding element is 1; otherwise, it is 0. Represents the sample center feature of the i-th category. and the sample center features of the kth class Cosine similarity between them.

7. A multi-view image classification method based on a progressive semantic binding mechanism according to claim 5, characterized in that, The category boundary reinforcement loss is represented as: ； ； in, This represents the category boundary reinforcement loss. This represents the latent feature representation of the j-th image of the i-th category under view v. This represents the view mean feature of the j-th image in the i-th category. Indicates the number of views. It is the normalized view mean feature of the j-th image in the i-th category. This represents the view center feature of the i-th category. This represents the view center feature of the k-th category. Indicates the number of categories. This indicates a small value to prevent division by zero. This represents the number of image samples of category i.

8. The multi-view image classification method based on a progressive semantic binding mechanism according to claim 1, characterized in that, The formula for calculating the total loss of the model is: ； in, This represents the total loss of the model. Represents cross-entropy loss, Indicates the reconstruction loss. This represents the category semantic contrast binding loss. This represents the sample semantic comparison binding loss. This represents the feature semantic comparison binding loss. This indicates that learning reinforces the discrimination of loss.

Images