Semi-supervised image classification method based on granular ball calculation and cooperation matrix

Abstract

The invention discloses a semi-supervised image classification method based on granular ball calculation and a cooperation matrix, and belongs to the technical field of image classification, and the method comprises the following steps: converting an image data set into a plurality of feature matrixes with the line number as the sample number and the column number as the feature number; sequentially inputting the feature matrixes of different views into an unsupervised granular ball generation model, and adaptively adding edges to a generation result based on a boundary distance of a granular ball so as to obtain a corresponding adjacent matrix; convolution operation is carried out on the view with the largest feature number, and a cooperation matrix containing local and global information is calculated for each layer of embedded representation; and sharing the calculated cooperation matrix in the same convolution layer of the other views, adding the convolution results of the views, and finally classifying prediction objects. According to the method, a topology construction method based on a granular ball and a cooperation matrix for capturing local and global information are integrated, and through experimental verification, a better effect is achieved in a semi-supervised image classification task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image classification, and particularly relates to a semi-supervised image classification method based on granular ball calculation and collaborative matrix. BACKGROUND

[0002] Graph Convolutional Network (GCN) is one of the main methods for processing graph structured data. GCN updates the node features and propagates the global information of the graph by performing convolution operation on the feature information of the node and the information of its neighbor nodes. It can effectively capture the relationship structure information of the nodes in the graph.

[0003] Multi-View Graph Convolution Network is an important direction in the research of Graph Convolutional Network (GCN) in recent years, and is mainly used for processing data of multiple views or modalities, which can provide complementary information for the representation of the graph. However, how to aggregate more efficient neighbor information in the convolution process and effectively extract and fuse information from multiple views is an important direction of multi-view graph convolution technology research. In multi-view, the object generally does not contain graph structure, and K-Nearest Neighbor algorithm can be used to construct the topology structure for the sample to be put into the learning of graph convolution. However, in recent years, some studies have pointed out that the K-Nearest Neighbor algorithm has many problems in constructing topology by distance, such as the value of k will bring label noise, which will affect the accuracy and expression ability of the model.

[0004] Granular Ball clustering algorithm is an algorithm based on geometric shapes (such as spheres) for clustering analysis. Unlike the traditional K-means clustering algorithm, Granular Ball clustering algorithm adopts a different clustering model, which is usually based on the assumption that the shape of each cluster is approximately spherical. The core idea of Granular Ball clustering algorithm is to assign data points to the cluster centers corresponding to a "ball" for clustering. The basic idea of Granular Ball clustering algorithm is to regard data points as a set of points in a multi-dimensional space, and represent clusters by a "spherical" model. K-means assumes that the shape of each cluster is spherical, and the distance between clusters is uniform, while Granular Ball clustering emphasizes that the geometric shape of the cluster is spherical, and the boundary of each cluster is uniformly distributed.

[0005] In summary, the existing multi-view learning method based on GCN still has some defects in the topology construction method and multi-view fusion strategy: low accuracy in classification, insufficient model expression ability, etc. Based on this, the present application designs a semi-supervised image classification method based on granular ball calculation and collaborative matrix to solve the above problems. SUMMARY

[0006] In view of the above-mentioned defects of the prior art, the present application provides a semi-supervised image classification method based on granular computing and collaborative matrix.

[0007] To achieve the above object, the present application is implemented by the following technical solutions:

[0008] A semi-supervised image classification method based on granular computing and collaborative matrix, comprising the following steps:

[0009] Step 1: Collecting a data set, converting the image data set into a plurality of feature matrices with the number of rows being the number of samples and the number of columns being the number of features, and the number of feature matrices corresponding to the number of views of the data set;

[0010] Step 2: Inputting the feature matrices {X1, X2,..., X V} of different views into an unsupervised granular ball generation model in sequence, adaptively adding edges based on the boundary distance of the granular ball for the generation result, so as to obtain the corresponding adjacency matrices {A1, A2,..., A V};

[0011] Step 3: Performing convolution operation on the view with the most number of features, and calculating a collaborative matrix containing local and global information for each layer of embedded representation thereof;

[0012] Step 4: Sharing the calculated collaborative matrix in the same convolution layer of the remaining views, adding the convolution results of each view, and finally classifying the predicted object.

[0013] Further, in step 1, the image data set is converted into a plurality of feature matrices with the number of rows being the number of samples and the number of columns being the number of features:

[0014] D=(X1, X2,..., X V )

[0015] X=(x1, x2,..., x N )

[0016] Wherein, D is a data set containing a plurality of views, V is the number of views, X v represents the feature matrix of the vth view; X represents the sample contained in each view, N represents the number of samples, and x n represents the nth sample in the view.

[0017] Further, in the step 2, the adjacency matrix is obtained by the granular ball algorithm, and the steps are as follows:

[0018] Step 21: First, obtain the granular ball clustering result by the unsupervised-based granular ball generation method;

[0019] Step 22: Then, calculate the boundary distance between each granular ball and other granular balls.

[0020] Step 23: For each sphere, select the sphere closest to its boundary and perform a cross-sphere full connection operation on the points within the two spheres.

[0021] Furthermore, in step 3, the method for obtaining the collaboration matrix is:

[0022] Step 31: Perform convolution operation on the view with the most features first and store the embedded representation of each layer;

[0023] Step 32: Use convolution kernels of different sizes to obtain local information and global information of different layers of embedding representation and express them as local correlation matrix and global correlation matrix;

[0024] Step 33: Use learnable parameters to fuse the local correlation matrix and the global correlation matrix to obtain and store the l-th layer collaboration matrix.

[0025] Furthermore, in step 4, the method for classifying the predicted object is:

[0026] Step 41: Multiply the l-th layer embedding representation of the remaining views by the collaboration matrix, and use the result as the input of the next convolutional layer;

[0027] Step 42: Perform weighted addition and normalization on the final convolution results of each view to perform classification prediction;

[0028] Step 43: Evaluate using the cross entropy loss function and backpropagate to update the weight matrix.

[0029] Furthermore, in step 2, the adjacency matrix is ​​obtained by the granular ball algorithm, and the specific steps are as follows:

[0030] Step 21: First, the sphere clustering result is obtained by the unsupervised sphere generation method, and the center and radius of each sphere are calculated. The specific sphere generation process is: set the maximum number of nodes in each sphere to It is used as the termination condition of the sphere splitting; let gb be a sphere containing m objects {x1, x2, ..., x m}, c is the center of gb, m is the size of gb, r is the radius of gb, d(x i , c) is the object x i The Euclidean distance to the center c, center c and radius r are calculated as follows:

[0031]

[0032] Step 22: Then calculate the boundary distance between each sphere and other spheres; for any two spheres gb I and gbJ , the boundary distance is defined as the distance between the centers of the two granules (‖c I -c J ‖) minus the sum of the radii of the granules (r I +r J ), where c I is the center of gb I , c J is the center of gb I , r I is the radius of gb J , and r J is the radius of gb J ;

[0033] The formula for calculating the boundary distance of two granules is as follows:

[0034] Dist(gb I , gb J ) = ‖c I -c J ‖ - (r I +r J )

[0035] Step 23: For each granule, select the granule closest to it in boundary distance, and perform a cross-granule full connection operation on the points within the two granules to finally obtain an adjacency matrix;

[0036] Find the neighbor granule N(gb I ) that meets the condition for granule gb I , and the formula is as follows:

[0037]

[0038] Where N(gb I ) represents the granule closest to granule gb I in boundary distance.

[0039] Further, the connection relationship of the edges is as follows:

[0040]

[0041] Further, in step 3, the collaboration matrix is obtained, and the specific steps are as follows:

[0042] Step 31: Perform global average pooling on the embedding representation H l of the l-th layer, where D h is the dimension of the hidden layer, and H l is obtained according to the following formula:

[0043]

[0044] where σ is a sigmoid activation function, H 0 is the feature matrix of the input layer, is the degree matrix in the symmetric normalized Laplacian matrix, α, β l are hyperparameters, is the symmetric normalized Laplacian matrix, W l is the weight matrix of the lth layer;

[0045] First, the lth layer embedding representation H l is globally averaged to obtain the information vector I:

[0046] I = GAP(H l )

[0047] where GAP is a global average pooling function that can change the dimension of H l from NxD to Nxl;

[0048] Step 32: Obtain local information and global information of embedding representation using different sizes of convolution kernels respectively; use a two-dimensional convolution kernel of a specified size to perform local aggregation on H l , the formula is as follows:

[0049] I lo = Conv2D(H l )

[0050] where the size k of the two-dimensional convolution kernel is

[0051] Apply a one-dimensional convolution kernel to extract the global relationship between nodes:

[0052] I go = Conv1D(I)

[0053] Step 33: Use learnable parameters to fuse and interact the local information matrix and the global information matrix, thereby obtaining the lth layer collaboration matrix T l and store it;

[0054] Combine the global features I go extracted by the one-dimensional convolution kernel with the local I lo features captured by the two-dimensional convolution kernel, and capture the correlation between them at different granularities through cross-correlation operation; the calculation method is as follows:

[0055]

[0056] where M1 and M2 represent the local correlation matrix and the global correlation matrix;

[0057] Initialize the learnable parameters Fuse M1 and M2 to obtain the final collaboration matrix Tl , the calculation process is as follows:

[0058]

[0059] Among them, σ represents the sigmoid activation function;

[0060] Finally, the collaboration matrix T l With the embedding representation H of the lth layer l Multiply to obtain the updated embedding representation H l* :

[0061]

[0062] Among them, H l is the embedding representation of the lth layer, H l* is the input of the (l+1)th layer, Represents element-wise matrix multiplication.

[0063] Furthermore, in step 4, the predicted objects are classified, and the specific steps are as follows:

[0064] Step 41: Embed the lth layer of the remaining views into representations With the collaboration matrix T l Multiply, and the result is used as the updated embedding representation As the input of the next convolutional layer, its formula is as follows:

[0065]

[0066] in, is the collaboration matrix T l-1 right Updated embedding representation of the (l-1)th layer; R = (1-β l )I N +β l W l Is the identity mapping of ResNet, α, β l is a hyperparameter; is the symmetric normalized Laplace matrix, W l is the weight matrix of the lth layer;

[0067] Step 42: Perform weighted addition and normalization on the final convolution results of each view to perform classification prediction;

[0068]

[0069] Where L represents the label sample set;

[0070] All G are weightedly fused to obtain the final embedding expression Z:

[0071]

[0072] wherein V represents the number of views;

[0073] Normalizing the final result:

[0074]

[0075] Step 43: evaluation is performed using the cross-entropy loss function, then the probability of each category prediction is obtained, and the weight matrix is updated by back propagation; the formula of the cross-entropy function is as follows:

[0076]

[0077] wherein X L represents a set of labeled samples, C represents the number of label categories, Y kj represents the true value of the kth sample for the jth label, represents the predicted value of the kth sample for the jth label.

[0078] Further, α = 0.1, I N is an N-order unit matrix.

[0079] Compared with the prior art, the application has the beneficial effects that the application integrates the topology construction method based on the particle ball and the collaborative matrix for capturing local and global information, and solves the semi-supervised image classification problem. Experiments show that the application achieves better results in the semi-supervised image classification task. BRIEF DESCRIPTION OF DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0081] Figure 1 is a flowchart of the embodiment of the present application;

[0082] Figure 2 is a flowchart of obtaining an adjacency matrix by a particle ball algorithm;

[0083] Figure 3 is a flowchart of convolution operation and classification of predicted objects;

[0084] Figure 4 is a framework diagram of the method of the present application;

[0085] Figure 5 Accuracy plot of the proposed method and other methods on 5 multi-view datasets with different label rates {5%,... 50%};

[0086] Figure 6 F1 score plot of the proposed method and other methods on 5 multi-view datasets with different label rates {5%,... 50%};

[0087] Figure 7 2D visualization plot of the proposed method on different datasets;

[0088] Figure 8 Loss function convergence plot of the proposed method and accuracy and F1 score plots;

[0089] Figure 9 Module ablation plot of the proposed method on different datasets. DETAILED DESCRIPTION

[0090] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments but not all embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0091] Embodiment one: please refer to the drawings in the specification Figures 1-4 A semi-supervised image classification method based on granular computing and collaborative matrix, named GBCM-GCN, includes the following steps:

[0092] Step 1: collect the dataset, convert the image dataset into a plurality of feature matrices with the number of rows being the number of samples and the number of columns being the number of features, and the number of feature matrices corresponding to the number of views of the dataset;

[0093] D = (X1, X2,..., X V )

[0094] X = (x1, x2,..., x N )

[0095] Wherein, D is a dataset containing multiple views, V is the number of views, X v represents the feature matrix of the vth view; X represents each view containing samples, N represents the number of samples, x n represents the nth sample in the view; each view contains the same sample and different features, and multiple different views collectively describe an object.

[0096] Step 2: input the feature matrix {X1, X2,..., Xn} of different views into the unsupervised granule generation model in turn, and add edges to the generated results based on the boundary distance of the granules, so as to obtain the corresponding adjacency matrix {A1, A2,..., An}. V}。 V}。

[0097] In step 2, the adjacency matrix is obtained by the granule algorithm, and the steps are as follows:

[0098] Step 21: first, the granule clustering result is obtained by the unsupervised granule generation method;

[0099] Step 22: then, the boundary distance between each granule and other granules is calculated;

[0100] Step 23: for each granule, select the granule with the closest boundary distance, and perform a cross-granule full connection operation on the points in the two granules.

[0101] Step 3: perform convolution operation on the view with the most features, and calculate the collaborative matrix containing local and global information for each layer embedding representation;

[0102] In step 3, the method for obtaining the collaborative matrix T l is as follows:

[0103] Step 31: first, perform convolution operation on the view with the most features and store the embedding representation of each layer;

[0104] Step 32: use different sizes of convolution kernels to obtain local information and global information of different layer embedding representations and represent them as local correlation matrix and global correlation matrix;

[0105] Step 33: use the learnable parameters to fuse and interact the local correlation matrix and the global correlation matrix, so as to obtain the l-th layer collaborative matrix T l and store it.

[0106] Step 4: share the calculated collaborative matrix in the same convolution layer of the remaining views, add the convolution results of each view, and finally classify the prediction object;

[0107] In step 4, the method for classifying the prediction object is as follows:

[0108] Step 41: multiply the l-th layer embedding representation H l of the remaining views with the collaborative matrix T l , and the result is used as the input of the next convolution layer;

[0109] Step 42: add the final layer convolution results of each view after weighting and normalization, and then perform classification prediction;

[0110] Step 43: evaluation is performed using the cross-entropy loss function, and the weight matrix is updated by backpropagation.

[0111] The application can adaptively determine the number of connected edges of different data scales, and eliminate label noise caused by the k value of the traditional K nearest neighbor algorithm. The topology generated based on the granule ball is more robust, and provides higher quality neighbor information for subsequent convolution operations.

[0112] The application better enhances the transmission of consistency information between views based on the shared collaborative matrix between views, fully utilizes local and global information of data, and makes the model have better generalization ability.

[0113] The application uses the granule ball algorithm to adaptively connect edges to construct an adjacency matrix instead of the traditional K nearest neighbor algorithm, obtains a higher quality topology, shares the collaborative matrix containing local and global information between different views, fully utilizes the consistency between views, classifies the predicted objects, and significantly improves the semi-supervised image classification performance.

[0114] Embodiment two: in step 2, the adjacency matrix is obtained by the granule ball algorithm, and the specific steps are as follows:

[0115] Step 21: first, the granule ball clustering result is obtained by the unsupervised granule ball generation method, and the center and radius of each granule ball are calculated.

[0116] The specific granule ball generation process is as follows: in order to balance the number and size of granule balls, the maximum number of nodes in each granule ball is set to as the termination condition of granule ball splitting; gb is a granule ball, which contains m objects {x1, x2,..., x m , c is the center of gb, m is the size of gb, r is the radius of gb, d(x i , c) is the Euclidean distance from object x i to the center c, and the calculation formulas of the center c and the radius r are as follows:

[0117]

[0118] Step 22: then, the boundary distance between each granule ball and other granule balls is calculated. For any two granule balls gb I and gb J , the boundary distance is defined as the center distance (‖c I -c J ‖) of the two granule balls minus the sum of the radii (r I +r J ), wherein c I is the center of gb I , c J is the center of gb IThe center of r I It's GB J The radius, r J It's GB J radius.

[0119] Therefore, the calculation formula for the boundary distance between the two balls is listed as follows:

[0120] Dist(gb I , gb J )=‖c I -c J ‖-(r I +r J )

[0121] Step 23: For each ball, select the ball closest to its boundary, perform a full connection operation across the two balls, and finally obtain the adjacency matrix. The number of edges added is determined by the number of balls and the number of nodes in the balls, which is the ball gb I Find the neighboring spheres N(gb I ), the formula is as follows:

[0122]

[0123] Among them, N(gb I ) represents the particle gb I The particle closest to the boundary.

[0124] The edge connection relationship is as follows:

[0125]

[0126] Example 3: In step 3, the collaboration matrix T is obtained l , the specific steps are as follows:

[0127] Step 31: Represent the embedding of layer l as H l Perform global average pooling, where D h is the dimension of the hidden layer, H l The formula for obtaining it is as follows:

[0128]

[0129] Where σ is the sigmoid activation function, H 0 is the feature matrix of the input layer, is the degree matrix in the symmetric normalized Laplace matrix, α, β l is a hyperparameter, where α = 0.1, is the symmetric normalized Laplace matrix, W l is the weight matrix of the lth layer; I N is a unit matrix of order N.

[0130] First, the l-th layer embedding representation H l is globally averaged to obtain the information vector I:

[0131] I = GAP(H l )

[0132] where GAP is a global average pooling function that can change the dimension of H l from NxD to Nxl.

[0133] Step 32: Use different sizes of convolution kernels to obtain local information and global information of embedding representation. Use a two-dimensional convolution kernel of a specified size to perform local aggregation on H l , the formula is as follows:

[0134] I lo = Conv2D(H l )

[0135] where the size k of the two-dimensional convolution kernel is

[0136] In order to capture global information and improve the representation ability, a one-dimensional convolution kernel is applied to extract the global relationship between nodes:

[0137] I go = Conv1D(I)

[0138] Step 33: Use learnable parameters to fuse and interact the local information matrix and the global information matrix, thereby obtaining the l-th layer collaborative matrix T l and storing it.

[0139] In order to realize the effective interaction of global information and local information, the global features I go extracted by the one-dimensional convolution kernel are combined with the local I lo features captured by the two-dimensional convolution kernel, and the correlation between them is captured at different granularities through cross-correlation operation. The calculation method is as follows:

[0140]

[0141] where M1 and M2 represent the local correlation matrix and the global correlation matrix.

[0142] In order to be able to flexibly allocate the weights of the correlation matrix, initialize the learnable parameter to adaptively fuse them, and the final collaborative matrix T l is obtained by fusion, and the calculation process is as follows:

[0143]

[0144] Among them, σ represents the sigmoid activation function.

[0145] This method effectively avoids redundant operations between local and global information while enhancing their interactivity. Ultimately, it prioritizes local features and suppresses features with less information.

[0146] Finally, the collaboration matrix T l With the embedding representation H of the lth layer l Multiply to obtain the updated embedding representation H l* :

[0147]

[0148] Among them, H l is the embedding representation of the lth layer, H l* is the input of the (l+1)th layer, Represents element-wise matrix multiplication.

[0149] Example 4: In step 4, the prediction object is classified. The specific steps are as follows:

[0150] Step 41: Embed the lth layer of the remaining views into representations With the collaboration matrix T l Multiply, and the result is used as the updated embedding representation As the input of the next convolutional layer. The formula is as follows:

[0151]

[0152] in, is the collaboration matrix T l-1 right Updated embedding representation of the (l-1)th layer; R = (1-β l )I N +β l W l It is the concept of identity mapping in ResNet, α, β l is a hyperparameter, where α = 0.1, is the symmetric normalized Laplace matrix, W l is the weight matrix of the lth layer; I N is the N-order identity matrix; considering that the model's performance may decrease with the aggregation of multiple neighbors, an identity mapping term (1-β l )It helps maintain the integrity of input information.

[0153] Step 42: the final layer convolution results of each view are weighted and added and then normalized to make a classification prediction.

[0154]

[0155] wherein L represents a label sample set;

[0156] All Gs are weighted and fused to obtain a final embedding expression Z:

[0157]

[0158] wherein V represents the number of views;

[0159] The final result is normalized:

[0160]

[0161] Step 43: the cross-entropy loss function is used for evaluation, then the probability of each category prediction is obtained, and the weight matrix is updated by back propagation. The formula of the cross-entropy function is as follows:

[0162]

[0163] wherein X L represents a labeled sample set, C represents the number of label categories, Y kj represents the true value of the kth sample for the jth label, represents the predicted value of the kth sample for the jth label.

[0164] Experimental example: the method (GBCM-GCN) of the application and the traditional multi-view method are used for experimental simulation

[0165] Five multi-view image data sets are selected, and the information is shown in Table 1.

[0166] Table 1: Five multi-view image data sets

[0167]

[0168] Two widely used machine learning metrics: ACC (accuracy) and F1-score are computed. The larger the value of these metrics, the better the classification performance. For the training set, 10% of each class is randomly selected as labeled samples, and the remaining 90% as unlabeled samples. 10% of the unlabeled samples are selected as the validation set, and the remaining 80% as the test set. For each method, 10 experiments are performed, and the average accuracy (ACC) and F1-score are reported. The hyperparameters of all competing methods are selected according to the validation performance and remain fixed during testing. The experiment is performed on a computer equipped with a 16GHz Intel(R) Core(TM) i9-13600 CPU and a NVIDIA RTX 3080ti graphics card. The operating system used is Windows 10, and the experiment uses the Python 3.11 programming language and the PyTorch library 1.12.0+cu 118. Each network is implemented using the PyTorch framework. The Adam (adaptive moment estimation) optimization algorithm is used for training, and the maximum number of iterations is 500. The results are shown in Table 2.

[0169] Table 2 Performance comparison of GBCM-GCN and 8 multi-view learning methods in semi-supervised image classification

[0170]

[0171] The 8 multi-view learning methods are: MLAN (multi-view attention network), MVAR (multivariate autoregressive model), Co-GCN (co-graph convolution network), DSRL (deep semantic representation learning), LGCN-FF (local graph convolution network-feed forward), JFGCN (joint feature graph convolution network), HGCN-MVSC (hypergraph convolution network-multi-view subspace clustering), MvRL-DP (multi-view representation learning-dynamic programming), GBCM-CCN (graph-based collaborative model-cross convolution network).

[0172] As can be seen from Table 2, GBCM-GCN achieves the best performance on all datasets. Specifically, GBCM-GCN improves the second best result of MNIST, NUS-WIDE and OUTSCENE datasets by 4.3%, 30.8% and 11.3%, respectively, at a label rate of 10%. Compared with models that only use the convolution results of a single view, GBCM-GCN can achieve more rich interaction between views, thus achieving superior classification performance.

[0173] Figures 5-6In the experiment, the labeled sample size is set to {5%, 10%,..., 50%}, the validation set is fixed to 10%, and the remaining data is used for testing. It can be seen that the ACC / F1 scores show an increasing trend with the increase of the label rate. The results show that the GBCM-GCN algorithm has stable performance in multi-view semi-supervised image classification, especially showing obvious advantages under low label rate. This is attributed to the collaborative matrix learning ability, which enables it to make full use of the captured local and global information even with only a small amount of labeled samples available. In summary, the combination of robust topology structure and collaborative matrix and the introduction of GCN into multi-view learning can enable the model to have strong expression learning ability.

[0174] Figure 7 It can be seen that the samples of each category show good aggregation and separability in the embedding space, indicating that the topology structure and convolution learning representation constructed by the method have strong discrimination ability. Compared with traditional multi-view methods, the method exhibits clearer class boundaries, more compact intra-class data distribution and more obvious inter-class distance on each dataset, reflecting the advantages of the method in representation learning and feature fusion. Especially on high-dimensional multi-view datasets such as NUS-WIDE and OutScene, the GBCM-GCN model significantly improves the separability of data distribution, further verifying the effectiveness of the collaborative matrix in capturing local and global information.

[0175] Figure 8 It can be seen that the loss function rapidly decreases at the beginning of training and converges within a short number of iterations, indicating that the model has good training stability and convergence speed. At the same time, the accuracy and F1 scores steadily increase with training and reach a stable state after the loss converges, indicating that the model has strong generalization ability while maintaining low training error. In addition, the F1 score and accuracy curves are highly consistent, further verifying the robustness of the model in dealing with class imbalance problems. Overall, Figure 8 The results fully demonstrate the strong performance and optimization effect of the method in semi-supervised image classification tasks.

[0176] Figure 9 It can be seen that each key module has a significant contribution to the overall performance. Specifically, when the adaptive topology structure constructed by the Granular Ball algorithm is removed, the classification performance decreases significantly, indicating the effectiveness of the method in constructing a robust adjacency matrix; when the collaborative matrix sharing mechanism is removed, the model cannot fully utilize the consistency information between views, resulting in weakened generalization ability and significant reduction in F1 score and accuracy; in addition, the embedding mechanism that does not consider the interaction between local and global features also weakens the model's ability to express multi-scale features. In summary, the results in the figure fully verify the important role of each key module in improving the performance of semi-supervised image classification.

[0177] The application integrates a granular ball-based topological construction method and a collaborative matrix for capturing local and global information, and solves the semi-supervised image classification problem.

[0178] The above examples are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements will not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A semi-supervised image classification method based on granular sphere calculation and collaborative matrix, characterized in that: The following steps are involved: Step 1: Collect the dataset and convert the image dataset into multiple feature matrices with the number of rows being the number of samples and the number of columns being the number of features. The number of feature matrices corresponds to the number of dataset views. Step 2: Substitute the feature matrices {X1, X2, ..., X V } are sequentially input into the unsupervised particle sphere generation model, and edges are adaptively added to the generated results based on the boundary distance of the particles, so as to obtain the corresponding adjacency matrix {A1, A2, ..., A V }; Step 3: Perform convolution on the view with the largest number of features and calculate the collaboration matrix containing local and global information for each layer of embedding representation; Step 4: Share the calculated collaboration matrix in the same convolutional layer of the remaining views, add the convolution results of each view, and finally classify the predicted object.

2. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 1, characterized in that: In step 1, the image dataset is converted into multiple feature matrices with the number of rows being the number of samples and the number of columns being the number of features: D=(X1,X2,...,X V ) X=(x1,x2,...,x N ) Where D is a dataset with multiple views, V is the number of views, and X v Represents the feature matrix of the vth view; X represents the samples contained in each view, N represents the number of samples, x n Represents the nth sample in this view.

3. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 1, characterized in that: In step 2, the adjacency matrix is ​​obtained by the granular ball algorithm, and the steps are: Step 21: First, obtain the sphere clustering result by using an unsupervised sphere generation method; Step 22: Then calculate the boundary distance between each sphere and other spheres; Step 23: For each sphere, select the sphere closest to its boundary and perform a cross-sphere full connection operation on the points within the two spheres.

4. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 1, characterized in that: In step 3, the method for obtaining the collaboration matrix is: Step 31: Perform convolution operation on the view with the most features first and store the embedded representation of each layer; Step 32: Use convolution kernels of different sizes to obtain local information and global information of different layers of embedding representation and express them as local correlation matrix and global correlation matrix; Step 33: Use learnable parameters to fuse the local correlation matrix and the global correlation matrix to obtain and store the l-th layer collaboration matrix.

5. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 1, characterized in that: In step 4, the method for classifying the predicted object is: Step 41: Multiply the l-th layer embedding representation of the remaining views by the collaboration matrix, and use the result as the input of the next convolutional layer; Step 42: Perform weighted addition and normalization on the final convolution results of each view to perform classification prediction; Step 43: Evaluate using the cross entropy loss function and backpropagate to update the weight matrix.

6. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 1, characterized in that: In step 2, the adjacency matrix is ​​obtained by the granular ball algorithm, and the specific steps are as follows: Step 21: First, the sphere clustering result is obtained by the unsupervised sphere generation method, and the center and radius of each sphere are calculated. The specific sphere generation process is: set the maximum number of nodes in each sphere to It is used as the termination condition of the sphere splitting; let gb be a sphere containing m objects {x1, x2, ..., x m }, c is the center of gb, m is the size of gb, r is the radius of gb, d(x i , c) is the object x i The Euclidean distance to the center c, center c and radius r are calculated as follows: Step 22: Then calculate the boundary distance between each sphere and other spheres; for any two spheres gb I and gb J The boundary distance is defined as the distance between the centers of the two spheres (‖c I -c J ‖) minus the sum of the radii of the spheres (r I +r J ), where c I It's GB I The center of c J It's GB I The center of r I It's GB J The radius, r J It's GB J radius; The boundary distance between two balls is calculated as follows: Dist(gb I ,gb J )=‖c I -c J ‖-(r I +r J ) Step 23: For each sphere, select the sphere closest to its boundary, perform a cross-sphere full connection operation on the points within the two spheres, and finally obtain the adjacency matrix; For pellet gb I Find the neighboring spheres N(gb I ), the formula is as follows: Among them, N(gb I ) represents the particle gb I The particle closest to the boundary.

7. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 6, characterized in that: The edge connection relationship is as follows:

8. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 7, characterized in that: In step 3, the collaboration matrix is ​​obtained, and the specific steps are as follows: Step 31: Represent the embedding of layer l as H l Perform global average pooling, where D h is the dimension of the hidden layer, H l The formula for obtaining it is as follows: Where σ is the sigmoid activation function, H 0 is the feature matrix of the input layer, is the degree matrix in the symmetric normalized Laplace matrix, α, β l is a hyperparameter, is the symmetric normalized Laplace matrix, W l is the weight matrix of the lth layer; First embed the lth layer into representation H l Perform global average pooling to obtain the information vector I: I=GAP(H l ) Among them, GAP is the global average pooling function, which can be used to l The dimension changes from N×D to N×1; Step 32: Use convolution kernels of different sizes to obtain local information and global information of the embedded representation; use a two-dimensional convolution kernel of a specified size to convolution H l Perform local aggregation, the formula is as follows: I lo =Conv2D(H l ) The size of the two-dimensional convolution kernel k is Apply a one-dimensional convolution kernel to extract the global relationship between nodes: I go =Conv1D(I) Step 33: Use learnable parameters to fuse the local information matrix and the global information matrix to obtain the l-th layer collaboration matrix T l and store; The global feature I extracted by the one-dimensional convolution kernel go The local I captured by the two-dimensional convolution kernel lo The features are combined and the correlation between them is captured at different granularities through cross-correlation operations; the calculation method is as follows: Among them, M1 and M2 represent the local correlation matrix and the global correlation matrix; Initialize learnable parameters Fusion M1, M2 to get the final collaboration matrix T l , the calculation process is as follows: Among them, σ represents the sigmoid activation function; Finally, the collaboration matrix T l With the embedding representation H of the lth layer l Multiply to obtain the updated embedding representation H l* : Among them, H l is the embedding representation of the lth layer, H l* is the input of the (l+1)th layer, Represents element-wise matrix multiplication.

9. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 8, characterized in that: In step 4, the predicted objects are classified, and the specific steps are as follows: Step 41: Embed the lth layer of the remaining views into representations With the collaboration matrix T l Multiply, and the result is used as the updated embedding representation As the input of the next convolutional layer, its formula is as follows: in, is the collaboration matrix T l-1 right Updated embedding representation of the (l-1)th layer; R = (1-β l )I N +β l W l Is the identity mapping of ResNet, α, β l is a hyperparameter; is the symmetric normalized Laplace matrix, W l is the weight matrix of the lth layer; Step 42: Perform weighted addition and normalization on the final convolution results of each view to perform classification prediction; Where L represents the label sample set; All G are weightedly fused to obtain the final embedding expression Z: Where V represents the number of views; Normalize the final result: Step 43: Use the cross entropy loss function for evaluation to obtain the predicted probability of each category and backpropagate to update the weight matrix; the formula of the cross entropy function is as follows: Among them, X L represents a set of labeled samples, C represents the number of label types, and Y kj represents the true value of the k-th sample for the j-th label, Represents the predicted value of the kth sample for the jth label.

10. The semi-supervised image classification method based on granular-spherical calculation and collaborative matrix according to claim 8 or 9, characterized in that: α=0.1, I N is the identity matrix of order N.

Images