An image classification method and system

Abstract

This invention belongs to the field of image classification technology and provides an image classification method and system. The method includes: acquiring an image dataset; converting the image dataset into multiple feature matrices with rows equal to the number of samples and columns equal to the number of features; sequentially inputting the feature matrices of different views into a particle generation model; adaptively adding edges to the generated results based on particle density and distance to obtain corresponding adjacency matrices; sorting the feature matrices according to the number of features and feeding them and their corresponding adjacency matrices into a convolution model for computation; feeding the feature matrices and adjacency matrices of the remaining views into the convolution operation, and fusing the embedding representations of each layer of the previous view with the embedding representations of the same layer of the next view, using them together as input for the next convolution operation; and finally performing classification prediction. This invention can improve the accuracy of image classification.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and more specifically, to an image classification method and system. Background Technology

[0002] Graph Convolutional Networks (GCNs) have become one of the main methods for processing graph-structured data since their inception. GCNs update node features and propagate global information of the graph by performing convolution operations on the feature information of nodes with the information of their neighbors. Their advantage lies in their ability to effectively capture the relational structure information of nodes in the graph.

[0003] Multi-view graph convolutional networks (GCNs) have become an important research direction in recent years, primarily used to process data with multiple views or modalities, which provide complementary information for graph representation. A key challenge in multi-view GCN research is how to efficiently aggregate neighbor information and effectively extract and fuse information from multiple views during the convolution process. In multi-view data, samples generally lack topological structure. Most researchers use the K-nearest neighbor algorithm to construct topological structures for graph convolution operations. However, recent studies have pointed out that the K-nearest neighbor algorithm's reliance on distance to construct topology has several problems. For example, the value of k can introduce label noise, affecting the model. This is a problem that needs to be addressed in topology construction.

[0004] Granular Ball clustering is an algorithm that performs clustering analysis based on geometric shapes (such as spheres). Unlike the traditional K-means clustering algorithm, Granular Ball clustering uses a different clustering model, typically based on the assumption that each cluster is approximately spherical. The core idea of ​​Granular Ball clustering is to cluster data points by assigning them to cluster centers corresponding to a certain "sphere". The basic idea of ​​Granular Ball clustering is to view data points as a set of points in a multi-dimensional space and represent clusters using a "spherical" model. Unlike the K-means algorithm, which assumes that each cluster is spherical and the distance between clusters is uniform, Granular Ball clustering emphasizes that the geometry of the clusters is spherical and the boundaries of each cluster are uniformly distributed.

[0005] In summary, existing topology construction methods and multi-view fusion strategies still have some problems. Therefore, it is necessary to propose a multi-view semi-supervised image classification method based on topology construction and ordered cross-view convolution. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides an image classification method, system, electronic device, non-transient computer-readable storage medium, and computer program product.

[0007] This invention provides an image classification method, comprising the following steps: S1: Collect an image dataset and convert it into multiple feature matrices with rows equal to the number of samples and columns equal to the number of features. In the formula, It is a dataset containing multiple views. It is the number of views. Indicates the first Feature matrix of each view; It represents the samples contained in each view. Represents the number of samples. This indicates the first in the view. One sample; S2, the feature matrices of different views are sequentially input into the particle generation model, and edges are adaptively added to the generated results based on particle density and distance to obtain the corresponding adjacency matrix. ; S3, sort the feature matrix according to the number of features to obtain and its corresponding adjacency matrix The data is fed into a convolutional model for computation; the view with the most features is convolved first.

[0008] S4. The feature matrices and adjacency matrices of the remaining views are fed into the convolution operation, and the embedding representations of each layer of the previous view are fused with the embedding representations of the same layer of the next view, which are then used as the input for the next convolution operation. Finally, classification prediction is performed.

[0009] Optionally, step S2 specifically includes: S21: Obtain the sphere clustering results and calculate the sphere center and radius by using an unsupervised sphere generation method; S22: Calculate the density of each granule and its distance from other granules based on the density peak algorithm; S23: For each sphere, select the sphere with a higher density and the closest distance, and perform a full connection operation across the two spheres.

[0010] Optionally, step S3 specifically includes: S31: Sort the different views in descending order based on the number of features, with the view with the most features placed first; S32: The feature matrix of the sorted first view and adjacency matrix Insert convolution operations, record the output of each convolutional layer, and store it.

[0011] Optionally, step S4 specifically includes: S41: Perform convolution operations on the remaining sorted views one by one until the view with the fewest features completes the convolution operation; S42: After the convolution process, obtain the node representation of each view, weight and sum the final convolution results of each view, then normalize them, and perform classification prediction. S43: Use the cross-entropy loss function for evaluation and backpropagate to update the weight matrix.

[0012] Optionally, the backpropagation update of the weight matrix includes: Equivalent features are extracted from the feature matrix and the adjacency matrix, and the update gradient of the weight matrix is ​​adjusted based on the equivalent features. The equivalent features are quantized features that fuse the attribute information of the feature matrix and the topological association information of the adjacency matrix. Based on the equivalent feature matrix, the update gradient of the weight matrix is ​​dynamically determined.

[0013] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method described in the preceding invention.

[0014] The present invention also provides a non-transient computer-readable storage medium having instructions embodied thereon that can be executed by one or more processors to perform a method as described in any of the preceding claims.

[0015] The present invention also provides a computer program product comprising an executable computer program that, when executed, implements the method as described in any of the preceding claims. Attached Figure Description

[0016] Figure 1 This is a flowchart of an image classification method disclosed in an embodiment of the present invention; Figure 2 This is a framework diagram of the method of the present invention; Figure 3 Line graph showing the accuracy of the method of this invention compared with other algorithms on seven multi-view datasets with label rates of {5%,...50%}; Figure 4 Line graph showing the F1 score of the method of this invention and other algorithms under label rates of 7 multi-view datasets {5%,...50%}. Detailed Implementation

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0020] like Figure 1 , Figure 2 As shown in the figure, an embodiment of the present invention discloses an image classification method, including the following steps: S1: Collect an image dataset and convert it into multiple feature matrices with rows equal to the number of samples and columns equal to the number of features. In the formula, It is a dataset containing multiple views. It is the number of views. Indicates the first Feature matrix of each view; It represents the samples contained in each view. Represents the number of samples. This indicates the first in the view. One sample; It is understandable that each view contains the same samples but has different features, and multiple different views together describe an object.

[0021] The image dataset of this invention may be the MNIST multi-view digital image dataset, the NUS-WIDE multi-view object image dataset, the ChestX-ray14 multi-view medical image dataset, the NEU-DET multi-view industrial defect image dataset, etc.

[0022] The MNIST multi-view digit image dataset is a classic image dataset containing 10 classes of handwritten digits from 0 to 9, totaling 10,000 samples, each a 28×28 pixel grayscale image. This dataset constructs a multi-view feature matrix using different feature extraction algorithms, including edge features (64-dimensional), texture features (32-dimensional), grayscale statistical features (30-dimensional), and shape features (9-dimensional). These views all correspond to the same handwritten digit image, differing only in feature dimensions and types, making it suitable for validating the basic performance of multi-view semi-supervised classification methods, especially for testing classification performance in low-labeling scenarios.

[0023] The NUS-WIDE multi-view object image dataset covers 12 categories of everyday objects and scenes, containing 2400 samples with an image resolution of 256×256 pixels. It includes images of cars, animals, plants, and buildings, with some samples containing complex backgrounds. The dataset constructs feature matrices for six views, including color features (500-dimensional), texture features (225-dimensional), depth features (144-dimensional), edge features (128-dimensional), local features (73-dimensional), and global features (64-dimensional), covering both low-level visual features and high-level semantic features. This effectively verifies the ability of multi-view fusion algorithms to integrate different types of features.

[0024] The ChestX-ray14 multi-view medical image dataset contains 112,120 samples of X-ray images for 14 types of chest diseases. These are grayscale medical images with a resolution of approximately 1024×1024 pixels. The images depict the structure of the human chest, and disease regions require specialized knowledge for identification. This dataset generates three views using medical image processing techniques, including lesion region features (128 dimensions), texture statistical features (64 dimensions), and global structural features (32 dimensions). The annotation cost is extremely high, making it suitable for validating the application value of semi-supervised classification methods in professional fields such as medical diagnosis.

[0025] The NEU-DET multi-view industrial defect image dataset contains 1800 industrial images of surface defects in hot-rolled steel strip, categorized into six types. These are 200×200 pixel grayscale images covering defect types such as cracks, iron oxide scale, and scratches. The dataset constructs feature matrices for three views: defect edge features (80-dimensional), grayscale gradient features (48-dimensional), and defect morphology features (24-dimensional). It is suitable for validating defect classification performance in industrial quality inspection scenarios, requiring high classification accuracy, and demonstrating the practical value of semi-supervised methods in industrial applications.

[0026] S2, the feature matrices of different views are sequentially input into the particle generation model, and edges are adaptively added to the generated results based on particle density and distance to obtain the corresponding adjacency matrix. ; S3, sort the feature matrix according to the number of features to obtain and its corresponding adjacency matrix The data is fed into a convolutional model for computation; the view with the most features is convolved first.

[0027] S4. The feature matrices and adjacency matrices of the remaining views are fed into the convolution operation, and the embedding representations of each layer of the previous view are fused with the embedding representations of the same layer of the next view, which are then used as the input for the next convolution operation. Finally, classification prediction is performed.

[0028] Optionally, step S2 specifically includes: S21: Obtain the sphere clustering results and calculate the sphere center and radius by using an unsupervised sphere generation method; The specific process of generating spheres is as follows: To balance the number and size of spheres, the maximum number of nodes in each sphere is set to 1. This is used as the termination condition for sphere splitting. Let gb be a sphere containing m objects. Let c be the center of the sphere, m be the number of samples in the sphere, r be the radius of the sphere, and d be the Euclidean distance from other objects to the center c. The formulas for calculating the sphere center c and the sphere radius r are as follows: ; S22: Calculate the density of each sphere and its distance from other spheres based on the density peak algorithm.

[0029] Given a center c and radius r, and considering that a sphere is a spherical structure, density refers to how tightly packed an object is at a given location. The density of a GB is defined as the mass per unit volume, where mass refers to the number of objects distributed within it. Specifically, the density of a GB is the ratio of its mass to the average distance of a sample from the center of the GB.

[0030] Therefore, before calculating the density, it is necessary to calculate the average distance from all points within the granule to the center c. The calculation formula is as follows: ; Therefore, the formula for calculating density is as follows: ; Substituting equation (3) into equation (4), we get ; Where m is the size of GB, r is the radius of GB, c is the center of GB, and d(x,c) is the Euclidean distance between sample x and center c. Two spheres... and The distance between them is determined as the Euclidean distance between the centers of the two spheres, and the calculation formula is as follows: .

[0031] S23: For each sphere, select the nearest sphere with a higher density, and perform a full connection operation across the two spheres. The densities ρ (gb) of the two spheres have already been obtained. i ),ρ(gb j ) and the distance d(c) between the two spheres i , c j ) We perform a full connection operation across the grain spheres to obtain the desired adjacency matrix. The total number of edges is determined by the number of samples within each grain sphere and the number of generated grain spheres. How to perform a full connection operation across each grain sphere... Find the neighboring balls that meet the criteria. As shown below: ; N( ) indicates for The sphere with the highest density and the closest distance. The edge connections are as follows: ; Optionally, step S3 specifically includes: S31: Sort the different views in descending order based on the number of features, with the view with the most features placed first; The original feature matrix is ,in arrive Since the number of features is different, the features of the V views are arranged in descending order of the number of features to obtain the feature matrix of the V views. The corresponding adjacency matrix is The feature matrix and adjacency matrix serve as the inputs for each view to enter the convolution operation.

[0032] S32: The feature matrix of the sorted first view and adjacency matrix Insert convolution operations, record the output of each convolutional layer, and store it.

[0033] The feature matrix of the first view and adjacency matrix Place it into the GCN model. The l-th layer embedding representation of the sorted first view. The formula is as follows: ;in, This refers to the concept of identity mapping in ResNet. These are hyperparameters, among which . It is a symmetric normalized Laplace matrix. It is the weight matrix of the l-th layer; , It is an N-order identity matrix. .

[0034] Optionally, step S4 specifically includes: S41: Perform convolution operations on the remaining sorted views sequentially until the view with the fewest features completes the convolution operation.

[0035] The l-th layer embedding of the v-th view is represented as: ; S42: After the convolution process, obtain the node representation of each view, weight and sum the final layer convolution results of each view, then normalize them, and perform classification prediction.

[0036] Where L is the total number of convolutional layers.

[0037] All Weighted fusion is performed to obtain the final embedding representation Z: ; Normalizing the final structure yields the predicted probability for each class: ; S43: Use the cross-entropy loss function for evaluation and backpropagate to update the weight matrix.

[0038] The formula for the cross-entropy function is as follows: ; in, Let C represent the set of labeled samples, and let C represent the number of label types. This represents the true value of the k-th sample with respect to the j-th label. This represents the predicted value of the k-th sample for the j-th label.

[0039] Optionally, the backpropagation update of the weight matrix includes: Equivalent features are extracted from the feature matrix and the adjacency matrix, and the update gradient of the weight matrix is ​​adjusted based on the equivalent features. The equivalent features are quantized features that fuse the attribute information of the feature matrix and the topological association information of the adjacency matrix. Based on the equivalent feature matrix, the update gradient of the weight matrix is ​​dynamically determined.

[0040] In traditional backpropagation, weight updates rely solely on the gradient calculated from the cross-entropy loss, which can easily lead to overfitting to noisy samples or ignoring the inherent structure of the data. For example, in processing the MNIST dataset, if a handwritten digit sample has abnormal features due to scanning noise, traditional gradient descent might over-adjust the weights to fit the noisy sample, reducing the model's generalization ability.

[0041] The equivalent features extracted in this invention can be used to perform structural correction on the features of abnormal samples through the topological relationships reflected by the adjacency matrix (such as particle density and cross-particle connectivity), so that the weight update follows both the classification loss guideline and the true distribution of the data. Specifically: (1) Calculate the feature weights at the particle level: For the v-th view after sorting Each sphere Feature weights The discriminative power, reflecting the characteristics of samples within the sphere, is calculated using the following formula: ;in, The particle density calculated for step S22 (reflecting the importance of particles in the topology; the higher the density, the stronger the sample association). For granules The standard deviation of all sample features (reflects the distinguishability of the feature attribute; the larger the standard deviation, the more significant the difference between this feature and other features).

[0042] (2) Generate sample-level equivalent feature vectors: For the nth sample of the vth view If it belongs to granules Then the equivalent feature vector of this sample is : ; For granules The mean of features of all samples (reflecting the global features of the sphere and avoiding the influence of noise from individual samples).

[0043] (3) Construct the view-level equivalent feature matrix: The equivalent feature matrix of the v-th view The set of equivalent feature vectors for all samples: .

[0044] Next, the traditional gradient is calculated based on the cross-entropy loss, and the auxiliary gradient is calculated based on the equivalent feature matrix mentioned above. The traditional gradient and the auxiliary gradient are then weighted and fused to obtain the total gradient.

[0045] Among them, the auxiliary gradient is based on the equivalent feature matrix. In the design, first define the auxiliary loss function: auxiliary loss. = Feature consistency loss + λ × Topology consistency loss

[0046] λ is the topological constraint coefficient (range [0,1]). Feature consistency loss = Topology consistency loss = . The output feature matrix of the l-th convolutional layer is a key intermediate result for feature extraction and cross-view fusion.

[0047] The auxiliary gradient is the partial derivative of the auxiliary loss with respect to the weight matrix W, i.e. , For the spherical-level feature weights The weight matrix is ​​formed.

[0048] The following is an experimental simulation comparing the method of this invention with traditional methods: In this comparative analysis experiment, six multi-view datasets were selected.

[0049] Information on the six multi-view datasets is shown in Table 1: Table 1: Six multi-view datasets:

[0050] Two widely used machine learning metrics were calculated: accuracy and F1 score. Higher values ​​for these metrics generally indicate better classification performance. For the training set, 10% of each class was randomly selected as labeled samples, and the remaining 90% were selected as unlabeled samples. 10% of the unlabeled samples were selected as the validation set, and the remaining 80% were selected as the test set. For each method, 10 experiments were conducted, and the mean accuracy (ACC) and F1 score were reported.

[0051] The hyperparameters of all competing methods were selected based on validation performance and remained fixed during testing. All other parameters were set to their default values, as described in the original documentation. Each network was implemented using the PyTorch framework and trained using the Adam (Adaptive Moment Estimation) optimization algorithm with a maximum of 500 iterations.

[0052] The ACC and F1 scores of GBOC-GCN and other baselines are summarized in Table 2: Table 2: Performance comparison of GBOC-GCN and 8 multi-view methods in semi-supervised classification:

[0053] As clearly shown in Table 2, GBOC-GCN achieves state-of-the-art performance across all datasets. Specifically, GBOC-GCN improves the labeling accuracy by 5.3%, 46.3%, and 17.8% compared to the second-best results on the MNIST, NUS-WIDE, and OutScene datasets, respectively, at 10%. Compared to models that utilize the convolutional results of only a single view, GBOC-GCN enables richer interactions between views, resulting in superior classification results.

[0054] in addition, Figure 3 , Figure 4This illustrates how ACC and F1 scores vary with different label percentages on the selected dataset. The labeled sample size was set to {5%, 10%, ..., 50%}, the validation set was fixed at 10%, and the remaining data was used for testing. Figure 3 , Figure 4 As can be seen, the ACC / F1 score increases with the label rate of all competing methods. The results show that the GBOC-GCN algorithm performs stably in multi-view semi-supervised classification, especially exhibiting a significant advantage at low label rates. This is attributed to the learning capability of directed convolutional fusion, which allows it to fully utilize semantic features even with only a small number of labeled samples available. In conclusion, combining robust topology with directed cross-view convolutional fusion and introducing GCN into multi-view learning can give the model strong representation learning capabilities.

[0055] This invention also discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method described in the preceding embodiment.

[0056] This invention also discloses a non-transient computer-readable storage medium having instructions embodied thereon that can be executed by one or more processors to perform the method described in any of the preceding embodiments.

[0057] Although the invention has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various modifications in form and detail may be made without departing from the spirit and scope of the invention. Accordingly, the disclosed invention should be considered merely illustrative and limited only by the scope specified in the appended claims.

Claims

1. An image classification method, characterized in that, Includes the following steps: S1: Collect an image dataset and convert it into multiple feature matrices with rows equal to the number of samples and columns equal to the number of features. ; In the formula, It is a dataset containing multiple views. It is the number of views. Indicates the first Feature matrix of each view; It represents the samples contained in each view. Represents the number of samples. This indicates the first in the view. One sample; S2, the feature matrices of different views are sequentially input into the particle generation model, and edges are adaptively added to the generated results based on particle density and distance to obtain the corresponding adjacency matrix. ; S3, sort the feature matrix according to the number of features to obtain and its corresponding adjacency matrix The data are fed into a convolutional model for computation; the view with the most features is convolved first. S4. The feature matrices and adjacency matrices of the remaining views are fed into the convolution operation, and the embedding representations of each layer of the previous view are fused with the embedding representations of the same layer of the next view, which are then used as the input for the next convolution operation. Finally, classification prediction is performed. Step S4 specifically includes: S41: Perform convolution operations on the remaining sorted views one by one until the view with the fewest features completes the convolution operation; S42: After the convolution process, obtain the node representation of each view, weight and sum the final convolution results of each view, then normalize them, and perform classification prediction. S43: Use the cross-entropy loss function for evaluation and backpropagate to update the weight matrix; The backpropagation update of the weight matrix includes: Equivalent features are extracted from the feature matrix and adjacency matrix. The update gradient of the weight matrix is ​​adjusted based on these equivalent features. These equivalent features are quantized features that fuse the attribute information of the feature matrix and the topological association information of the adjacency matrix. Based on the equivalent feature matrix, the update gradient of the weight matrix is ​​dynamically determined. Specifically: 1) Calculate the feature weights at the particle level For the v-th view after sorting ( ), each sphere Feature weights The discriminative power, reflecting the characteristics of samples within the granule, is calculated using the following formula: ;in, Particle density; For granules Standard deviation of all sample characteristics; 2) Generate sample-level equivalent feature vectors For the nth sample of the vth view If it belongs to granules Then the equivalent feature vector of this sample is : ； in, For granules The characteristic mean of all samples within the sample; 3) Construct the view-level equivalent feature matrix The equivalent feature matrix of the v-th view The set of equivalent feature vectors for all samples: Next, the traditional gradient is calculated based on the cross-entropy loss, and the auxiliary gradient is calculated based on the equivalent feature matrix mentioned above. The traditional gradient and the auxiliary gradient are then weighted and fused to obtain the total gradient. The auxiliary gradient is based on the equivalent feature matrix. In the design, first define the auxiliary loss function: Auxiliary loss = Feature consistency loss + λ × Topology consistency loss; λ is the topology constraint coefficient; Feature consistency loss = Topology consistency loss = ; The output feature matrix of the l-th convolutional layer is a key intermediate result for feature extraction and cross-view fusion. The auxiliary gradient is the partial derivative of the auxiliary loss with respect to the weight matrix W, i.e. , For granular-level feature weights The weight matrix is ​​formed.

2. The image classification method according to claim 1, characterized in that: Step S2 specifically includes: S21: Obtain the sphere clustering results and calculate the sphere center and radius by using an unsupervised sphere generation method; S22: Calculate the density of each granule and its distance from other granules based on the density peak algorithm; S23: For each sphere, select the sphere with a higher density and the closest distance, and perform a full connection operation across the two spheres.

3. The image classification method according to claim 2, characterized in that: Step S3 specifically includes: S31: Sort the different views in descending order based on the number of features, with the view with the most features placed first; S32: The feature matrix of the sorted first view and adjacency matrix Insert convolution operations, record the output of each convolutional layer, and store it.

4. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method described in any one of claims 1-3.

5. A non-transient computer-readable storage medium having instructions embodied thereon, characterized in that, The instructions can be executed by one or more processors to perform the method for implementing any one of claims 1-3.

6. A computer program product comprising an executable computer program, characterized in that: When the computer program is executed, it implements the method as described in any one of claims 1-3.

Images