An image classification method based on hyper-dimensional computing
By optimizing the hyperdimensional calculation method through PCA dimensionality reduction and EMA smoothing techniques, the problems of feature dimension curse and hypervector instability in complex high-dimensional data are solved, achieving efficient image classification, improving classification accuracy and stability, and expanding application scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 58TH RES INST OF CETC
- Filing Date
- 2026-04-21
- Publication Date
- 2026-07-14
Smart Images
Figure CN122391724A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data encoding and processing technology, and in particular to an image classification method based on hyperdimensional computation. Background Technology
[0002] Hyperdimensional computing (HDC) is a biologically rational computing paradigm inspired by the brain's cognitive mechanisms. This paradigm maps input data to a high-dimensional hyperspace (typically with thousands of dimensions) and performs symbolic representation and computation. It features low computational complexity, high robustness, and strong parallelism, and has attracted widespread attention in classification tasks on low-resource devices such as edge computing.
[0003] In low-dimensional, simple data scenarios, existing hyperdimensional computation methods have achieved good classification results. For example, on the MNIST handwritten digit dataset (28×28 grayscale image, 784-dimensional features), existing methods, represented by OnlineHD, can achieve a classification accuracy of over 97%, verifying the effectiveness of hyperdimensional computation in simple classification tasks.
[0004] However, in practical applications, many classification tasks involve complex, high-dimensional data, such as the CIFAR-10 color image dataset (32×32×3 RGB images, 3072-dimensional features). In such complex, high-dimensional data scenarios, existing hyperdimensional computation methods exhibit significant performance limitations. For example, OnlineHD's classification accuracy on the CIFAR-10 dataset is only around 16%, which is insufficient for practical application requirements.
[0005] Analysis reveals that existing hyperdimensional computation methods have the following main shortcomings when dealing with complex high-dimensional data: First, there is the curse of dimensionality. The high-dimensional original features of complex datasets contain a large amount of spatial redundancy and noise. Directly encoding in superdimensional space can easily lead to sparse distribution of hypervectors in the hyperspace, which reduces the discriminative power of the hypervectors and makes it difficult for the model to effectively distinguish between different categories of data features.
[0006] Second, hypervector representations lack stability. Existing hyperdimensional computation methods (such as OnlineHD) employ an online iterative update mechanism, which is highly sensitive to noisy samples. During training, hypervectors are prone to fluctuations due to noise, affecting the model's convergence speed and generalization performance.
[0007] Third, feature preprocessing and hypervector optimization lack coordinated design. Existing methods have not designed effective feature preprocessing mechanisms for the characteristics of high-dimensional and complex data, nor have they proposed targeted optimization schemes for the hypervector fluctuation problem during the iterative update process. This results in feature quality and hypervector representation capability mutually restricting each other, making it difficult to fully leverage the advantages of hyperdimensional computation.
[0008] Currently, there is a lack of effective solutions to address the performance bottleneck of hyperdimensional computation in classifying complex high-dimensional data. How to improve the adaptability and classification performance of hyperdimensional computation on complex datasets through collaborative optimization of feature processing and hypervector update mechanisms has become a pressing technical problem in this field. Summary of the Invention
[0009] To address the aforementioned technical problems, this invention provides an image classification method based on hyperdimensional computation, comprising the following steps: Step S1: Obtain the training dataset and the test dataset; Step S2: Perform principal component analysis preprocessing on the training dataset and the test dataset respectively to remove redundant information and noise, and obtain the dimensionality-reduced low-dimensional core features; Step S3: Map the dimensionality-reduced training features and test features to the hyperspace to generate hypervector representations; Step S4: Initialize the class hypervector and optimize and update the class hypervector using an error-driven online iterative mechanism; Step S5: Introduce the exponentially weighted moving average smoothing technique to smooth the updated class hypervectors, and combine it with the gradient clipping strategy to suppress the fluctuation of the class hypervectors; Step S6: Use the smoothed class hypervector to perform classification prediction on the test hypervector and output the classification result.
[0010] In one embodiment of the present invention, the specific process of principal component analysis preprocessing in step S2 includes: Calculate the mean vector of the training dataset, and center the training and test datasets respectively; Calculate the covariance matrix of the centered training data and perform eigenvalue decomposition to obtain eigenvalues and eigenvectors; Arrange the eigenvalues in descending order, and select the eigenvectors corresponding to the k largest eigenvalues to form the projection matrix, where k is the feature dimension after dimensionality reduction; The dimensionality of the centered training and test data is compressed using the projection matrix to obtain the dimensionality-reduced features.
[0011] In one embodiment of the present invention, the feature dimension k after dimensionality reduction is determined by retaining more than 95% of the variance of the original data.
[0012] In one embodiment of the present invention, the specific process of generating the hypervector representation in step S3 includes: Define the hyperspace dimension M, and initialize the random projection matrix and bias vector; For each dimension-reduced feature vector, a hypervector is generated through random projection and nonlinear transformation, where the formula for the nonlinear transformation is: Where H is the random projection matrix, B is the bias vector, z is the eigenvector after dimensionality reduction, and h is the generated hypervector.
[0013] In one embodiment of the present invention, the specific process of optimizing and updating the class hypervector using an error-driven online iterative mechanism in step S4 includes: Calculate the cosine similarity between each batch of hypervectors and all other types of hypervectors; Based on similarity, determine whether the predicted category of each sample is consistent with the true category, and identify misclassified samples; Calculate the update coefficients of the class hypervectors for misclassified samples, where the update coefficients include the enhancement coefficient of the correct class and the suppression coefficient of the incorrect class; The correct class hypervector and the incorrect class hypervector corresponding to the misclassified sample are updated according to the update coefficient.
[0014] In one embodiment of the present invention, the formulas for calculating the enhancement coefficient of the correct class and the suppression coefficient of the incorrect class are as follows: Enhancement coefficient:
[0015] Inhibition coefficient:
[0016] Where scores is the cosine similarity between the batch hypervector and each type of hypervector, and wrong is the label of the incorrect sample. For real category labels, To predict the category label.
[0017] In one embodiment of the present invention, the calculation formula for the exponentially weighted moving average smoothing process in step S5 is as follows: in, This is a smoothing coefficient, with a value range of 0 < < 1, Let be the class hypervector after the t-th iteration. Let be the smoothed supervector after the t-th iteration.
[0018] In one embodiment of the present invention, the specific process of the gradient clipping strategy in step S5 includes: Calculate the difference between the current class hypervector and the previous smoothed hypervector; perform gradient clipping on the difference, using the following formula: in, The cropping threshold, The 2-norm of the difference is used; the clipped class hypervector is used as the input for the next exponentially weighted moving average smoothing.
[0019] In one embodiment of the present invention, the specific process of using the smoothed class hypervector to perform classification prediction on the test hypervector in step S6 includes: Calculate the cosine similarity between each test hypervector and each smooth class hypervector; convert the similarity into class probability using the softmax function; select the class with the highest probability as the predicted label for the test sample.
[0020] In one embodiment of the present invention, the principal component analysis preprocessing in step S2 is replaced by linear discriminant analysis, local linear embedding, or autoencoder; the nonlinear transformation in step S3 is replaced by sigmoid function, tanh function, or ReLU function; and the exponentially weighted moving average smoothing in step S5 is replaced by moving average smoothing or Gaussian smoothing.
[0021] Compared with the prior art, the above-mentioned technical solution of the present invention has the following advantages: The image classification method of the present invention provides an efficient feature preprocessing scheme, which removes redundant information and noise in high-dimensional data by PCA dimensionality reduction technology, retains core features, solves the curse of feature dimensionality problem, and provides a compact and highly discriminative feature representation for hyperdimensional encoding.
[0022] Furthermore, a stable hypervector update optimization mechanism is designed, which uses EMA smoothing technology to suppress the fluctuation of hypervector-like structures during online iteration, thereby improving the stability and reliability of hypervector representation and enhancing the convergence and generalization ability of the model.
[0023] To achieve the synergistic effect of feature preprocessing and hypervector optimization, a complete hyperdimensional computation classification framework is constructed, namely a hyperdimensional computation classification framework (PE-OnlineHD) that integrates PCA feature purification and EMA representation smoothing co-optimization. This significantly improves the classification accuracy and performance stability of hyperdimensional computation methods on complex high-dimensional datasets and expands the application scenarios of hyperdimensional computation in complex classification tasks on low-resource devices such as edge computing. Attached Figure Description
[0024] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0025] Figure 1 This is a block diagram illustrating the principle of the PE-OnlineHD hyperdimensional image classification method proposed in this invention. Figure 2 This is a flowchart of the steps of the image classification method based on hyperdimensional computation in this invention. Detailed Implementation
[0026] like Figure 1 As shown, the image classification method provided in this embodiment consists of three core modules: a PCA preprocessing module, a hyperdimensional encoding and iterative optimization module, and an EMA smoothing module. First, the PCA preprocessing module reduces the dimensionality of the high-dimensional raw data, removing redundancy and noise to obtain compact core features. Next, hyperdimensional encoding maps the reduced features to a hyperspace, and an error-driven online iterative mechanism optimizes the class hypervectors. Finally, the EMA smoothing module smooths the iteratively updated class hypervectors, suppressing fluctuations and improving stability.
[0027] like Figure 2 As shown, this embodiment provides an image classification method based on hyperdimensional computation, which includes the following steps: Step 1: Dataset Acquisition. Obtain the training and test datasets for the hyperdimensional computation classification task. The training dataset is denoted as... ,in D represents the number of training samples, and D represents the original feature dimension (e.g., D=3072 in the CIFAR-10 dataset). Each training sample contains a corresponding class label. C is the total number of categories (e.g., C=10 for the CIFAR-10 dataset); the test dataset is denoted as... ,in The number of test samples, and the category label for each test sample. This invention utilizes a training dataset to train a hyperdimensional computational model, which then performs classification and prediction on unlabeled samples in the test dataset.
[0028] Step 2: PCA Preprocessing (Feature Dimensionality Reduction). Perform PCA preprocessing on both the training and test datasets to remove redundant information and noise, obtaining low-dimensional core features. The specific process is as follows: (1) Data centralization: Calculate the mean vector of the training dataset. ,in For the training set The original feature vectors of each sample. The training and test datasets are then centered separately: Centralized training data:
[0029] Centralized test data: (Use the mean vector of the training set to ensure consistent data distribution) (2) Covariance matrix decomposition: Calculate the centered training data covariance matrix For the covariance matrix Perform eigenvalue decomposition to obtain the eigenvalue matrix. and eigenvector matrix ,Right now .
[0030] (3) Principal component selection: Arrange the eigenvalues in descending order and select the eigenvectors corresponding to the largest eigenvalues to form the projection matrix. ,in k The feature dimensions after dimensionality reduction are determined by retaining more than 95% of the variance of the original data (e.g., k=150 for the CIFAR-10 dataset). For the dimensionality reduction of the first 1 eigenvector.
[0031] (4) Dimension compression: The centered training data and test data are compressed using projection moments.
[0032] Dimensionality compression is performed to obtain the reduced-dimensional features: Dimensionality reduction training characteristics:
[0033] Dimensionality reduction test features:
[0034] PCA preprocessing significantly reduces feature dimensionality while preserving core data information, solving the redundancy and noise problems of high-dimensional data and providing high-quality input features for subsequent hyperdimensional encoding.
[0035] Step 3: Hyperdimensional encoding. This involves encoding the dimensionality-reduced training features... and test features Each vector is mapped to a hyperspace to generate a hypervector representation. The specific process is as follows: Hyperspace parameter initialization: Set the hyperspace dimension to 1. (This invention selects) =4000), initialize the random projection matrix ,in The elements follow a standard Gaussian distribution. Initialize the bias vector Its elements follow a uniform distribution .
[0036] (2) Hypervector generation: For each dimension-reduced feature vector (Including training features and test features), generate hypervectors through random projection and nonlinear transformation. The calculation formula is as follows: in, and For element-wise trigonometric functions, nonlinear transformations are used to enhance the discriminative power of hypervectors. Let be the dimension of the hypervector. The generated set of training hypervectors is denoted as . The set of test hypervectors is denoted as . .
[0037] Step 4, Hyperdimensional Iterative Optimization (Error-Driven Update). Initialize the class hypervector matrix. (Initially a vector of 0), an error-driven online iterative mechanism is used to optimize and update the class hypervectors. For each batch of samples in the training set... ( To train hypervectors in batches, Perform the following operations for the corresponding category label: (1) Similarity calculation: Calculate the cosine similarity between each batch of hypervectors and all types of hypervectors: in , This refers to the batch sample size. Indicates the first The batch of samples and the first Similarity of class hypervectors.
[0038] Error detection: Determine whether the predicted class of each sample matches the true class, and identify misclassified samples. (Prediction Category) (Error sample marker,) Indicates the first (One sample was misclassified) (3) Update coefficient calculation: Calculate the update coefficients of the class hypervectors for misclassified samples to enhance the correct class hypervectors and suppress the incorrect class hypervectors. Enhancement coefficient (correct class):
[0039] Suppression coefficient (error class):
[0040] (4) Class hypervector update: Update the correct class hypervector and incorrect class hypervector corresponding to the misclassified sample according to the update coefficient: Through iterative updates, the class hypervectors gradually approach the core features of the corresponding class, thereby improving the inter-class discrimination.
[0041] Step 5: EMA Smoothing Optimization. To address the fluctuation problem of the hypervector class during iteration, the Exponential Moving Average (EMA) smoothing technique is introduced to optimize the updated hypervector class. Simultaneously, a gradient clipping strategy is combined to ensure update stability. EMA smooth update: Let the first The supervectors after the next iteration are EMA-smoothed hypervectors The calculation formula is as follows: in, Smoothing coefficient (value range 0 < < 1), control the weight distribution between the historical smoothed hypervector and the currently updated hypervector, (The initial smooth hypervector is the initial class hypervector).
[0042] Gradient clipping: Calculates the difference between the current class hypervector and the previous smoothed hypervector. Gradient clipping is applied to the differences to prevent excessively large update amplitudes from causing fluctuations. , in, The cropping threshold (selected in this invention) =0.5), for The 2-norm. The clipped class hypervector is updated as follows: and will As input for the next EMA smoothing By leveraging the synergistic effect of EMA smoothing and gradient clipping, the volatility of hypervectors is significantly reduced, improving the stability and convergence of the model.
[0043] The entire model learns without employing the complex weight update rules of neural networks (such as gradient descent), only requiring the processing of the class hypervector matrix in the hyperdimensional representation. It performs online iterative optimization, thus having lower training complexity and faster convergence compared to neural networks.
[0044] Step 6: Classification result prediction. Use the optimized smooth class hypervectors. For test hypervectors Perform classification prediction: Similarity calculation: Calculate the cosine similarity between each test hypervector and each type of smooth hypervector. (2) Probability transformation: The similarity is transformed into class probability using the softmax function: in, For the first One test hypervector, This test sample belongs to the first The probability of a class.
[0045] (3) Category prediction: Select the category with the highest probability as the predicted label for the test sample: The image classification method described in this embodiment has at least the following advantages or beneficial effects: 1. Effectively solves the curse of feature dimensionality problem: This invention introduces a PCA preprocessing module, which significantly reduces the feature dimensionality of high-dimensional data by feature decomposition and principal component selection, while retaining more than 95% of the original data variance. It also eliminates redundant information and noise, providing a compact and highly discriminative feature representation for hyperdimensional encoding. This fundamentally alleviates the problem of sparse hypervector distribution caused by high-dimensional data and improves the discriminative ability of hypervectors.
[0046] 2. Improve the stability of hypervector representation: Through the synergistic effect of the EMA smoothing module and the gradient clipping strategy, the drastic fluctuations of class hypervectors caused by noise samples during online iterative updates are effectively suppressed, making the updates of class hypervectors smoother and more stable, enhancing the convergence and generalization ability of the model, and solving the core defect of unstable hypervector representation in traditional methods.
[0047] 3. Achieving Synergistic Effects of Feature Processing and Hypervector Optimization: This invention innovatively constructs an integrated collaborative framework of "PCA dimensionality reduction - hyperdimensional encoding - EMA smoothing," with each module working collaboratively. Specifically, the PCA dimensionality reduction module purifies and removes redundancy from the original high-dimensional data, providing low-noise, high-discrimination input for subsequent hyperdimensional encoding, effectively reducing the computational complexity of hyperdimensional encoding; the EMA smoothing module dynamically regulates the stability of the hypervector iterative optimization process, suppressing parameter fluctuations during iteration and ensuring the convergence and reliability of hypervector optimization, ultimately significantly improving the classification accuracy and computational efficiency of hyperdimensional computation on complex high-dimensional datasets. The experimental results are shown in Table 1. On the CIFAR-10 dataset, the PE-OnlineHD method proposed in this invention improves the classification accuracy by more than 6% compared with the traditional OnlineHD method, and the inference latency is only 1 / 6 of that of the traditional OnlineHD method, showing significant advantages in both classification accuracy and computational efficiency. To clarify the technical contribution of each module, a control experiment was set up, where P-OnlineHD is a control method that combines PCA dimensionality reduction with OnlineHD, and E-OnlineHD is a control method that combines EMA smoothing with OnlineHD.
[0048] Table 1. Performance comparison of PE-OnlineHD (pca_dim=100) and OnlineHD on the CIFAR-10 dataset Maintaining the advantage of low computational complexity: The computational process of PCA dimensionality reduction and EMA smoothing is simple and efficient, without the need for complex neural network structures and a large number of parameter training. It is compatible with the low complexity characteristics of hyperdimensional computing itself, ensuring the deployability of the method on low-resource devices such as edge computing, and expanding the application scenarios of hyperdimensional computing.
[0049] Furthermore, in addition to the above-described implementation scheme, some technical details in this technical solution can be replaced by the following alternative solutions: 1. Feature Dimensionality Reduction Method Alternatives: The PCA preprocessing module can be replaced with other efficient dimensionality reduction methods, such as Linear Discriminant Analysis (LDA), Locally Linear Embedding (LLE), or Autoencoders. LDA is suitable for scenarios with high class discriminative requirements, further improving the inter-class separation of the reduced features. Autoencoders are suitable for non-linear data scenarios, capable of learning more complex feature mapping relationships, but with slightly increased computational complexity; the choice should be made based on the resource constraints of the specific application scenario.
[0050] 2. Replacement of Hyperdimensional Encoding Methods: The nonlinear transformation in the hyperdimensional encoding process can be replaced with other forms, such as the sigmoid function, tanh function, or ReLU function, i.e. or Different nonlinear transformation functions have a certain impact on the distribution characteristics of hypervectors, and can be adapted and selected according to the feature distribution of the dataset; in addition, the hyperspace dimension It can be adjusted according to the complexity of the data. A higher hyperspace dimension may improve discriminative power, but it will increase computation and storage overhead.
[0051] 3. Smoothing Strategy Alternatives: The EMA smoothing module can be replaced with moving average (MA) smoothing or Gaussian smoothing. Moving average smoothing is simple to calculate and suitable for scenarios with high real-time requirements; Gaussian smoothing further enhances the smoothing effect through weighted averaging, but its computational complexity is slightly higher than EMA. The choice can be made based on the model's stability requirements; smoothing coefficient... With gradient clipping threshold Cross-validation can also be used to dynamically adjust the settings to suit the characteristics of different datasets.
[0052] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.
Claims
1. An image classification method based on hyperdimensional computation, characterized in that, Includes the following steps: Step S1: Obtain the training dataset and the test dataset; Step S2: Perform principal component analysis preprocessing on the training dataset and the test dataset respectively to remove redundant information and noise, and obtain the dimensionality-reduced low-dimensional core features; Step S3: Map the dimensionality-reduced training features and test features to the hyperspace to generate hypervector representations; Step S4: Initialize the class hypervector and optimize and update the class hypervector using an error-driven online iterative mechanism; Step S5: Introduce the exponentially weighted moving average smoothing technique to smooth the updated class hypervectors, and combine it with the gradient clipping strategy to suppress the fluctuation of the class hypervectors; Step S6: Use the smoothed class hypervector to perform classification prediction on the test hypervector and output the classification result.
2. The image classification method based on hyperdimensional computation according to claim 1, characterized in that, The specific process of principal component analysis preprocessing in step S2 includes: Calculate the mean vector of the training dataset, and center the training and test datasets respectively; Calculate the covariance matrix of the centered training data and perform eigenvalue decomposition to obtain eigenvalues and eigenvectors; Arrange the eigenvalues in descending order, and select the eigenvectors corresponding to the k largest eigenvalues to form the projection matrix, where k is the feature dimension after dimensionality reduction; The dimensionality of the centered training and test data is compressed using the projection matrix to obtain the dimensionality-reduced features.
3. The image classification method based on hyperdimensional computation according to claim 2, characterized in that, The feature dimension k after dimensionality reduction is determined by retaining more than 95% of the variance of the original data.
4. The image classification method based on hyperdimensional computation according to claim 1, characterized in that, The specific process of generating the hypervector representation in step S3 includes: Define the hyperspace dimension M, and initialize the random projection matrix and bias vector; For each dimension-reduced feature vector, a hypervector is generated through random projection and nonlinear transformation, where the formula for the nonlinear transformation is: Where H is the random projection matrix, B is the bias vector, z is the eigenvector after dimensionality reduction, and h is the generated hypervector.
5. The image classification method based on hyperdimensional computation according to claim 1, characterized in that, The specific process of optimizing and updating the class hypervector using the error-driven online iterative mechanism in step S4 includes: Calculate the cosine similarity between each batch of hypervectors and all other types of hypervectors; Based on similarity, determine whether the predicted category of each sample is consistent with the true category, and identify misclassified samples; Calculate the update coefficients of the class hypervectors for misclassified samples, where the update coefficients include the enhancement coefficient of the correct class and the suppression coefficient of the incorrect class; The correct class hypervector and the incorrect class hypervector corresponding to the misclassified sample are updated according to the update coefficient.
6. The image classification method based on hyperdimensional computation according to claim 5, characterized in that, The formulas for calculating the enhancement coefficient of the correct class and the inhibition coefficient of the incorrect class are as follows: Enhancement coefficient: Inhibition coefficient: Where scores is the cosine similarity between the batch hypervector and each type of hypervector, and wrong is the label of the incorrect sample. For real category labels, To predict the category label.
7. The image classification method based on hyperdimensional computation according to claim 1, characterized in that, The calculation formula for the exponentially weighted moving average smoothing process in step S5 is as follows: in, This is a smoothing coefficient, with a value range of 0 < < 1, Let be the class hypervector after the t-th iteration. Let be the smoothed supervector after the t-th iteration.
8. The image classification method based on hyperdimensional computation according to claim 7, characterized in that, The specific process of the gradient clipping strategy in step S5 includes: Calculate the difference between the current class hypervector and the previous smoothed hypervector; perform gradient clipping on the difference, using the following formula: in, The cropping threshold, The 2-norm of the difference is used; the clipped class hypervector is used as the input for the next exponentially weighted moving average smoothing.
9. The image classification method based on hyperdimensional computation according to claim 1, characterized in that, The specific process of using the smoothed class hypervector to classify and predict the test hypervector in step S6 includes: Calculate the cosine similarity between each test hypervector and each smooth class hypervector; convert the similarity into class probability using the softmax function; select the class with the highest probability as the predicted label for the test sample.
10. The image classification method based on hyperdimensional computation according to claim 1, characterized in that, The principal component analysis preprocessing in step S2 is replaced by linear discriminant analysis, local linear embedding, or autoencoder; the nonlinear transformation in step S3 is replaced by sigmoid function, tanh function, or ReLU function; and the exponentially weighted moving average smoothing in step S5 is replaced by moving average smoothing or Gaussian smoothing.