Implementation method of a lightweight hyper-dimensional computing classification model

Abstract

The present invention discloses a method for implementing a lightweight hyperdimensional computing classification model. The method comprises the following steps: firstly determining the data volume, data value range and hyperdimensional vector dimension of a sample to be learned, dividing an address memory and a continuous item memory in a hardware memory, determining the number of numerical samples, obtaining an address hyperdimensional vector, and obtaining a numerical hyperdimensional vector by negating a NAND gate; for an input sample, firstly binding the address hyperdimensional vector of the position of each data point with the numerical hyperdimensional vector corresponding to the numerical value of the data point by multiplication operation, and then forming all data points into a set by addition operation and performing binarization processing; when the sample is a training sample, adding it to the category hyperdimensional vector corresponding to the category by addition operation, and when the sample is a test sample, performing a bit-by-bit comparison with the category hyperdimensional vector to obtain the category to which it belongs; the present invention can effectively realize online, fast, incremental, real-time learning and high-accuracy recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperdimensional computing, and mainly relates to a method for implementing a lightweight hyperdimensional computing classification model. Background Art

[0002] Hyperdimensional computing is an emerging cognitive model inspired by the working mechanism of the hippocampus in the brain to generate and store memories. It regards the generation and recall of memories as a high-dimensional global random mapping and similarity matching. In recent years, hyperdimensional computing has become an effective method for on-chip, real-time, incremental learning and recognition due to its advantages such as fast learning process, independence from large amounts of data drive, good model interpretability, lightweight model, high energy efficiency, and excellent robustness, and has been well applied in many fields such as speech recognition, language recognition, handwritten digit recognition, human pose recognition, bio-signal processing, DNA pattern matching, multi-task learning, multi-sensor information fusion, and brain-computer interface.

[0003] Hyperdimensional computing uses the high-dimensional distributed representation of data as the recognition object, and encodes different types of information into hyperdimensional vectors, that is, vectors with thousands or more dimensions. Information is randomly and globally distributed on each element of the hyperdimensional vector, and is uniquely referred to and represented by the hyperdimensional vector. The commonly used operations in hyperdimensional computing are bitwise addition, bitwise multiplication, and shift. Among them, the addition operation retains the information of each element, and the operation result remains similar to each element, so it is often used to represent a set with multiple elements; the multiplication operation binds two hyperdimensional vectors and maps them as a whole to another position in the high-dimensional space, and the distance between the hyperdimensional vectors remains unchanged during the process, so it is often used to bind a group of addresses and values or independent variables and dependent variables; the shift operation can retain the element order, so it is often used to retain order information such as values in a sequence, letters in a word, etc., and increase the information representation ability of the hyperdimensional vector. In the high-dimensional space, various computational behaviors with special mathematical properties can be realized through operations on hyperdimensional vectors, thereby generating the functions of learning and memory. These operations and operation combinations have good hardware friendliness, extremely low requirements for hardware performance, simple calculations, convenient deployment, and fast operations. Especially compared with the neural network method based on gradient descent and iterative operations, hyperdimensional computing can achieve excellent performance close to advanced machine learning algorithms with extremely low computational costs and ultra-fast learning processes. At the same time, in special cases such as few available samples and large noise interference, the hyperdimensional computing model can still maintain stable working performance. When new training samples are available, the existing model can be directly and incrementally updated through simple multiplication and addition operations to make the model more accurate, without the need to retrain the entire model.

[0004] Among the currently commonly used classification and recognition algorithms and models, traditional classification algorithms such as support vector machines, Bayesian classifiers, decision trees, and K-nearest neighbor algorithms achieve class division based on the distribution and statistical laws of the data itself, with limited accuracy and low intelligence; while machine learning algorithms based on deep neural networks, while achieving high recognition accuracy, due to the large number of parameters, complex floating-point iterative operations, and large-scale model structures, have high requirements for hardware performance, and the training is time-consuming and complex, making it difficult to deploy in integrated circuits or embedded systems. Generally speaking, hyperdimensional computing can well achieve the optimal compromise between recognition accuracy and efficiency. Summary of the Invention

[0005] Object of the Invention: Based on the research direction in the above background technology, the present invention provides a method for implementing a lightweight hyperdimensional computing classification model. By using simple logic gates, counters, random number generators, and hardware memories, a lightweight on-chip cognitive model is constructed, which can complete online, fast, incremental, and real-time learning of training samples, and fast classification and recognition of test samples; without complex and time-consuming backend processing such as long-distance data transmission and large-scale model parameter training, it can be deployed simply and quickly in integrated circuits or embedded systems.

[0006] Technical Solution: To achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for implementing a lightweight hyperdimensional computing classification model, comprising the following steps:

[0008] Step S1: Determine the data volume and data value range of the samples to be learned; denote the number of data points as N and the value range as [min, max]; determine the dimension D of the hyperdimensional vector;

[0009] Step S2: Divide an address memory with an N×D-bit space in the hardware memory, and randomly assign a binary number 0 or 1 to each bit address; denote the hyperdimensional vector with a size of 1×D for each row as H n , n = 1, 2,..., N, which is the address hyperdimensional vector, representing the position of the nth data point among N data points;

[0010] Step S3: Determine the number of numerical sampling points M, and equally divide the value range [min, max] into M levels; the value Y represented by the mth level is as follows: m is expressed as:

[0011]

[0012] where m = 1, 2,..., M;

[0013] Step S4: Divide a continuous item memory with an M×D-bit space in the hardware memory, and assign a binary number 0 or 1 to each bit of the 1×D hyperdimensional vector in the first row, denoted as L1; L1 corresponds to the lower bound min of the value range;

[0014] Step S5: Randomly select D / 2 / (M - 1) bits from L1, invert them through a NOT gate to represent the next value, and each element is flipped only once; repeat the operation until all M-level values are represented; at this time, it is repeated (M - 1) times to obtain the value hyperdimensional vector L m , m = 1, 2,..., M; L m represents the m-level value Y in the M-level value range m ; The number of different bits between any two value hyperdimensional vectors L i and L j , i, j ∈ {1, 2,..., M}, depends on the difference between Y i and Y j ; In particular, the number of different bits between L1 and L M is D / 2, that is, L1 and L M are orthogonal;

[0015] Step S6: After inputting a sample, encode it based on the hyperdimensional vectors in the address memory and the continuous item memory obtained in Steps S1 - S5 as follows:

[0016]

[0017] V n ∈ {L1, L2,..., L M}, n = 1, 2,..., N

[0018] where H n is the address hyperdimensional vector in the address memory; V n is the value hyperdimensional vector corresponding to the value of the nth data point in the continuous item memory; [[·]] represents binarization processing; R is the sample hyperdimensional vector finally obtained in the encoding process;

[0019] After the above encoding, obtain the number of 1s in each bit through a counter and compare it with N / 2. When the number of 1s in this bit is greater than N / 2, the operation result of this bit takes 1, otherwise it takes 0; after obtaining the encoding result R, when this sample is a training sample, go to Step S7; when this sample is a test sample, go to Step S8;

[0020] Step S7: After obtaining the sample hyperdimensional vector R of the training sample, record it as k according to the category to which the label belongs, and add it to the category hyperdimensional vector C k corresponding to the category through an addition operation, specifically as follows:

[0021] Ck +=R

[0022] where k = 1, 2, ..., K, and K is the total number of categories;

[0023] Step S8, after obtaining the sample hyper-dimensional vector R of the test sample, perform a bit-by-bit comparison with the binarized category hyper-dimensional vector [[C k ; the category represented by [[C k that has the largest number of identical elements with R is the category to which the test sample belongs.

[0024] Advantageous effects:

[0025] The implementation method of a lightweight hyper-dimensional computing classification model provided by the present invention realizes ultra-fast, high-accuracy, on-chip real-time learning and recognition of information through simple logic gate circuits, counters, random number generators, and hardware memories in an integrated, miniaturized, and low-power hardware circuit. It is easy to deploy in integrated circuits or embedded systems, and the device has a small volume, low cost, and simple on-site debugging.

[0026] The present invention gives full play to the advantages of the lightweight hyper-dimensional computing classification and recognition algorithm, realizes and deploys the model through a hardware circuit, and achieves online, fast, incremental, real-time learning and high-accuracy recognition at an extremely low computing cost. The algorithm idea is relatively simple, facilitating the application implementation in engineering and production. Description of the Drawings

[0027] Figure 1 is a flowchart of the implementation method of the lightweight hyper-dimensional computing classification model provided by the present invention;

[0028] Figure 2 is a hardware circuit diagram of the lightweight hyper-dimensional computing classification model provided by the present invention. Detailed Embodiments

[0029] The following further describes the present invention with reference to the drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0030] As Figure 1-2 shown, an implementation method of a lightweight hyper-dimensional computing classification model includes the following specific steps:

[0031] Step S1, determine the data volume and data value range of the samples to be learned; denote the number of data points as N and the value range as [min, max]; determine the dimension D of the hyper-dimensional vector;

[0032] Step S2: Divide an address memory with an N×D-bit space in the hardware memory, and randomly assign a binary number 0 or 1 to each bit of the address by a random number generator; Denote a hyperdimensional vector with a size of 1×D per row as H n , n = 1, 2, ..., N, which is the address hyperdimensional vector and represents the position of the nth data point among N data points; Since the values are randomly selected, the number of different bits between any two different address hyperdimensional vectors is close to D / 2 in probability. In the hyperdimensional computing model, two hyperdimensional vectors with the number of different bits close to D / 2 are defined as orthogonal to each other; The closer the number of different bits is to 0, the more similar the two hyperdimensional vectors are; The closer to D, the more different bits the two hyperdimensional vectors have. Due to the high dimension, it is almost impossible for different address hyperdimensional vectors to be the same. Therefore, H n is independent and unique.

[0033] Step S3: Determine the number M of numerical sampling points, and equally divide the value range [min, max] into M levels; The value Y m represented by the mth level is as follows:

[0034]

[0035] where m = 1, 2, ..., M;

[0036] Step S4: Divide a continuous item memory with an M×D-bit space in the hardware memory, and assign a binary number 0 or 1 to each bit of the 1×D hyperdimensional vector in the first row by a random number generator, denoted as L1; L1 corresponds to the lower bound min of the value range;

[0037] Step S5: Randomly select D / 2 / (M - 1) bits in L1, invert them through a NOT gate to represent the next value, and each element is flipped only once; Repeat the operation until all M levels of values are represented; At this time, it is repeated (M - 1) times in total to obtain the numerical hyperdimensional vector L m , m = 1, 2, ..., M; L m represents the value Y m in the mth level of the M-level value range; The number of different bits between any two numerical hyperdimensional vectors L i and L j , i, j ∈ {1, 2, ..., M}, depends on the difference between Y i and Y j ; In particular, the number of different bits between L1 and L M is D / 2, that is, L1 and L M are orthogonal;

[0038] Step S6: When a sample is input, encode it based on the hyperdimensional vectors in the address memory and the continuous item memory obtained in Steps S1 - S5 as follows:

[0039]

[0040] V n ∈ {L1, L2, ..., L M}, n = 1, 2, ..., N

[0041] where H n is the address hyperdimensional vector in the address memory; V n is the value hyperdimensional vector corresponding to the value of the nth data point in the continuous item memory; [[·]] represents binarization processing; R is the sample hyperdimensional vector finally obtained in the encoding process;

[0042] First, bind the address hyperdimensional vector at the location of each data point to the value hyperdimensional vector corresponding to the value of the data point through multiplication operation, and then form a set of all data points through addition operation. Since the result obtained by addition is non-binary, binarization processing is required.

[0043] The bit-by-bit binary multiplication operation of two D-dimensional hyperdimensional vectors can be implemented by XOR gates in the hardware circuit. The operation of bit-by-bit addition of N D-dimensional hyperdimensional vectors followed by binarization processing can obtain the number of 1s in each bit through a counter. Compare the number of 1s in each bit obtained by the counter with N / 2. When the number of 1s in this bit is greater than N / 2, the operation result of this bit is 1, otherwise it is 0; after obtaining the encoding result R, when the sample is a training sample, go to step S7; when the sample is a test sample, go to step S8;

[0044] Step S7, after obtaining the sample hyperdimensional vector R of the training sample, record it as k according to the category to which the label belongs, and add it to the category hyperdimensional vector C k corresponding to the category through addition operation as follows:

[0045] C k += R

[0046] where k = 1, 2, ..., K, and K is the total number of categories;

[0047] Step S8, after obtaining the sample hyperdimensional vector R of the test sample, perform a bit-by-bit comparison with the binarized category hyperdimensional vector [[C k ; the category represented by [[C k with the largest number of identical elements to R is the category to which the test sample belongs.

[0048] Based on the above theory and process, combined with the attached Figure 2A hardware implementation of the present invention will be described. First, after determining the dimension D of the hyperdimensional vector, the number N of data points, and the number M of numerical sampling points according to steps S1 and S3, an N×D-bit space is divided in the hardware memory as the address memory, and an M×D-bit space is divided as the continuous item memory; according to step S2, an N×D random binary numbers are generated using a random number generator and stored in the address memory, with each row serving as an address hyperdimensional vector; then, according to step S4, D random binary numbers are generated again using a random number generator and stored in the first row of the continuous item memory. According to step S5, D / 2 / (M - 1) of them are randomly selected and inverted through a NOT gate and filled into the next row, and each element is only flipped once. This step is repeated (M - 1) times until all M rows of the continuous item memory are filled, with each row serving as a numerical hyperdimensional vector; referring to step S6 and Appendix Figure 1 , for the nth data point in the sample, n = 1, 2,..., N, the address hyperdimensional vector H n and the numerical hyperdimensional vector V n are obtained from the address memory and the continuous item memory. The two hyperdimensional vectors are bitwise passed through an exclusive-OR gate circuit, the exclusive-OR operation result is retained and the same operation is performed on the next data point until all N data points have completed the above operation. At this time, there are N exclusive-OR operation results for each element; to complete the binarization process in step S6, a counter is used to obtain the number of 1s among the N exclusive-OR operation results at each position, and then a numerical comparator is used to compare its size relationship with N / 2. If the number is greater than or equal to N / 2, the final value of this bit is 1, otherwise the value is 0, and the sample hyperdimensional vector R is obtained; if this sample is a training sample, then according to step S7, according to the category k to which its label belongs, R is added bitwise to the category hyperdimensional vector C k , where k = 1, 2,..., K, and K is the total number of categories; if this sample is a test sample, then according to step S8, each element of each category hyperdimensional vector C k is compared with half of the number of training samples using a numerical comparator. If it is greater than or equal to half of the number of training samples, the final value of this bit is 1, otherwise the value is 0. Through this method, the binarization process is achieved; after obtaining the binarized category hyperdimensional vector [[C k , then an exclusive-OR gate is used to bitwise determine whether each element in the sample hyperdimensional vector R and [[C k corresponds identically. A counter is used to record the number of bits in each [[C k that are the same as this sample hyperdimensional vector. Again, a numerical comparator is used to find the [[C k with the most identical bits to R, and the corresponding category is the category to which this test sample belongs.

[0049] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for implementing a lightweight hyper-dimensional computing classification model, characterized in that, Including the following steps: Step S1: Determine the data volume and data value range of the samples to be learned; Denote the number of data points as N and the value range as [min, max]; Determine the dimension D of the hyperdimensional vector; Step S2: Divide an address memory with an N×D-bit space in the hardware memory, and randomly assign a binary number 0 or 1 to each bit address; Denote a hyperdimensional vector with a size of 1×D for each row as H n , where n = 1, 2,..., N, which is the address hyperdimensional vector and represents the position of the nth data point among the N data points; Step S3: Determine the number of numerical sampling points M, and equally divide the value range [min, max] into M levels; the value Y represented by the m-th level is as follows: m is as follows: where m = 1, 2,..., M; Step S4: Divide a continuous item memory of M×D bits in the hardware memory, and assign a binary number 0 or 1 to each bit of the 1×D hyperdimensional vector in the first row, denoted as L1; L1 corresponds to the lower bound min of the value range; Step S5: Randomly select D / 2 / (M - 1) bits from L1, invert them through a NOT gate to represent the next value, and each element is flipped only once; repeat the operation until all M-level values are represented; at this time, the operation is repeated (M - 1) times to obtain the numerical hyperdimensional vector L m , m = 1, 2,..., M; L m represents the m-th level value Y in the M-level value range m ; Any two numerical hyperdimensional vectors L i and L j , where i, j ∈ {1, 2,..., M}, the number of different bits between them depends on the difference between Y i and Y j ; in particular, the number of different bits between L1 and L M is D / 2, that is, L1 and L M are orthogonal; Step S6: After inputting a sample, encode based on the hyperdimensional vectors in the address memory and the continuous item memory obtained in Steps S1 - S5 as follows: V n ∈ {L1, L2,..., L M}, n = 1, 2,..., N Where H n is the address hyper-dimensional vector in the address memory; V n is the value hyper-dimensional vector corresponding to the value of the nth data point in the continuous item memory; represents binarization processing; R is the sample hyper-dimensional vector finally obtained in the encoding process; After the above encoding, obtain the number of 1s in each bit through a counter and compare it with N / 2. When the number of 1s in this bit is greater than N / 2, the operation result of this bit is 1, otherwise it is 0; After obtaining the encoding result R, when this sample is a training sample, go to Step S7; when this sample is a test sample, go to Step S8; After obtaining the sample hyper-dimensional vector R of the training sample in step S7, it is denoted as k according to the category to which the label belongs, and is added to the category hyper-dimensional vector C corresponding to the category through an addition operation k Specifically, it is as follows: C k + = R where k = 1, 2,..., K, and K is the total number of categories; After obtaining the sample hyper-dimensional vector R of the test sample in step S8, perform a bit-by-bit comparison with the binarized class hyper-dimensional vector ; The class represented by that has the largest number of identical elements with R is the class to which the test sample belongs.

Images