Analog Feature Transformer Based on Improved Discrete Cosine Transform

Abstract

The present invention provides an analog feature converter based on an improved discrete cosine transform, comprising a comparator, a counter and a classifier; an input end of the counter is connected to an output end of the comparator, and an output end of the counter is connected to an input end of the classifier; it includes a discrete training stage and an online calculation stage; applying this technical solution can greatly reduce the sampling rate of information, thereby reducing the hardware cost and power consumption.

Description

Technical Field

[0001] The present invention relates to the field of technical signal conversion, and in particular to an analog feature converter based on an improved discrete cosine transform. Background Art

[0002] With the wave of artificial intelligence and big data, the Internet of Things technology has made rapid development, and Internet of Things (IOT) intelligent terminals have been widely used in various industries. The IOT sensing data generally has the characteristics of Random-Sparse-Event (RSE) or burst. Using the classical Nyquist rate-based Analog-Digital Converter (ADC) to collect data will waste the limited hardware resources of IOT intelligent terminals, which is extremely unfavorable for low-power design. In recent years, both the industrial community and the academic community have proposed that an Analog-to-Information Converter (AIC) can be used to achieve information sampling. The AIC is a new generation of analog-digital converter architecture that achieves low-power design of the system by reducing the sampling rate. It uses the sparsity or compressibility of the signal to sample the signal in the way of "information rate". Such signal processing technologies are widely used in IOT applications and wearable devices, such as speech recognition applications, gesture detection systems, heart rate monitors, etc. Analog-to-Feature Converters (AFCs) are mostly designed specifically for specific tasks, and suppress other irrelevant signals by highlighting feature-related signals through feature enhancement filters. After collecting the analog signal features through the feature enhancement filter, the AFC directly makes a decision at the edge end, effectively solving the real-time calculation problem, which is of great significance to the development of IOT technology.

[0003] The AIC aims at reconstructing the signal, but there is currently real-time detection and decision-making for IOT. Although the AIC can sample the signal in the way of effective information rate and improve the analog-to-digital conversion efficiency, the current research based on the AIC is more aimed at signal reconstruction. In order to better reconstruct the original signal, the AIC inevitably collects some redundant information. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an analog feature converter based on an improved discrete cosine transform, which greatly reduces the information sampling rate, thereby reducing the hardware cost and power consumption.

[0005] To achieve the above object, the present invention adopts the following technical solutions: an analog feature converter based on an improved discrete cosine transform, comprising a comparator, a counter and a classifier; an input end of the counter is connected to an output end of the comparator, and an output end of the counter is connected to an input end of the classifier; it includes a discrete training stage and an online calculation stage;

[0006] The discrete training stage includes sparse coding, SGD algorithm and dictionary update strategy;

[0007] The sparse coding is directed at a predefined MDCT dictionary, according to

[0008]

[0009] In the formula, the first term is the reconstruction error term; the latter two terms are regularization constraint terms, λ1 and λ2 are regularization parameters which are both constants, x is the signal sample, α is the sparse coefficient corresponding to the signal x, and D is the optimal MDCT joint basis dictionary finally learned and obtained;

[0010] The Feature-si1.gn algorithm is used to solve the sparse coefficient α * , and the time complexity of this algorithm depends on the dimension n of the sample data and the amount of sample data b for each iteration, that is, the time complexity of the algorithm is O(nb);

[0011] The objective function for classifier learning is written as:

[0012]

[0013] In the formula, α * (x i , D) is the feature encoding coefficient of x i ; is the loss function of the classifier,; W is the general term of the normal vector and bias of the hyperplane required for training inside the classification vector machine, that is, the classifier model, and l represents the label set corresponding to the input signal; the specific loss function is expressed as follows:

[0014]

[0015] In the formula, W = (w1, w2,... w k );

[0016] The iterative method of the SGD algorithm is to iterate along the negative direction of the gradient from the position of the objective function corresponding to the current sample, and the gradient calculation formula is:

[0017]

[0018] In the formula,[[]] denotes the differential symbol, T is the transpose of a matrix, M represents a finite signal set containing M N-dimensional samples, and R K is the k-dimensional high-dimensional space mapped from the original sample space, and β * is a vector in R K ; R K is the k-dimensional high-dimensional space mapped from the original sample space; it depends on D, W, x, α; let Λ denote the non-zero coefficient index set of the sparse coefficient α * ; taking the classifier parameters as an example, the forms of the previous and next iterations are shown as follows:

[0019]

[0020] where W t is the classifier model iterated at the t-th time, ρ represents the learning rate, v is the regularization parameter, is the gradient calculation formula, and L represents the label set;

[0021] The selection strategy of dictionary atoms is to measure the correlation between atoms and features by mutual information, and to guide the learning of dictionary atoms through the labels in the classifier; each sparse coefficient corresponds to an atom in the dictionary. As long as the MI value of the coefficient is estimated, the correlation between each dictionary atom and the task features can be obtained;

[0022] Suppose the t-th signal x in the signal set X t corresponds to the feature vector α t =(α t1 , α t2 ,..., α tK ). First, the i-th feature vectors (α 1i , α 2i ,..., α Mi ) of all signals are quantized into H levels, and α' ti ∈{1, 2,..., H} is used to represent the quantized α ti , then the joint probability of the quantized feature and the classification task can be expressed as:

[0023]

[0024] where l is the class label, taking l = 1 or l = 0; l t represents the class label corresponding to the t-th signal x t ; h is the quantization level, and h ∈{1, 2,..., H}; H represents the number of levels, and |·| is the set cardinality; therefore, the MI between each task category and the atom is:

[0025]

[0026] In the formula, p i(h) and p(l) are the marginal distributions of the i-th atom and class label respectively; the MI value of each atom is calculated after each iteration, and after all the iteration times, the dictionary atom with the highest relevance is selected;

[0027] The algorithm verification process is as follows: the processed data set is divided into a training set and a test set;

[0028] Secondly, the training set is used as the input of the MPSGD algorithm, and the sparse matrix of the initial MDCT dictionary is calculated through the Feature-sign software package, and then the final dictionary and classifier model are continuously iteratively learned; finally, in the test stage, the learned dictionary above is used to extract features from the test set signal, and the extracted sparse coefficients are used as the input of the classification model for classification;

[0029] The online calculation part includes a PWM modulation module and a counter. The input signal x(t) is compared with the designed modulation waveform, and its amplitude information is mapped into the pulse width of the PWM waveform, and then the counter is used to count and observe it in combination with the PN sequence; it includes the following steps:

[0030] (1) Generate test signals and modulation signals: First, the FPGA chassis module needs to be set to generate analog ECG signals and modulation signals;

[0031] (2) Generate a PWM waveform: The generated modulation signal and test signal are simultaneously input into the PWM modulation circuit to generate a PWM waveform, which is input into the next chassis;

[0032] (3) Measure the PWM width: The chassis counts the width of the PWM waveform and sets the counter parameters;

[0033] (4) Data acquisition end: The counted value is collected and saved as a.txt file and sent to the PC receiving end;

[0034] (5) Classification test: After the PC receives the data, it is transmitted to the working area for classification.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. Compared with the SVM method based on AIC, this method does not need to restore the signal, but directly classifies the eigenvalue, reduces the intermediate steps, removes a large amount of redundancy, and greatly saves system resources.

[0037] 2. Through experimental verification, the task-driven feature extraction model used in the present invention has a dictionary dimension of dozens of times higher than the sample dimension in general predefined dictionaries, while the dimension of the learning dictionary in this model is greatly reduced, which also shows that there is a high degree of redundancy in the traditional predefined dictionary dimension.

[0038] 3. The model based on the reconstruction task can obtain more compact dictionary atoms and lower-dimensional features. In the decomposition algorithm, the number of dictionary atoms directly determines the complexity of the algorithm. Therefore, this method can also save the computing resources of the algorithm.

[0039] 4. After the hardware-in-the-loop verification, it is proved that the present invention still has a high resolution under the condition of high compression ratio, and the detection accuracy is better than the feature extraction methods in the relevant literature in recent years.

[0040] 5. The hardware circuit structure of this method is simple and has low requirements for hardware accuracy. Therefore, the hardware circuit made by the present invention has the advantages of small occupied area and low power consumption, and is very suitable for portable wearable devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is the system block diagram of the preferred embodiment of the present invention;

[0042] Figure 2 It is the five-fold cross-validation sequence diagram of the preferred embodiment of the present invention;

[0043] Figure 3 It is the verification flowchart of the preferred embodiment of the present invention;

[0044] Figure 4 It is the signal reconstruction comparison diagram of the preferred embodiment of the present invention, where (a) is the original waveform and (b) is the reconstructed waveform;

[0045] Figure 5 It is the hardware-in-the-loop test system of the preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] The present invention will be further described below with reference to the drawings and embodiments.

[0047] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0048] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0049] The present invention is a novel analog feature converter design based on an improved discrete cosine transform. To address the problem of redundant feature coefficients still existing in signal decomposition using traditional Fourier bases in classification tasks, an improved AFC sampling structure using the Modified Discrete Cosine Transform (MDCT) basis is proposed on the basis of the AIC structure. As Figure 1 shown, the present invention includes two stages, one is offline training and the other is online calculation. The online measurement extracts the encoded feature sequence of the analog signal according to the fitted demodulation sequence obtained from offline training. Below, I will take the classification of electrocardiogram signals as an example to elaborate on the present invention in detail according to these two parts.

[0050] 1. Offline training

[0051] In terms of algorithms, the main technical content in offline training is to verify the offline training part of the present invention using electrocardiogram signals as an example.

[0052] Sparse coding: For the predefined MDCT dictionary, according to

[0053]

[0054] In the formula, the first term is the reconstruction error term; the last two terms are the regularization constraint terms, λ1 and λ2 are regularization parameters and are both constants, x is the signal sample, α is the sparse coefficient corresponding to the signal x, and D is the optimal MDCT joint basis dictionary finally learned and obtained.

[0055] The calculation process of solving the sparse coefficient matrix of the sample is relatively complex. At present, many researchers have proposed relevant algorithms to solve this problem, and there are corresponding software packages available for use.

[0056] The Feature-si1.gn algorithm is used to solve the sparse coefficient α * , and the time complexity of this algorithm depends on the dimension n of the sample data and the amount of sample data b in each iteration, that is, the time complexity of the algorithm is O(nb);

[0057] The objective function of classifier learning can be written as:

[0058]

[0059] In the formula, α * (x i ,D) is the feature encoding coefficient of x i ; is the loss function of the classifier,; W is the general term of the normal vector and bias of the hyperplane required for training inside the classification vector machine (i.e., the classifier model), l represents the label set corresponding to the input signal. For example, for a binary classification problem, take l i = 1 or li = 0, where v is a regularization parameter. The specific loss function can be expressed as follows:

[0060]

[0061] In the formula, W = (w1, w2,... w k ).

[0062] SGD: The iterative method of the algorithm is to iterate along the negative gradient direction from the position of the objective function corresponding to the current sample. The gradient calculation formula is:

[0063]

[0064] In the formula, represents the differential symbol, T is the transpose of the matrix, M represents a finite signal set containing M N-dimensional samples, and R K is the k-dimensional high-dimensional space mapped from the original sample space, and β * is a vector in R K that depends on D, W, x, α, and R K is the k-dimensional high-dimensional space mapped from the original sample space. Denote Λ as the index set of non-zero coefficients of the sparse coefficient α * . Taking the classifier parameters as an example, the forms of two consecutive iterations can be expressed as follows:

[0065]

[0066] where W t is the classifier model iterated at the t-th time, ρ represents the learning rate, v is a regularization parameter, is the gradient calculation formula, and L represents the label set.

[0067] Dictionary update strategy: The selection strategy of dictionary atoms is to measure the correlation between atoms and features using mutual information (MI), and use the labels in the classifier to guide the learning of dictionary atoms. This can ensure that the finally learned dictionary atoms are learned from the initially predefined dictionary, avoiding the uncertainty of updating the dictionary by the stochastic gradient descent method. Its main idea is: Each sparse coefficient corresponds to an atom in the dictionary. As long as the MI value of the coefficient is estimated, the correlation between each dictionary atom and the task features can be obtained.

[0068] Assume that the feature vector corresponding to the t-th signal x t in the signal set X is α t = (α t1 , α t2 ,..., α tK), directly calculating the mutual information (MI) between the label and the feature has an excessive computational cost because there may be too many feature values. To obtain a reliable estimate of the MI for each feature \(i\), first, the \(i\)-th feature values (\(\alpha\) 1i , \(\alpha\) 2i ,..., \(\alpha\) Mi ) of all signals are quantized into \(H\) levels, and \(\alpha'\) ti \(\in \{1, 2,..., H\}\) represents the quantized \(\alpha\) ti . Then, the joint probability of the quantized feature and the classification task can be expressed as:

[0069]

[0070] where \(l\) is the class label, taking \(l = 1\) or \(l = 0\); \(l\) t represents the class label corresponding to the \(t\)-th signal \(x\) t . \(h\) is the quantization level, and \(h \in \{1, 2,..., H\}\), where \(H\) represents the number of levels; \(|\cdot|\) is the set cardinality. Therefore, the MI between each task category and the atom is:

[0071]

[0072] In the formula, \(p\) i (h) and \(p(l)\) are the marginal distributions of the \(i\)-th atom and the class label, respectively. After each iteration, the MI value of each atom is calculated. After all the iteration times, the dictionary atom with the highest correlation is selected.

[0073] Finally, the model training algorithm based on the above principle is as follows:

[0074]

[0075]

[0076] The algorithm verification process of the present invention is as follows: First, referring to Figure 2 , the processed data set is divided into a training set and a test set according to Figure 4 ;

[0077] Secondly, the training set is used as the input of the MPSGD algorithm, and the sparse matrix of the initial MDCT dictionary is calculated through the Feature - sign software package, and then the final dictionary and classifier model are continuously iteratively learned; finally, in the test stage, the learned dictionary is used to extract features from the test set signals, and the extracted sparse coefficients are used as the input of the classification model for classification. The above verification process is as Figure 3 shown.

[0078] The experiment distributes data according to the five-fold cross-validation method, selects 8000 samples as the training set, and the remaining 2000 samples as the test set. First, the optimal experimental parameters are selected through parameter comparison, mainly including the regularization parameter λ1, the number of iterations Iter, and the dictionary dimension K. The specific comparison method follows the idea of controlling variables: when comparing the regularization parameter λ1, the number of iterations Iter and the dictionary dimension K parameters remain unchanged; when comparing the dictionary dimension K, the regularization parameter λ1 and the number of iterations Iter remain unchanged. According to machine learning experience, the parameters to be selected are set as λ1 ∈ {0.001, 0.01, 0.1, 1}, the number of iterations Iter is incremented from 0 upwards, and the dictionary dimension K ∈ {1200, 1400, 1600, 1800}. The batch dimension of data processing is set to 100, that is, 100 samples are calculated in one iteration, and the sample extraction method is random sampling without replacement. Therefore, 80 iterations complete one full traversal of the dataset. In the training stage, iterative learning is performed through Algorithm 1 to learn the dictionary D and the classification model W corresponding to each parameter. In the test stage, the trained W is used for classification testing. Analyzing from the dictionary dimension K, the specific classification results are shown in Table 1. When K exceeds the sample dimension, the impact on Acc is not significant, that is, the model is not sensitive to the dictionary dimension. Generally, the predefined dictionary dimension is dozens of times higher than the sample dimension, while the learned dictionary dimension in this model is greatly reduced, which also indicates that there is a high redundancy in the traditional predefined dictionary dimension.

[0079] Table 1 Classification results for different dictionary dimensions

[0080]

[0081] The reconstruction task optimization model is used for learning experiments, and the reconstruction effect is as Figure 4 shown.

[0082] Compared with the ordinary sparse coding model, the reconstruction signal-to-noise ratio is only increased by 1.2 dB, but the dictionary dimension and the feature dimension are greatly reduced, indicating that the model based on the reconstruction task can obtain more compact dictionary atoms and lower-dimensional features.

[0083] According to the classifier and dictionary model obtained above, the remaining 2000 samples are tested. The specific results are shown in Table 2, and its accuracy can still reach 97.95%, indicating that the joint optimization model in this chapter can still maintain a good classification accuracy while reducing the feature dimension.

[0084] Table 2 SVM classification results with a dictionary dimension of 1400

[0085]

[0086] 2. Online calculation

[0087] As Figure 1 shown, the online calculation part mainly consists of a PWM modulation module and a counter. The input signal x(t) is compared with the designed modulation waveform, and its amplitude information is mapped into the pulse width of the PWM waveform. Then, the counter is used to count and observe it in combination with the PN sequence. Due to the digital characteristics of the counter, it also completes the work of analog-to-digital conversion.

[0088] This design consists of a mixer, a low-pass filter, and a sub-Nyquist rate low-speed ADC. Among them, the mixer and the low-pass filter complete the task of analog signal feature extraction. The input signal is mixed with the modulation signal, then integrated and sampled. Finally, the digital features of the analog signal are obtained, and a vector machine is used for classification.

[0089] To verify the online test part of the present invention, we built a hardware-in-the-loop test system. The structure of this part is as Figure 5 shown, including a data sending end display 1, an NI-9263 module 2 for generating a sawtooth signal, a PWM modulation module 3, an NI CompactRIO device 4 for counting, an NI-9025 module 5 for receiving the counted value, and a data receiving end display 6.

[0090] The main operations of this experiment are divided into five steps:

[0091] (1) Generate test signals and modulation signals: First, the FPGA chassis module needs to be set to generate analog electrocardiogram signals and modulation signals.

[0092] (2) Generate PWM waveforms: The generated modulation signals and test signals are simultaneously input into the PWM modulation circuit to generate PWM waveforms, which are input into the next chassis.

[0093] (3) Measure the PWM width: The chassis counts the width of the PWM waveform and sets the counter parameters.

[0094] (4) Data acquisition end: Collect the counted values, save them as.txt files, and send them to the PC receiving end.

[0095] (5) Classification test: After the PC receives the data, it is transmitted to the working area for classification.

[0096] Finally, 2000 electrocardiogram samples were selected for classification testing, including 1000 samples with normal heart rate and 1000 samples with abnormal heart rate. The hardware circuit was used to classify the signals. The specific classification results are shown in Table 3. It can be found that among the 1000 normal heart rate samples, the number of misclassified cases is 45, and among the 1000 abnormal samples, the number of misclassified cases is 49. The accuracy rate reaches 95.3%. Although the classification accuracy rate is slightly lower than the simulation result, considering the uncertain factors such as noise in the hardware implementation, the overall classification accuracy rate is still relatively optimistic.

[0097] Table 3 SVM classification results of the measured data of the NI-Compact hardware platform

[0098]

[0099] The results show that the present invention can still obtain a higher classification accuracy rate under the condition of equal or higher compression ratio, which proves that the feature conversion rate of this system is better than the existing CS-based feature extraction methods and is expected to be widely applied in low-power, portable wearable systems.

Claims

1. An analog feature converter based on an improved discrete cosine transform, characterized in that It includes a comparator, a counter, and a classifier; the input end of the counter is connected to the output end of the comparator, and the output end of the counter is connected to the input end of the classifier; the analog feature converter includes a discrete training stage and an online calculation stage; The discrete training stage includes sparse coding, the SGD algorithm, and a dictionary update strategy; The sparse coding is for a predefined MDCT dictionary, according to In the formula, the first term is the reconstruction error term; the last two terms are regularization constraint terms, λ1 and λ2 are regularization parameters which are both constants, x is the signal sample, α is the sparse coefficient corresponding to the signal x, and D is the optimal MDCT joint basis dictionary finally learned and obtained; The Feature-si1.gn algorithm is used to solve the sparse coefficient α * , and the time complexity of this algorithm depends on the dimension n of the sample data and the amount of sample data b in each iteration, that is, the time complexity of the algorithm is O(nb); The objective function for classifier learning is written as: where α * (x i , D) is the feature coding coefficient of x i ; is the loss function of the classifier; W is the general term of the normal vector and bias of the hyperplane required for training inside the classification vector machine, that is, the classifier model, and l represents the label set corresponding to the input signal; the specific loss function is expressed as follows: where W = (w1, w2,... w k ); The iterative method of the SGD algorithm is to iterate along the negative gradient direction from the position of the objective function corresponding to the current sample, and the gradient calculation formula is: wherein, represents the differential symbol, T is the transpose of the matrix, M represents a finite signal set containing M N-dimensional samples, R K is the k-dimensional high-dimensional space mapped by the original sample space, β * is a vector in R K ; it depends on D, W, x, α; let Λ be the non-zero coefficient index set of the sparse coefficient α * ; taking the classifier parameters as an example, the forms of the two consecutive iterations are expressed as follows: Among which W t is the classifier model iterated at the t-th time, ρ represents the learning rate, v is the regularization parameter, is the gradient calculation formula, and L represents the label set; The selection strategy of dictionary atoms is to use mutual information to measure the correlation between atoms and features, and the learning of dictionary atoms is guided by the labels in the classifier; Each sparse coefficient corresponds to an atom in the dictionary. As long as the MI value of the coefficient is estimated, the correlation between each dictionary atom and the task feature can be obtained; Suppose the \(t\)-th signal \(x\) in the signal set \(X\) t The corresponding feature vector is \(\alpha\) t =( \(\alpha\) t1 , \(\alpha\) t2 ,..., \(\alpha\) tK ). First, the \(i\)-th feature vector of all signals (\(\alpha\) 1i , \(\alpha\) 2i ,..., \(\alpha\) Mi ) is quantized into \(H\) levels, and \(\alpha'\) ti \(\in \{1, 2,..., H\}\) represents the quantized \(\alpha\) ti . Then the joint probability of the quantized feature and the classification task can be expressed as: where \(l\) is the class label, taking \(l = 1\) or \(l = 0\); \(l\) t represents the class label corresponding to the \(t\)-th signal \(x\) t ; \(h\) is the quantization level, and \(h\in\{1,2,\cdots,H\}\); \(H\) represents the number of levels, and \(|\cdot|\) is the set cardinality; thus, the MI between each task category and the atom is: where p i (h) and p(l) are the marginal distributions of the i-th atom and the class label, respectively; after each iteration, the MI value of each atom is calculated, and after all the iteration times, the dictionary atom with the highest relevance is selected; The algorithm verification process is as follows: the processed data set is divided into a training set and a test set; Secondly, the training set is used as the input of the MPSGD algorithm, and the sparse matrix of the initial MDCT dictionary is calculated through the Feature-sign software package, and then the final dictionary and classifier model are continuously iteratively learned; finally, in the test stage, the learned dictionary above is used to extract features from the test set signal, and the extracted sparse coefficients are used as the input of the classification model for classification; The online calculation part includes a PWM modulation module and a counter. The input signal x(t) is compared with the designed modulation waveform, and its amplitude information is mapped into the pulse width of the PWM waveform, and then the counter is used to count and observe it in combination with the PN sequence; it includes the following steps: (1) Generate a test signal and a modulation signal: First, the FPGA chassis module needs to be set to generate an analog electrocardiogram signal and a modulation signal; (2) Generate a PWM waveform: The generated modulation signal and test signal are simultaneously input into the PWM modulation circuit to generate a PWM waveform and input it into the next chassis; (3) Measure the PWM width: The chassis counts the width of the PWM waveform and sets the counter parameters; (4) Data acquisition end: The counted value is collected and saved as a.txt file and sent to the PC receiving end; (5) Classification test: After the PC end receives the data, it is transmitted to the working area for classification.

Images