Speech recognition method based on STO array and dynamic time planning algorithm

Abstract

The invention discloses a speech recognition method based on an STO array and a dynamic time planning algorithm. The method includes processing speech signals using an STO array and identifying them using an improved dynamic time planning algorithm. The STO can generate a stable microwave signal in the absence of an external magnetic field. The method first pre-processes the signal in the time domain, then performs a fast Fourier transform, and then converts the power spectrum into a current signal by calculation and passes it into the STO array, obtaining the output frequency corresponding to each current segment as a feature vector. Finally, the feature vector is processed using an improved dynamic time planning algorithm to achieve recognition. The method utilizes an array constructed of nano-scale STO devices, combines the characteristics of STO such as its ability to flip without a magnetic field and high frequency adjustability, and improves the algorithm to limit the distance and increase the training library, thereby realizing a speech recognition hardware improvement system, which is expected to achieve highly integrated and high-recognition speech recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech signal recognition, relates to a speech signal processing method based on a spin device, and specifically to a speech recognition method based on an STO array and a dynamic time planning algorithm. Background Art

[0002] Research on speech recognition can be traced back to the middle of the last century, when Bell Labs developed the world's first computer-based speech recognition system, which could recognize simple English pronunciation input. Since then, research on speech recognition has entered a period of rapid development around the world, and many speech recognition technologies have been invented: In the 1980s, the method of using hidden Markov models (HMM) for speech recognition was proposed, and artificial neural network technology (ANN) also began to be applied to the field of speech recognition, followed by the emergence of dynamic time warping (DTW) technology.

[0003] A spin nano-oscillator (STO) is a nanoscale microwave oscillator based on the STT effect. It uses a polarization current to induce a stable precession of the magnetic moment in the free layer, thereby generating an alternating microwave signal. Regarding the application of STO in speech recognition, researchers have proposed combining STO with neural network reservoir computing to apply STO to speech recognition. However, this approach is complex and difficult to implement, remaining at the theoretical stage. Whether it can be realized remains to be determined, making practical application difficult. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention proposes a speech recognition method based on the STO array and dynamic time planning algorithm, which improves the existing dynamic time planning algorithm. By detecting the signal processed by the STO array, the recognition of isolated Chinese and English word speech can be realized.

[0005] The speech recognition method based on the STO array and the dynamic time planning algorithm specifically includes the following steps:

[0006] Step 1: Collect the speech signal to be recognized, filter out noise through preprocessing, perform endpoint detection, output the time domain speech signal, then perform fast Fourier transform on the time domain speech signal, and then calculate the power spectrum of the frequency domain signal after Fourier transform.

[0007] Preferably, the preprocessing method is to perform pre-emphasis, framing and windowing on the signal in sequence.

[0008] Step 2: Divide the power spectrum obtained in step 1 into multiple frequency bands from low to high. Then, amplify the average frequency amplitude in each frequency band by the same multiple as the polarization current value and input it into an STO array composed of multiple identical STOs (spin nano-oscillators). The corresponding oscillation frequency is obtained at the output end. The number of STOs in the STO array is greater than the number of frequency bands into which the power spectrum is divided.

[0009] Preferably, the polarization current value input into the STO array is within the STO driving current range.

[0010] Preferably, the STO is capable of stable oscillation without magnetic moment reversal.

[0011] Step 3: Arrange the oscillation frequencies output by the STO array in step 2 in order to obtain a frequency vector, calculate the differential parameters of the frequency vector, and obtain the speech feature vector {T(1), T(2), T(3), …, T(n), .., T(N)} of the speech signal to be recognized as a detection template.

[0012] Step 4: Collect speech signals with known labels. After processing them using the methods of steps 1 to 3, the speech feature vectors {R(1), R(2), R(3), …, R(k), .., R(K)} of each known speech signal are used as matching templates and saved in the matching database.

[0013] Step 5: Input the detection template obtained in step 3 and the matching module stored in the matching database into the improved dynamic time planning algorithm, and perform matching by calculating the Euclidean distance between the two speech feature vectors to complete speech recognition. The specific steps are as follows:

[0014] s5.1. Define two cumulative distance matrices D and matching distance matrices d of size N*K, where N and K are the number of feature vectors in the detection template and the matching template, respectively.

[0015] s5.2. Under dynamic programming, sequentially calculate the Euclidean distance between the nth feature vector in the detection template and its corresponding kth feature vector in the matching template, and record this distance as the matching distance d(n, k) in the matching distance matrix d. The dynamic programming is as follows: calculate the cumulative distances D1, D2, and D3 of the three possible predecessor feature vectors (n, k). Then, select the minimum of the three cumulative distances D1, D2, and D3, add it to the matching distance d(n, k), and record this as the cumulative distance for (n, k) in the cumulative distance matrix D.

[0016] s5.3. If the current cumulative distance D(n, k) exceeds the set range, the calculation is terminated and the result D(N, K) = ∞ is output; otherwise, the distance calculation between all feature vectors is completed and the actual calculation result of D(N, K) is output.

[0017] s5.4. Repeat s5.1 to s5.3 to calculate the distance between the detection template and each matching template, and then take the label corresponding to the matching template with the smallest distance as the detection result.

[0018] The present invention has the following beneficial effects:

[0019] 1. The STO array used can generate stable oscillations without an external magnetic field, generating frequency-adjustable microwave signals solely through current drive. Compared to materials with an external magnetic field, it offers advantages such as smaller size, lower power consumption, lower costs, and greater flexibility. Furthermore, it can reduce the mutual influence caused by the dynamic coupling of the STO itself and prevent the influence of external magnetic fields on voice signals.

[0020] 2. Combining the STO array with an improved time planning algorithm, the improved time planning algorithm will impose distance restrictions during the recognition process. When the difference between the test template and the reference template is too large, the matching will be automatically interrupted, which reduces the complexity of the system. It can also recognize speech signals of different lengths, improve the reliability and recognition rate of speech recognition, and use the STO array to achieve not only nano-level speech recognition, but also hardware acceleration. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural diagram of STO;

[0022] Figure 2 This is the STO oscillation waveform obtained from micromagnetic simulation;

[0023] Figure 3 The relationship between the polarization current of STO and its oscillation frequency;

[0024] Figure 4 Schematic diagram of the STO array;

[0025] Figure 5 A flow chart for generating a speech feature vector;

[0026] Figure 6 Schematic diagram of dynamic time planning algorithm. DETAILED DESCRIPTION

[0027] The present invention will be further explained below with reference to the accompanying drawings;

[0028] Figure 1Schematic diagram of the spin nanooscillator structure designed and used in this example. The STO has a diameter of 600nm and is generally divided into three layers: from top to bottom: a free layer, a non-magnetic layer, and a pinned layer. The free layer is made of CoFeB, has a height of 12nm, and a magnetic moment density of 5nm; the non-magnetic layer is made of Mgo and is smaller than the free layer; and the pinned layer is made of CoFeB, with the same height and magnetic moment density as the free layer. This size ensures stable oscillation of the STO and prevents magnetic moment reversal.

[0029] The STO structure described above was programmed in the micromagnetic simulation software OOMMF, which records the simulation results by stably oscillating the structure independently of an external magnetic field. During the simulation, OOMMF assumes that the magnetic moment obeys the Landau-Lifshitz-Gilbert dynamic equation during the state change:

[0030]

[0031] where M is the magnetic moment of a single atom, is the gyromagnetic ratio, H eff is the effective field size, α is the damping coefficient, M s is the saturation magnetization.

[0032] Figure 3 The waveform diagram of the STO oscillation time and the X-direction magnetization component obtained by micromagnetic simulation software. Figure 4 The output operating frequency of the STO changes as a result of inputting polarization currents of different magnitudes into the input end, as simulated in micromagnetic simulation software. It can be seen that when the input polarization current increases, the corresponding output operating frequency also increases, and the relationship between the two is approximately linear. Therefore, the output operating frequency of the STO can reflect the magnitude of the input polarization current.

[0033] The speech recognition method based on the STO array and the dynamic time planning algorithm specifically includes the following steps:

[0034] Step 1: Collect the speech signal to be recognized, perform pre-emphasis, framing and windowing on it in sequence, filter out noise, perform endpoint detection, output the time domain speech signal, then perform fast Fourier transform on the time domain speech signal, and then calculate the power spectrum of the frequency domain signal after Fourier transform.

[0035] Step 2: Divide the power spectrum obtained in step 1 into multiple frequency bands from low to high, and then amplify the average frequency amplitude in each frequency band by the same multiple as the polarization current value, and input it into the STO array composed of multiple identical STOs to obtain the corresponding oscillation frequency at the output end. Figure 2As shown in the figure, the number of STOs in the STO array matches the number of frequency bands in the spectrum diagram. The current signal at the input terminal enters the STO, which generates a microwave oscillation signal and outputs it at the output terminal. To ensure stable STO oscillation, the polarization current value input to the STO array should be within the STO drive current range.

[0036] Step 3: Arrange the oscillation frequencies output by the STO array in step 2 in order to obtain a frequency vector, and calculate the differential parameter d(n) of each frequency in turn:

[0037]

[0038] Where C is the frequency parameter.

[0039] The differential parameters are combined into the speech feature vector {T(1), T(2), T(3), …, T(n), .., T(N)} of the speech signal to be recognized as a detection template, such as Figure 5 shown.

[0040] Step 4: Collect speech signals with known labels. After processing them using the methods of steps 1 to 3, the speech feature vectors {R(1), R(2), R(3), …, R(k), .., R(K)} of each known speech signal are used as matching templates and saved in the matching database.

[0041] Step 5: In order to compare the similarity between the matching template and the detection template, the distance D[T, R] between them is calculated. The smaller the distance, the higher the similarity between the two. The one with the highest similarity is the detection result corresponding to the detection template. The detection template obtained in step 3 and the matching module stored in the matching database are input into the improved dynamic time planning algorithm. The Euclidean distance between the two speech feature vectors is calculated to match them and complete the speech recognition. The working principle diagram of the dynamic time planning algorithm is shown in the figure below. Figure 6 As shown in the figure, the dynamic time planning algorithm is based on the idea of ​​dynamic programming and can solve the template matching problem of different pronunciation lengths. In addition, the training phase of the dynamic time planning algorithm is simple and requires almost no additional calculations. The specific steps are as follows:

[0042] s5.1. Divide the matching templates collected in step 4 into groups of 10. Compare the detection template with the 10 matching templates in the group simultaneously.

[0043] s5.2. Define two N*K cumulative distance matrices, D, and matching distance matrix d, where N and K are the number of feature vectors in the detection and matching templates, respectively. When N=K, the reference and test templates can be directly aligned for recognition. However, in most cases, different speech signals may have different total feature vectors due to differences in volume and pronunciation habits.

[0044] s5.3. Under dynamic programming, sequentially calculate the Euclidean distance between the nth feature vector in the detection template and its corresponding kth feature vector in the matching template, and record this distance as the matching distance d(n, k) in the matching distance matrix d. The dynamic programming is as follows: calculate the cumulative distances D1, D2, and D3 of the three possible predecessor feature vectors (n, k). Then, select the minimum of the three cumulative distances D1, D2, and D3, add it to the matching distance d(n, k), and record this as the cumulative distance of (n, k) in the cumulative distance matrix D.

[0045] s5.4. If the current cumulative distance D(n, k) exceeds the set range, the calculation is terminated and the result D(N, K) = ∞ is output; otherwise, the distance calculation between all feature vectors is completed and the actual calculation result of D(N, K) is output.

[0046] s5.5. Repeat s5.2 to s5.4 to calculate the distance between the detection template and each group of matching templates, and then take the label corresponding to the matching template with the smallest distance as the detection result.

Claims

1. A speech recognition method based on an STO array and a dynamic time planning algorithm, characterized in that: The method specifically comprises the following steps: Step 1: Collect the speech signal to be recognized, filter out noise through preprocessing, perform endpoint detection, output the time domain speech signal, perform fast Fourier transform on the time domain speech signal, and calculate the power spectrum of the frequency domain signal; Step 2: Divide the power spectrum obtained in step 1 into multiple frequency bands from low to high frequency, then amplify the average frequency amplitude in each frequency band by the same multiple as the polarization current value, and input it into an STO array composed of multiple identical STOs, and obtain the corresponding oscillation frequency at the output end; the number of STOs in the STO array is greater than the number of frequency bands into which the power spectrum is divided; Step 3: Arrange the oscillation frequencies output by the STO array in the order of the frequency band division of the power spectrum to obtain a frequency vector, calculate the differential parameter of the frequency vector, and obtain the speech feature vector {T(1), T(2), T(3), …, T(n), .., T(N)} of the speech signal to be recognized as a detection template; Step 4: Collect the speech signals with known labels, process them through the methods of steps 1 to 3, and use the speech feature vectors {R(1), R(2), R(3), …, R(m), …, R(M)} of each known speech signal as a matching template and save them in the matching database; Step 5: Input the detection template obtained in step 3 and the matching module stored in the matching database into the improved dynamic time planning algorithm, calculate the Euclidean distance between the two speech feature vectors, and use the label of the matching module with the smallest distance value as the recognition result of the detection template to complete the speech recognition; the improved dynamic time planning algorithm sets a limit range for the dynamic distance during the distance calculation process. When the cumulative distance exceeds the set range, the calculation is terminated and the calculation result is set to infinity.

2. The speech recognition method based on the STO array and dynamic time planning algorithm as claimed in claim 1, characterized in that: The preprocessing method in step 1 is to perform pre-emphasis, framing and windowing on the signal in sequence.

3. The speech recognition method based on the STO array and dynamic time planning algorithm as claimed in claim 1, characterized in that: The STOs that make up the STO array can oscillate stably without their magnetic moments flipping.

4. The speech recognition method based on the STO array and dynamic time planning algorithm according to claim 1 or 3, characterized in that: The STO includes a free layer, a non-magnetic layer and a fixed layer from top to bottom, wherein the material of the free layer is CoFeB, with a diameter of 600nm, a height of 12nm and a magnetic moment density of 5nm; the material of the non-magnetic layer is Mgo, and the material of the fixed layer is CoFeB.

5. The speech recognition method based on the STO array and dynamic time planning algorithm according to claim 1 or 3, characterized in that: The polarization current value input into the STO array is within the STO driving current range.

6. The speech recognition method based on the STO array and dynamic time planning algorithm as claimed in claim 1, characterized in that: The calculation method of the differential parameter of the nth frequency in the frequency vector is: Where C is the frequency parameter.

7. The speech recognition method based on the STO array and dynamic time planning algorithm as claimed in claim 1, characterized in that: Step 5: The specific steps are as follows: s5.

1. Define two cumulative distance matrices D and matching distance matrices d of size N*K, where N and K are the number of feature vectors in the detection template and the matching template, respectively. s5.

2. Under dynamic programming, sequentially calculate the Euclidean distance between the nth feature vector in the detection template and its corresponding kth feature vector in the matching template, and record it as the matching distance d(n, k) in the matching distance matrix d. The dynamic programming is as follows: calculate the cumulative distances D1, D2, and D3 of the three possible predecessor feature vectors (n, k); then select the minimum of the three cumulative distances D1, D2, and D3 and add it to the matching distance d(n, k), and record it as the cumulative distance (n, k) in the cumulative distance matrix D. s5.

3. If the current cumulative distance D(n, k) exceeds the set range, the calculation ends and the result D(N, K) = ∞ is output; Otherwise, the distance calculation between all feature vectors is completed and the actual calculation result of D(N, K) is output; s5.

4. Repeat s5.1 to s5.3 to calculate the distance between the detection template and each matching template, and then take the label corresponding to the matching template with the smallest distance as the detection result.

Images