Single event upset resistant reinforcement method based on Transform speech recognition model

Abstract

The invention discloses an anti-single event upset reinforcement method based on a Transform speech recognition model, and the method is used for improving the speech recognition efficiency and accuracy of astronauts in a spaceflight task, and comprises the steps: constructing a training set, and carrying out the preprocessing of the data of the training set; the preprocessed training set is input into a Transform voice recognition model, and a trained Transform voice recognition model is obtained; parameters in the trained Transform speech recognition model are modified, a single event upset event and errors possibly occurring in a high-radiation environment are simulated, and an SEU influence model is established; and based on the SEU influence model, carrying out robustness analysis on the Transform speech recognition model, determining a sensitive module, and carrying out triple modular redundancy technology reinforcement on the sensitive module to obtain a single event upset resistant reinforced speech recognition model.

Description

Technical Field

[0001] The present invention relates to the fields of aerospace and aviation, deep learning and radiation tolerance technology, and in particular to a single-particle upset resistance reinforcement method based on a Transformer speech recognition model. Background Art

[0002] With the continuous advancement of deep space exploration missions, the importance of intelligent and autonomous systems in the aerospace field has become increasingly prominent. In high-radiation environments, especially in deep space missions, astronauts need to rely on voice recognition technology for human-computer interaction when performing complex tasks to improve the efficiency and accuracy of task execution. However, since the radiation level in the space environment is significantly higher than that in the ground environment, single-event upsets (SEUs) will occur in semiconductor components, which may cause parameter errors and inaccurate outputs in deep learning models when facing radiation effects. This puts the robustness and reliability of existing models in high-radiation environments under severe tests.

[0003] At present, the reinforcement methods for single-particle upsets mainly include triple-mode redundancy (TMR) and error detection and correction code (ECC). Triple-mode redundancy prevents errors caused by single-particle upsets by replicating the computing unit three times and processing it in parallel to implement a majority voting mechanism. Although TMR is effective in improving system reliability, its hardware overhead is large, especially in complex deep learning models, this overhead is often unbearable. Error detection and correction code (ECC) is another common reinforcement scheme that encodes data to ensure that the correct data can be recovered when an error occurs. However, the application of ECC in deep learning models may increase computing latency, and the design and implementation of complex models is relatively complex. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and propose an anti-single-event upset reinforcement method based on the Transformer speech recognition model.

[0005] In order to achieve the above object, the present invention proposes a single-event upset reinforcement method based on a Transformer speech recognition model, which is used to improve the efficiency and accuracy of speech recognition of astronauts in space missions, including:

[0006] Step 1) construct a training set and preprocess the training set data;

[0007] Step 2) inputting the preprocessed training set into the Transformer speech recognition model to obtain a trained Transformer speech recognition model;

[0008] Step 3) Modify the parameters in the trained Transformer speech recognition model, simulate single-event upset events and errors that may occur in a high-radiation environment, and establish an SEU impact model;

[0009] Step 4) Based on the SEU impact model, perform robustness analysis on the Transformer speech recognition model, identify sensitive modules, and reinforce the sensitive modules with triple-module redundancy technology to obtain a speech recognition model that is reinforced against single-particle upsets.

[0010] Preferably, the step 1) comprises:

[0011] Use LibriSpeech, an open source dataset that includes speech and corresponding text transcriptions;

[0012] For speech, use the torchaudio library to read the audio file in LibriSpeech and extract the audio signal; calculate the filter bank energy characteristics of the audio signal;

[0013] For text, a subword-based segmentation method is used to divide text data into smaller units.

[0014] Preferably, the Transformer speech recognition model in step 2) includes:

[0015] A feature extractor, used to extract meaningful feature information from the input audio;

[0016] 6 encoders, each consisting of a multi-head self-attention mechanism and a feedforward neural network, used to encode input features;

[0017] 6 decoders are used to decode the features output by the encoder and finally generate the corresponding text output.

[0018] Preferably, the step 3) comprises:

[0019] Convert the weight parameter value of the trained Transformer speech recognition model into a 32-bit binary form to simulate a single-particle upset event. The 32-bit binary includes: 1 sign bit, 8 exponent bits, and 32 mantissa bits;

[0020] Randomly change a bit in the binary representation from "1 to 0" or "0 to 1" to simulate the errors that may occur in a high radiation environment, and then obtain the SEU impact model.

[0021] Preferably, the sensitive module in step 4) includes: a multi-head self-attention mechanism of the first two encoders.

[0022] Preferably, the triple-module redundancy technology is used to reinforce the sensitive module to obtain a speech recognition model reinforced against single-event upset, including:

[0023] The computing unit of the sensitive module is replicated three times to form three computing units running in parallel;

[0024] A majority voting mechanism is implemented for three computing units running in parallel. When one of the computing units is affected by a single-particle upset, the correct output is selected by taking two out of three.

[0025] A speech recognition model reinforced against single-particle upset is obtained.

[0026] Compared with the prior art, the advantages of the present invention are:

[0027] 1. Targeted reinforcement: This invention conducts a detailed robustness analysis on the modules of the Transformer speech recognition model, identifies the modules that are sensitive to single event upset (SEU), and then performs targeted reinforcement. This method avoids the waste of resources in traditional reinforcement methods such as checksums and triple module redundancy (TMR), and achieves more accurate protection;

[0028] 2. Resource optimization: Compared with the traditional comprehensive reinforcement method, the present invention only applies triple-module redundancy technology to key modules, significantly reducing hardware overhead and computational complexity, thereby reducing the demand for system resources while ensuring system performance. This makes it more practical in aerospace applications with limited resources;

[0029] 3. Performance improvement: By separately reinforcing the modules that have a greater impact on SEU, the stability and reliability of the model in a high-radiation environment are ensured, thereby effectively improving the accuracy and response speed of speech recognition, providing safer and more efficient support for astronauts' operations;

[0030] 4. Scalability and compatibility: The method of the present invention has good scalability. Different modules can be reinforced and configured according to the needs of different application scenarios, providing a flexible solution for the application of Transformer speech recognition models in a variety of extreme environments in the future. The reinforcement method of the present invention is compatible with existing deep learning models and their training frameworks, and does not require major structural changes to the model, which is easy to integrate and implement, reducing the difficulty of deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of the single-event upset resistance reinforcement method based on the Transformer speech recognition model of the present invention;

[0032] Figure 2 It is the Transformer model structure;

[0033] Figure 3 It is the modified Transformer model structure. DETAILED DESCRIPTION

[0034] The present invention provides a method for strengthening the Transformer speech recognition model against single-particle upsets. By analyzing the robustness of the parameters of each module in the model under single-particle upsets, the modules that are more sensitive to SEU and have a greater impact on the model output results are identified, and triple-module redundancy (TMR) reinforcement is applied to these modules separately. Under the premise of ensuring the performance of the model, this method improves the stability and reliability of the Transformer speech recognition model in a high-radiation environment with less resource consumption.

[0035] The main steps include:

[0036] 1. Module robustness analysis: For different modules in the Transformer speech recognition model, such as the embedding layer, multi-head attention mechanism, and feedforward network layer, evaluate the sensitivity of their parameters under single-particle upset conditions to determine which modules are more vulnerable to SEU.

[0037] 2. Selective reinforcement: The sensitive modules identified in the robustness analysis (such as weight parameters in the attention mechanism or computing nodes in the feedforward network) are reinforced with triple-module redundancy, while other modules maintain their original structure to reduce overhead.

[0038] 3. Reinforcement verification: The performance of the reinforced model is evaluated through simulation verification and experiments to ensure the robustness of the model in a high radiation environment.

[0039] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments, and it is explained how to use the anti-single-particle upset reinforcement method of the present invention to improve the stability and reliability of the Transformer speech recognition model in a high-radiation environment.

[0040] Example

[0041] like Figure 1 As shown, an embodiment of the present invention provides a single-event upset reinforcement method based on a Transformer speech recognition model, which aims to improve the efficiency and accuracy of astronauts' speech recognition in space missions with less performance resource consumption. The method includes the following steps:

[0042] Step 1) Construct a training set and preprocess the training set data; specifically:

[0043] The training data used in this paper is the open source dataset LibriSpeech, which mainly includes a large amount of speech and corresponding text transcription. The specific steps of data preprocessing are as follows:

[0044] 1. Audio processing: Use the torchaudio library to read the audio files in LibriSpeech and extract the audio signals.

[0045] 2. Feature extraction: Calculate the FBank (filter bank energy) features of the audio signal for subsequent model training. The FBank feature can effectively capture the audio features in the speech signal and help improve the accuracy of recognition.

[0046] 3. Text processing: For the text part, a subword-based segmentation method is used to divide the text data into smaller units to meet the input requirements of the Transformer model. This method not only reduces the size of the vocabulary, but also enhances the model's ability to process unseen words.

[0047] Step 2) inputting the preprocessed training set into the Transformer speech recognition model to obtain a trained Transformer speech recognition model;

[0048] The Transformer model structure used in the present invention is as follows: Figure 2 As shown:

[0049] 1. Feature extractor: used to extract meaningful feature information from the input audio.

[0050] 2. Encoder: Contains 6 layers of encoders, each layer consists of a multi-head self-attention mechanism and a feedforward neural network, responsible for encoding the input features.

[0051] 3. Decoder: It also contains 6 layers of decoders, which decodes the features output by the encoder and finally generates the corresponding text output.

[0052] During the model building process, the torch library was used to manage and optimize the model parameters to ensure that the model can run efficiently.

[0053] Step 3) Modify the parameters in the trained Transformer speech recognition model, simulate single-event upset events and errors that may occur in a high-radiation environment, and establish an SEU impact model;

[0054] Considering that the parameters of the Transformer model are stored in SRAM storage cells, their storage form is essentially binary. According to the IEEE754 protocol, the representation of 32-bit floating point numbers is as follows:

[0055] Sign bit (S): 1 bit

[0056] Exponent bit (E): 8 bits

[0057] Mantissa (M): 23 digits

[0058] Therefore, when a single event upset occurs, a certain bit of the model weight may change. To this end, the steps of establishing the SEU impact model in the present invention are as follows:

[0059] 1. Convert the model weight parameter values ​​to 32-bit binary form to simulate a single-particle upset event.

[0060] 2. Randomly change a bit in the binary representation from "1 to 0" or "0 to 1" to simulate errors that may occur in a high radiation environment.

[0061] Step 4) Based on the SEU impact model, perform robustness analysis on the Transformer speech recognition model, identify sensitive modules, and reinforce the sensitive modules with triple-module redundancy technology to obtain a speech recognition model that is reinforced against single-particle upsets.

[0062] Robustness analysis steps:

[0063] 1. Deploy the Transformer speech recognition model into a simulation environment and analyze the sensitivity of the parameters of each module to errors by injecting single-particle upset faults.

[0064] 2. Analyze the main modules in the model (embedding layer, multi-head attention mechanism, feedforward network layer, decoder, etc.) one by one to evaluate the impact of parameter flipping on speech recognition performance.

[0065] Triple-mode redundancy reinforcement steps:

[0066] For the sensitive modules identified in the robustness analysis (the multi-head self-attention mechanism of the first two layers of encoders), triple-module redundancy technology is used for reinforcement, such as Figure 3 The specific implementation includes:

[0067] 1. Copy the computing unit: Copy the computing unit of the sensitive module (the multi-head self-attention module of the first two layers of encoder) three times to form three computing units running in parallel.

[0068] 2. Implement majority voting mechanism: When one of the computing modules is affected by a single-particle upset, the correct output is selected by taking two out of three to ensure the accuracy of the final result. This mechanism effectively prevents speech recognition errors caused by single-particle upsets.

[0069] Simulation verification steps:

[0070] The reinforced Transformer speech recognition model was simulated and verified, with single-particle upset faults injected to test the differences in recognition accuracy, processing delay, and hardware overhead before and after reinforcement. The performance of the model in a real radiation environment was also evaluated through experimental data.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. A single-event upset reinforcement method based on the Transformer speech recognition model, used to improve the efficiency and accuracy of astronauts' speech recognition in space missions, including: Step 1) construct a training set and preprocess the training set data; Step 2) inputting the preprocessed training set into the Transformer speech recognition model to obtain a trained Transformer speech recognition model; Step 3) Modify the parameters in the trained Transformer speech recognition model, simulate single-event upset events and errors that may occur in a high-radiation environment, and establish an SEU impact model; Step 4) Based on the SEU impact model, perform robustness analysis on the Transformer speech recognition model, identify sensitive modules, and reinforce the sensitive modules with triple-module redundancy technology to obtain a speech recognition model that is reinforced against single-particle upsets.

2. The single event upset resistance reinforcement method based on the Transformer speech recognition model according to claim 1 is characterized in that: The step 1) comprises: Use LibriSpeech, an open source dataset that includes speech and corresponding text transcriptions; For speech, use the torchaudio library to read the audio file in LibriSpeech and extract the audio signal; calculate the filter bank energy characteristics of the audio signal; For text, a subword-based segmentation method is used to divide text data into smaller units.

3. The single event upset resistance reinforcement method based on the Transformer speech recognition model according to claim 1 is characterized in that: The Transformer speech recognition model in step 2) includes: A feature extractor, used to extract meaningful feature information from the input audio; 6 encoders, each consisting of a multi-head self-attention mechanism and a feedforward neural network, used to encode input features; 6 decoders are used to decode the features output by the encoder and finally generate the corresponding text output.

4. The single event upset resistance reinforcement method based on the Transformer speech recognition model according to claim 1 is characterized in that: The step 3) comprises: Convert the weight parameter value of the trained Transformer speech recognition model into a 32-bit binary form to simulate a single-particle upset event. The 32-bit binary includes: 1 sign bit, 8 exponent bits, and 32 mantissa bits; Randomly change a bit in the binary representation from "1 to 0" or "0 to 1" to simulate the errors that may occur in a high radiation environment, and then obtain the SEU impact model.

5. The single event upset resistance reinforcement method based on the Transformer speech recognition model according to claim 3 is characterized in that: The sensitive module in step 4) includes: a multi-head self-attention mechanism of the first two encoders.

6. The single event upset resistance reinforcement method based on the Transformer speech recognition model according to claim 5 is characterized in that: The triple-module redundancy technology is used to reinforce the sensitive module to obtain a speech recognition model that is reinforced against single-event upset, including: The computing unit of the sensitive module is replicated three times to form three computing units running in parallel; A majority voting mechanism is implemented for three computing units running in parallel. When one of the computing units is affected by a single-particle upset, the correct output is selected by taking two out of three. A speech recognition model reinforced against single-particle upset is obtained.

Images