Device for intelligently recording switching operation process of transformer substation

Abstract

The invention discloses a device for intelligently recording a switching operation process of a transformer substation. The device comprises an audio acquisition module, a feature extraction module, a deep learning model module, a post-processing module, a user interface module, a storage module and a power management module, the device for intelligently recording the switching operation process of the transformer substation can work in a network-free environment, and the use flexibility is improved. And a Faster-Whisper model is adopted, so that the recognition speed is greatly improved, and nearly real-time speech recognition is realized. And through a deep learning model and post-processing optimization, the recognition accuracy is remarkably improved. The device can be deployed in small equipment such as a recording pen or a mobile phone, and is convenient for a user to carry and use. And a model quantification technology is adopted, so that the requirement on equipment hardware is reduced, and the system can operate on low-power-consumption equipment.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recording technology, and in particular to a device for intelligent recording of a switching operation process of a transformer substation. Background Art

[0002] Substation switching operation recording is an effective monitoring measure to strengthen the management of the entire switching operation process. Through recording, the integrity and standardization of the operation process can be ensured, thereby preventing the occurrence of electrical misoperation. Through recording, problems in the operation can be discovered and corrected in a timely manner. If misoperation or accidents occur, the cause can be found out, the responsibility can be analyzed, and corresponding corrective measures can be taken by listening to the recording files.

[0003] At present, the voice recognition technology on the market is mainly used in smart phones, smart speakers and other devices, but these devices usually need to be connected to the Internet for voice recognition, and cannot be used without a network environment in the substation switching operation environment. In addition, the accuracy and real-time performance of traditional voice recognition systems need to be improved, especially in noisy environments such as substation switching operation environments. The recognition effect is not ideal. Existing portable recording devices such as voice recorders usually only have recording functions and lack the ability to convert real-time speech to text. Users need to export the recording files to a computer for transcription. This method is inefficient and cannot meet the immediate needs of substation switching operation recordings. Therefore, it is necessary to propose a device for intelligent recording of substation switching operation processes to solve the above problems. Summary of the invention

[0004] The present invention provides a device for intelligently recording the switching operation process of a substation, so as to solve the problem that the existing recording device cannot meet the recording demand of the switching operation of the substation.

[0005] The present invention provides a device for intelligent recording of a substation switching operation process, comprising: an audio acquisition module, a feature extraction module, a deep learning model module, a post-processing module, a user interface module, a storage module, and a power management module; The audio acquisition module comprises: A microphone array is used to collect sound signals and convert them into electrical signals; Preamplifier, used to amplify the signal to a suitable level, with the gain set to 40dB; ADC, used to sample the signal at a sampling rate of 96kHz and 24-bit accuracy; DSP, used to pre-process the digital signal and transmit the pre-processed audio data to the feature extraction module; The feature extraction module is used to perform pre-emphasis processing, frame processing, windowing processing, fast Fourier transform, Mel filter bank processing, logarithmic operation, discrete cosine transform, and dynamic feature extraction on the pre-processed audio signal; the feature vector finally obtained includes 13-dimensional MFCC coefficients, 13-dimensional first-order difference coefficients and 13-dimensional second-order difference coefficients, with a total of 39-dimensional features; The deep learning model module is used to convert the extracted features into text; The post-processing module is used to optimize and correct the original output of the model; The user interface module is used to interact with the user, display the recognition results and provide corresponding operation functions; The storage module is used to manage the data storage of the device; The power management module is used to control the power supply and power consumption of the device.

[0006] Furthermore, the DSP is used to perform the following preprocessing on the digital signal: Use microphone arrays to localize sound sources and enhance directionality; Use the minimum mean square error algorithm to suppress environmental noise; Uses an adaptive filtering algorithm to eliminate echo generated by the device's speakers.

[0007] Furthermore, the feature extraction module is used to perform the following processing on the preprocessed audio signal: The input signal x(n) is pre-emphasized to enhance the high frequency part: y(n) = x(n) α * x(n-1) Wherein, y(n) is the signal after pre-emphasis processing; α is the pre-emphasis coefficient, and its value range is 0.9-1.0; The continuous audio signal is divided into short time frames, each frame lasts for 25ms and the frame shift is 10ms; for a sampling rate of 96kHz, each frame contains 2400 sampling points, and adjacent frames overlap by 1440 sampling points; Apply a Hamming window function to each frame of the signal to reduce spectral leakage: w(n) = 0.54 0.46 * cos(2πn / (N-1)) Wherein, w(n) is the windowed signal; N is the frame length, and n is an integer from 0 to N-1; Perform 2048-point fast Fourier transform on the windowed signal to obtain the spectrum; Map the linear spectrum to the Mel frequency scale using 40 triangular filters; Take the logarithm of the Mel filter output to compress the dynamic range; Perform discrete cosine transform on the logarithmic Mel spectrum to obtain Mel frequency cepstrum coefficients; Compute the first- and second-order difference coefficients of Mel-frequency cepstral coefficients to capture the dynamic characteristics of speech.

[0008] Furthermore, the deep learning model module converts the extracted features into text based on the Faster-Whisper model of the Transformer architecture. The model structure includes: Encoder: The encoder is composed of multiple stacked Transformer encoding layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network; Decoder: The decoder consists of multiple Transformer decoding layers, including multi-head self-attention, encoder-decoder attention, and feed-forward neural networks; Input embedding layer, used to map the input feature sequence into a high-dimensional embedding space; Position encoding, used to add position information to the input sequence, using the sine-cosine position encoding method; The output layer is used to map the output of the decoder to the vocabulary space to obtain the final recognition result.

[0009] Furthermore, the model training and optimization steps include: Collect and process Chinese speech data sets, including various accents and background noises, with a total duration of no less than 10,000 hours; Use the Chinese speech dataset for pre-training, the training time is no less than 7 days, and 8 NVIDIA A100 GPUs are used; Fine-tune the dataset of the target scene for no less than 48 hours; Distill the knowledge of the large model into a small model to reduce the model size and computational complexity. The distillation process uses a soft label with a temperature of 2 and the distillation time is no less than 72 hours. Perform 8-bit fixed-point quantization on model weights and convert floating-point operations to fixed-point operations to increase inference speed and reduce memory usage; Use L1 regularization and gating mechanisms to prune the model, reducing the number of parameters by 20%-30% while keeping the accuracy loss within 1%; The model specifications are as follows: Number of encoder layers: 6 Decoder layers: 6 Number of attention heads: 8 Model Dimensions: 512 Feedforward network dimension: 2048 Vocabulary size: 50,000.

[0010] Furthermore, the model reasoning process includes: Input the 39-dimensional feature sequence output by the feature extraction module into the model; The encoder processes the input sequence and generates a contextual representation; The decoder generates the output sequence step by step, focusing on the encoder output and the generated part at each step; Use a beam search algorithm with a width of 5 to select the best result from multiple candidate sequences; The model outputs a probability distribution of characters at each time step.

[0011] Furthermore, the post-processing module optimizes and corrects the original output of the model, including the following steps: Use the 5-gram language model to re-score the candidate results of the beam search. The training corpus of the language model should be no less than 10GB of plain text data. P(W) =λ* P_acoustic(W) + (1-λ) * P_lm(W) Among them, λ is the weight coefficient of the acoustic model and the language model, ranging from 0.5 to 0.8; Use the BERT-based sequence labeling model to predict punctuation marks. The model structure is: BERT encoding layer, bidirectional LSTM layer, fully connected layer, CRF layer; punctuation marks include period, comma, question mark, and exclamation mark; Convert colloquial numerical expressions into standard numerical writing; identify and standardize common units; Construct a homophone dictionary and correct homophone errors based on context information, using the edit distance algorithm with a threshold of 2; Maintain a domain dictionary to match and replace relevant segments in the recognition results; Use the pre-trained BERT model to understand the semantics of sentences and detect and correct semantically incoherent parts; The recognition results of consecutive short speech segments are merged, and the sliding window smoothing technique is used to eliminate the incoherence between segments.

[0012] Furthermore, the user interface module includes: Display screen, using 3.5-inch touch screen, resolution of 640x480, supports capacitive touch; Button group, including power button, recording start / pause button, volume adjustment button; Indicator light, used to display the working status of the device, including power indicator, recording indicator, and recognition status indicator; Graphical user interface, developed based on the Qt framework, with the following functions: Display recognized text in real time, support automatic scrolling and highlight the currently recognized part; Provide text editing function, allowing users to manually correct recognition errors; File management interface, supports browsing, playing and exporting recognized audio files; Settings interface, used to adjust recognition parameters, audio input and output options; Battery level and storage space display; The workflow of the user interface module includes: After the system starts, the welcome interface is displayed, and the user can choose to start a new recognition task or open an existing file; After entering the recognition interface, the recognition results are displayed in real time, with an update interval of 300ms to ensure smooth display; Users use the touch screen to select, copy, and paste text, and long-press a word to pop up replacement suggestions; During the recognition process, the current audio processing progress and the remaining time estimate are displayed through a progress bar; After the recognition is completed, it provides options for saving, exporting, and sharing, and supports multiple formats.

[0013] Furthermore, the storage module includes: Internal storage uses a 64GB eMMC storage chip with a read speed of no less than 250MB / s and a write speed of no less than 150MB / s; Expandable storage, supports microSD card expansion storage up to 512GB; File system, using ext4 file system, supports log function; The storage management strategy of the storage module is as follows: The original audio is stored in a lossless compressed format; The text results are stored in UTF-8 encoding and timestamp information is recorded; Automatically back up new data to the SD card every 30 minutes; When the remaining space is less than 10%, the user is automatically reminded and a quick cleanup option is provided; AES-256 encryption algorithm is used to encrypt and store sensitive data.

[0014] Furthermore, the power management module includes: Battery, uses 3000mAh lithium polymer battery, supports fast charging; Power management chip, used for voltage regulation, battery charge and discharge management, and power supply control of each module; The power consumption monitoring unit is used to monitor the power consumption of each module in real time and adjust the working mode as needed; The power management strategy of the power management module is as follows: Adjust the CPU and GPU operating voltage in real time according to system load; After 30 seconds of no operation, it will automatically enter low-power standby mode, and the wake-up time will not exceed 1 second; Dynamically switch unnecessary hardware modules based on usage scenarios; Automatically adjust recognition accuracy and response speed according to battery power level, giving priority to core functions when battery is low; Monitor device temperature and actively reduce system performance to avoid overheating when the temperature exceeds 45°C; The device for intelligent recording of the switching operation process of a substation has all modules integrated on a PCB with a size not exceeding 100mm x 60mm x 15mm and a multi-layer PCB design; a graphene heat sink and an internal copper foil heat dissipation channel are used to ensure that the core chip temperature does not exceed 60°C under normal working conditions; an embedded operating system based on Linux is adopted, POSIX threads are used to realize multi-task parallel processing, and each functional module communicates through a message queue and shared memory; a real-time scheduler is used to assign a higher priority to key tasks to ensure that the processing delay does not exceed 50ms; a memory pool technology is used to pre-allocate commonly used objects to reduce fragmentation and delay caused by dynamic memory allocation; the NEON SIMD instruction set is used to optimize the feature extraction and model inference process; the recognition parameters are automatically adjusted according to the environmental noise level and the user's speaking speed; a secure boot mechanism is implemented, a hardware trust root is used to verify the system integrity, all network communications are encrypted using the TLS 1.3 protocol, and access control and audit functions are provided when data is exported; I2C and SPI interfaces are reserved to support external connection of other sensors or modules; the software architecture adopts a modular design; the user's usage habits are recorded and the personalized recognition model is gradually optimized.

[0015] The present invention has the following beneficial effects: A device for intelligent recording of the switching operation process of a substation according to the present invention can work in a network-free environment, thereby improving the flexibility of use. The Faster-Whisper model is adopted to greatly improve the recognition speed and realize near real-time speech recognition. The recognition accuracy is significantly improved through deep learning models and post-processing optimization. It can be deployed in small devices such as voice recorders or mobile phones, making it convenient for users to carry and use. The model quantization technology is adopted to reduce the requirements for device hardware, so that the system can run on low-power devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic diagram of the composition of a device for intelligently recording the switching operation process of a substation provided by the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The technical solutions provided by the embodiments of the present invention are described in detail below in conjunction with the drawings.

[0019] See also Figure 1 , which is a device for intelligent recording of substation switching operation process provided by the present invention, including: an audio acquisition module 101, a feature extraction module 102, a deep learning model module 103, a post-processing module 104, a user interface module 105, a storage module 106 and a power management module 107.

[0020] The audio acquisition module 101 is responsible for collecting the user's voice signal in real time, and it mainly consists of the following components: Microphone array: It consists of multiple highly sensitive MEMS microphones arranged in a ring to achieve 360-degree omnidirectional sound pickup. The number of microphones can be 2-8, preferably 4.

[0021] Preamplifier: used to amplify the weak signal collected by the microphone, it adopts low-noise operational amplifier design, and the gain can be adjusted in the range of 20dB-60dB.

[0022] Analog-to-digital converter (ADC): Converts the amplified analog signal into a digital signal. It uses a 24-bit high-precision ADC with a sampling rate of 48kHz or 96kHz.

[0023] Digital Signal Processor (DSP): used to achieve preliminary noise suppression and sound source localization. It uses a low-power DSP chip with a main frequency range of 100MHz-300MHz.

[0024] The workflow of audio collection is as follows: Step 1: The microphone array collects sound signals and converts them into electrical signals.

[0025] Step 2: The preamplifier amplifies the signal to a suitable level with the gain set to 40dB.

[0026] Step 3: The ADC samples the signal at a sampling rate of 96kHz and 24-bit accuracy.

[0027] Step 4: DSP pre-processes the digital signal, including: Beamforming: Use microphone arrays to localize sound sources and enhance directionality, improving the signal-to-noise ratio by 30%-50%.

[0028] Adaptive noise cancellation: Adopts the least mean square error (LMS) algorithm to suppress environmental noise, with a noise reduction of up to 15dB-20dB.

[0029] Echo Cancellation: Uses an adaptive filtering algorithm to eliminate the echo generated by the device speaker, with echo attenuation reaching over 40dB.

[0030] Step 5: Transfer the preprocessed audio data to the feature extraction module.

[0031] The feature extraction module 102 is responsible for further processing and feature extraction of the pre-processed audio signal. It mainly includes the following steps: Step 1: Pre-emphasis The input signal x(n) is pre-emphasized to enhance the high frequency part: y(n) = x(n) α * x(n-1) Wherein, α is the pre-emphasis coefficient, the value range is 0.9-1.0, preferably 0.97.

[0032] Step 2: Framing The continuous audio signal is divided into short time frames, each frame lasts 25ms and the frame shift is 10ms. For a sampling rate of 96kHz, each frame contains 2400 samples, and adjacent frames overlap by 1440 samples.

[0033] Step 3: Add window Apply a Hamming window function to each frame of the signal to reduce spectral leakage: w(n) = 0.54 0.46 * cos(2πn / (N-1)) Wherein, N is the frame length, and n is an integer from 0 to N-1.

[0034] Step 4: Fast Fourier Transform (FFT) Perform 2048-point FFT on the windowed signal to obtain the spectrum.

[0035] Step 5: Mel filter bank Map the linear spectrum to the Mel frequency scale using 40 triangular filters.

[0036] Step 6: Logarithm Operation Take the logarithm of the Mel filter output to compress the dynamic range.

[0037] Step 7: Discrete Cosine Transform (DCT) Perform DCT on the logarithmic Mel spectrum to obtain Mel frequency cepstral coefficients (MFCC).

[0038] Step 8: Dynamic Feature Extraction The first-order and second-order difference coefficients of MFCC are calculated to capture the dynamic characteristics of speech.

[0039] The final feature vector contains 13-dimensional MFCC coefficients, 13-dimensional first-order difference coefficients, and 13-dimensional second-order difference coefficients, totaling 39-dimensional features.

[0040] The deep learning model module 103 is the core of the system and is responsible for converting the extracted features into text. The present invention adopts the Faster-Whisper model based on the Transformer architecture and performs targeted optimization. The model mainly includes the following parts: Encoder: The encoder is composed of multiple stacked Transformer encoding layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network.

[0041] Decoder: The decoder is also composed of multiple Transformer decoding layers, including multi-head self-attention, encoder-decoder attention and feedforward neural network.

[0042] Input embedding layer maps the input feature sequence to a high-dimensional embedding space.

[0043] Position encoding, adding position information to the input sequence, using the sine-cosine position encoding method.

[0044] The output layer maps the output of the decoder to the vocabulary space to obtain the final recognition result.

[0045] Model training and optimization: Step 1: Data Preparation Collect and process a large-scale Chinese speech dataset, including various accents, background noise, etc., with a total duration of no less than 10,000 hours.

[0046] Step 2: Pre-training A large-scale general speech dataset is used for pre-training, with a training time of no less than 7 days and 8 NVIDIA A100 GPUs.

[0047] Step 3: Fine-tuning Fine-tune using a dataset of the target scenario, such as meeting minutes, class notes, and other domain-specific data. The fine-tuning time is no less than 48 hours.

[0048] Step 4: Knowledge Distillation The knowledge of the large model is distilled into a small model to reduce the model size and computational complexity. The distillation process uses a soft label with a temperature of 2 and the distillation time is no less than 72 hours.

[0049] Step 5: Quantification The model weights are quantized to 8-bit fixed-point and floating-point operations are converted to fixed-point operations to increase inference speed and reduce memory usage.

[0050] Step 6: Pruning The model is pruned using L1 regularization and gating mechanisms, reducing the number of parameters by 20%-30% while keeping the accuracy loss within 1%.

[0051] Model specifications: Number of encoder layers: 6 Decoder layers: 6 Number of attention heads: 8 Model Dimensions: 512 Feedforward network dimension: 2048 Vocabulary size: 50,000 Model inference process: Step 1: Feature sequence input The 39-dimensional feature sequence output by the feature extraction module is input into the model.

[0052] Step 2: Encoder Processing The encoder processes the input sequence and generates a contextual representation.

[0053] Step 3: Decoder Generation The decoder generates the output sequence step by step, and at each step it pays attention to the encoder's output and the generated part.

[0054] Step 4: Beam Search A beam search algorithm with a width of 5 is used to select the best result from multiple candidate sequences.

[0055] Step 5: Output probability distribution The model outputs a probability distribution of characters at each time step.

[0056] The post-processing module 104 is responsible for optimizing and correcting the original output of the model to improve the quality of the final recognition result. It mainly includes the following steps: Step 1: Language model re-scoring Use the 5-gram language model to re-score the candidate results of the beam search. The training corpus of the language model is no less than 10GB of plain text data.

[0057] P(W) = λ * P_acoustic(W) + (1-λ) * P_lm(W) Here, λ is the weight coefficient of the acoustic model and the language model, ranging from 0.5 to 0.8, preferably 0.7.

[0058] Step 2: Punctuation Prediction Use the BERT-based sequence labeling model to predict punctuation marks. The model structure is: BERT Encoding Layer Bidirectional LSTM layer Fully connected layer CRF Layer Punctuation marks include period, comma, question mark, exclamation mark, etc.

[0059] Step 3: Normalize numbers and units Convert colloquial numerical expressions into standard numerical expressions, such as "one thousand two hundred and three" into "1230".

[0060] Recognize and normalize common units such as "kilogram", "kilometer", etc.

[0061] Step 4: Homophone Correction Build a homophone dictionary and correct possible homophone errors by combining context information. Use the edit distance algorithm with a threshold of 2.

[0062] Step 5: Custom dictionary matching Maintain a domain-specific dictionary (such as professional terms, names of people and places, etc.) to match and replace relevant fragments in the recognition results.

[0063] Step 6: Semantic Understanding and Error Correction Use the pre-trained BERT model to understand the semantics of sentences and detect and correct semantically incoherent parts.

[0064] Step 7: Merge and smooth the results The recognition results of consecutive short speech segments are merged, and the sliding window smoothing technique is used to eliminate the incoherence between segments.

[0065] The user interface module 105 is responsible for interacting with the user, displaying the recognition results and providing corresponding operation functions. It mainly includes the following parts: The display uses a 3.5-inch touch screen with a resolution of 640x480 and supports capacitive touch.

[0066] The button group includes physical buttons such as the power button, recording start / pause button, and volume adjustment button.

[0067] Indicator light, used to display the working status of the device, including power indicator, recording indicator, recognition status indicator, etc.

[0068] Graphical user interface (GUI), developed based on the Qt framework, with the following main functions: Display recognized text in real time, support automatic scrolling and highlighting of the currently recognized part.

[0069] Provides text editing function, allowing users to manually correct recognition errors.

[0070] The file management interface supports browsing, playing and exporting recognized audio files.

[0071] In the settings interface, you can adjust recognition parameters, audio input and output, and other options.

[0072] Battery level and storage space display.

[0073] User interface workflow: Step 1: After the system starts, the welcome interface is displayed and the user can choose to start a new recognition task or open an existing file.

[0074] Step 2: After entering the recognition interface, the recognition results are displayed in real time, with an update interval of 300ms to ensure smooth display.

[0075] Step 3: Users can use the touch screen to select, copy, and paste text. Long press a word to pop up replacement suggestions.

[0076] Step 4: During the recognition process, the current audio processing progress and the remaining time estimate are displayed through a progress bar.

[0077] Step 5: After the recognition is completed, options such as saving, exporting, and sharing are provided, and multiple formats such as TXT, DOC, PDF, etc. are supported.

[0078] The storage module 106 is responsible for managing the data of the system, including audio data, recognition results, user settings, etc. It mainly includes the following parts: The internal storage uses a 64GB eMMC storage chip with a read speed of no less than 250MB / s and a write speed of no less than 150MB / s.

[0079] Expandable storage, supports microSD card expansion storage up to 512GB.

[0080] The file system uses the ext4 file system, which supports the log function and improves data reliability.

[0081] Storage Management Policy: Step 1: Audio Data Storage The original audio is stored in a lossless compression format such as FLAC, with a compression ratio of about 50%.

[0082] Step 2: Storage of recognition results The text results are stored in UTF-8 encoding and record timestamp information to facilitate synchronization with the audio.

[0083] Step 3: Data backup Automatically back up new data to the SD card every 30 minutes, if an SD card exists.

[0084] Step 4: Storage space management When the remaining space is less than 10%, the user is automatically reminded and a quick cleanup option is provided.

[0085] Step 5: Data Encryption AES-256 encryption algorithm is used to encrypt and store sensitive data.

[0086] The power management module 107 is responsible for controlling the power supply and power consumption of the system to extend the use time of the device. It mainly includes the following parts: The battery uses a 3000mAh lithium polymer battery and supports fast charging.

[0087] The power management chip (PMIC) is responsible for voltage regulation, battery charge and discharge management, and power supply control of each module.

[0088] The power consumption monitoring unit monitors the power consumption of each module in real time and adjusts the working mode as needed.

[0089] Power Management Policy: Step 1: Dynamic Voltage Scaling Adjust the operating voltage of the CPU and GPU in real time according to the system load, reducing power consumption by 10%-20%.

[0090] Step 2: Smart Sleep After detecting no operation for 30 seconds, it automatically enters low-power standby mode, and the wake-up time does not exceed 1 second.

[0091] Step 3: Module switch control Dynamically switch unnecessary hardware modules based on usage scenarios, such as turning off the display backlight in pure recognition mode.

[0092] Step 4: Adaptive calculation Automatically adjust recognition accuracy and response speed based on battery power level, giving priority to core functions when battery is low.

[0093] Step 5: Temperature Management Monitors device temperature and proactively reduces system performance to avoid overheating when the temperature exceeds 45°C.

[0094] Specifically, the system integration and optimization are as follows: Hardware Integration: All modules are integrated on a compact PCB with dimensions no larger than 100mm x 60mm x 15mm. Multi-layer PCB design is used to optimize signal integrity and electromagnetic compatibility.

[0095] Thermal Design: Use graphene heat sink and internal copper foil heat dissipation channel to ensure that the core chip temperature does not exceed 60°C under normal operating conditions.

[0096] Software architecture: It uses an embedded operating system based on Linux and uses POSIX threads to implement multi-task parallel processing. Each functional module communicates through message queues and shared memory.

[0097] Real-time optimization: Use the real-time scheduler to assign higher priority to critical tasks such as audio acquisition and feature extraction, ensuring that processing delays do not exceed 50ms.

[0098] Memory management: Use memory pool technology to pre-allocate commonly used objects and reduce fragmentation and latency caused by dynamic memory allocation.

[0099] Parallel computing: Use the NEON SIMD instruction set to optimize the feature extraction and model inference process, improving computing efficiency by 30%-50%.

[0100] Adaptive function: The system can automatically adjust recognition parameters according to factors such as the ambient noise level and the user's speaking speed, such as adjusting the threshold of voice activity detection and changing the window size of feature extraction.

[0101] Security design: Implement a secure boot mechanism and use hardware trust root to verify system integrity. All network communications are encrypted using TLS 1.3 protocol. Provide access control and audit functions when exporting data.

[0102] Scalability: I2C, SPI and other interfaces are reserved to support external connection of other sensors or modules. The software architecture adopts modular design to facilitate subsequent function expansion.

[0103] User habit learning: The system will record the user's usage habits, such as common vocabulary, speaking style, etc., and gradually optimize the personalized recognition model.

[0104] In addition, other lightweight speech recognition models, such as DeepSpeech, Wav2Letter, etc., can also be considered to adapt to different hardware conditions and application scenarios. The feature extraction method can try to use filter bank features (FilterBank Features) or the original waveform directly as input. The post-processing module can integrate multiple language models, such as n-gram, LSTM, etc., to further improve the recognition accuracy.

[0105] Through the above detailed technical solutions, the device for intelligent recording of substation switching operation process of the present invention can provide high-precision and low-latency speech-to-text services in an offline environment, meeting the needs of users in various scenarios. The modular design and optimization strategy of the system ensure that it achieves excellent performance under limited hardware resources, while having good scalability and user experience.

[0106] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A device for intelligent recording of the switching operation process of a substation, characterized in that: include: An audio acquisition module (101), a feature extraction module (102), a deep learning model module (103), a post-processing module (104), a user interface module (105), a storage module (106), and a power management module (107); The audio acquisition module (101) comprises: A microphone array is used to collect sound signals and convert them into electrical signals; Preamplifier, used to amplify the signal to a suitable level, with the gain set to 40dB; ADC, used to sample the signal at a sampling rate of 96kHz and 24-bit accuracy; DSP, used for preprocessing the digital signal and transmitting the preprocessed audio data to the feature extraction module (102); The feature extraction module (102) is used to perform pre-emphasis processing, frame processing, windowing processing, fast Fourier transform, Mel filter bank processing, logarithmic operation, discrete cosine transform, and dynamic feature extraction on the pre-processed audio signal; the feature vector finally obtained includes 13-dimensional MFCC coefficients, 13-dimensional first-order difference coefficients and 13-dimensional second-order difference coefficients, with a total of 39-dimensional features; The deep learning model module (103) is used to convert the extracted features into text; The post-processing module (104) is used to optimize and correct the original output of the model; The user interface module (105) is used to interact with the user, display the recognition results and provide corresponding operation functions; The storage module (106) is used for managing the data storage of the device; The power management module (107) is used to control the power supply and power consumption of the device.

2. A device for intelligent recording of substation switching operation process as claimed in claim 1, characterized in that: The DSP is used to perform the following preprocessing on the digital signal: Use microphone arrays to localize sound sources and enhance directionality; Use the minimum mean square error algorithm to suppress environmental noise; Uses an adaptive filtering algorithm to eliminate echo generated by the device's speakers.

3. A device for intelligent recording of the switching operation process of a substation as claimed in claim 1, characterized in that The feature extraction module (102) is used to perform the following processing on the pre-processed audio signal: The input signal x(n) is pre-emphasized to enhance the high frequency part: y(n) = x(n) - α * x(n-1) Among them, y(n) is the signal after pre-emphasis processing; x(n) is the current sampling value; n is the index of the sampling point; α is the pre-emphasis coefficient, ranging from 0.9 to 1.0; The continuous audio signal is divided into short-time frames, each of which lasts 25ms and has a frame shift of 10ms. For a sampling rate of 96kHz, each frame contains 2400 sampling points, and adjacent frames overlap by 1440 sampling points. Apply a Hamming window function to each frame of signal to reduce spectral leakage: w(n) = 0.54 - 0.46 * cos(2πn / (N-1)) Where, w(n) is the windowed signal; N is the frame length, and n is an integer from 0 to N-1; Perform 2048-point fast Fourier transform on the windowed signal to obtain the spectrum; Map the linear spectrum to the Mel frequency scale using 40 triangular filters; Take the logarithm of the Mel filter output to compress the dynamic range; Perform discrete cosine transform on the logarithmic Mel spectrum to obtain Mel frequency cepstrum coefficients; Compute the first- and second-order difference coefficients of Mel-frequency cepstral coefficients to capture the dynamic characteristics of speech.

4. A device for intelligent recording of substation switching operation process as claimed in claim 1, characterized in that: The deep learning model module (103) is based on the Faster-Whisper model of the Transformer architecture, which converts the extracted features into text. The model structure includes: Encoder: The encoder is composed of multiple stacked Transformer encoding layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network; Decoder: The decoder consists of multiple Transformer decoding layers, including multi-head self-attention, encoder-decoder attention, and feed-forward neural networks; Input embedding layer, used to map the input feature sequence into a high-dimensional embedding space; Position encoding, used to add position information to the input sequence, using the sine-cosine position encoding method; The output layer is used to map the output of the decoder to the vocabulary space to obtain the final recognition result.

5. A device for intelligent recording of substation switching operation process as claimed in claim 4, characterized in that: The model training and optimization steps include: Collect and process Chinese speech data sets, including various accents and background noises, with a total duration of no less than 10,000 hours; Use the Chinese speech dataset for pre-training, the training time is no less than 7 days, and 8 NVIDIA A100 GPUs are used; Fine-tune the dataset of the target scene for no less than 48 hours; Distill the knowledge of the large model into a small model to reduce the model size and computational complexity. The distillation process uses a soft label with a temperature of 2 and the distillation time is no less than 72 hours. Perform 8-bit fixed-point quantization on model weights and convert floating-point operations to fixed-point operations to increase inference speed and reduce memory usage; Use L1 regularization and gating mechanisms to prune the model, reducing the number of parameters by 20%-30% while keeping the accuracy loss within 1%; The model specifications are as follows: Number of encoder layers: 6 Decoder layers: 6 Number of attention heads: 8 Model Dimensions: 512 Feedforward network dimension: 2048 Vocabulary size: 50,000.

6. A device for intelligent recording of the switching operation process of a substation as claimed in claim 5, characterized in that: The model inference process includes: Input the 39-dimensional feature sequence output by the feature extraction module into the model; The encoder processes the input sequence and generates a contextual representation; The decoder generates the output sequence step by step, focusing on the encoder output and the generated part at each step; Use a beam search algorithm with a width of 5 to select the best result from multiple candidate sequences; The model outputs a probability distribution of characters at each time step.

7. A device for intelligent recording of substation switching operation process as claimed in claim 6, characterized in that: The post-processing module (104) optimizes and corrects the original output of the model, including the following steps: Use the 5-gram language model to re-score the candidate results of the beam search. The training corpus of the language model should be no less than 10GB of plain text data. P(W) = λ * P_acoustic(W) + (1-λ) * P_lm(W) Among them, λ is the weight coefficient of the acoustic model and the language model, ranging from 0.5 to 0.8; P(W) is the total probability of the recognition result W; P_acoustic(W) is the probability of W given by the acoustic model; P_lm(W) is the probability of W given by the language model; Use the BERT-based sequence labeling model to predict punctuation marks. The model structure is: BERT encoding layer, bidirectional LSTM layer, fully connected layer, CRF layer; punctuation marks include period, comma, question mark, and exclamation mark; Convert colloquial numerical expressions into standard numerical writing; identify and standardize common units; Construct a homophone dictionary and correct homophone errors based on context information, using the edit distance algorithm with a threshold of 2; Maintain a domain dictionary to match and replace relevant segments in the recognition results; Use the pre-trained BERT model to understand the semantics of sentences and detect and correct semantically incoherent parts; The recognition results of consecutive short speech segments are merged, and the sliding window smoothing technique is used to eliminate the incoherence between segments.

8. The device for intelligent recording of the switching operation process of a substation as claimed in claim 1, characterized in that: The user interface module (105) comprises: Display screen, using 3.5-inch touch screen, resolution of 640x480, supports capacitive touch; Button group, including power button, recording start / pause button, volume adjustment button; Indicator light, used to display the working status of the device, including power indicator, recording indicator, and recognition status indicator; Graphical user interface, developed based on the Qt framework, with the following functions: Display recognized text in real time, support automatic scrolling and highlight the currently recognized part; Provide text editing function, allowing users to manually correct recognition errors; File management interface, supports browsing, playing and exporting recognized audio files; Settings interface, used to adjust recognition parameters, audio input and output options; Battery level and storage space display; The workflow of the user interface module (105) includes: After the system starts, the welcome interface is displayed, and the user can choose to start a new recognition task or open an existing file; After entering the recognition interface, the recognition results are displayed in real time, with an update interval of 300ms to ensure smooth display; Users use the touch screen to select, copy, and paste text, and long-press a word to pop up replacement suggestions; During the recognition process, the current audio processing progress and the remaining time estimate are displayed through a progress bar; After the recognition is completed, it provides options for saving, exporting, and sharing, and supports multiple formats.

9. The device for intelligent recording of the switching operation process of a substation as claimed in claim 1, characterized in that: The storage module (106) comprises: Internal storage uses a 64GB eMMC storage chip with a read speed of no less than 250MB / s and a write speed of no less than 150MB / s; Expandable storage, supports microSD card expansion storage up to 512GB; File system, using ext4 file system, supports log function; The storage management strategy of the storage module (106) is as follows: The original audio is stored in a lossless compressed format; The text results are stored in UTF-8 encoding and timestamp information is recorded; Automatically back up new data to the SD card every 30 minutes; When the remaining space is less than 10%, the user is automatically reminded and a quick cleanup option is provided; AES-256 encryption algorithm is used to encrypt and store sensitive data.

10. The device for intelligent recording of the switching operation process of a substation according to claim 1, characterized in that: The power management module (107) comprises: Battery, uses 3000mAh lithium polymer battery, supports fast charging; Power management chip, used for voltage regulation, battery charge and discharge management, and power supply control of each module; The power consumption monitoring unit is used to monitor the power consumption of each module in real time and adjust the working mode as needed; The power management strategy of the power management module (107) is as follows: Adjust the CPU and GPU operating voltage in real time according to system load; After 30 seconds of no operation, it will automatically enter low-power standby mode, and the wake-up time will not exceed 1 second; Dynamically switch unnecessary hardware modules based on usage scenarios; Automatically adjust recognition accuracy and response speed according to battery power level, giving priority to core functions when battery is low; Monitor device temperature and actively reduce system performance to avoid overheating when the temperature exceeds 45°C; The device for intelligent recording of the switching operation process of a substation has all modules integrated on a PCB with a size not exceeding 100mm x 60mm x 15mm and a multi-layer PCB design; a graphene heat sink and an internal copper foil heat dissipation channel are used to ensure that the core chip temperature does not exceed 60°C under normal working conditions; an embedded operating system based on Linux is adopted, POSIX threads are used to realize multi-task parallel processing, and each functional module communicates through a message queue and shared memory; a real-time scheduler is used to assign a higher priority to key tasks to ensure that the processing delay does not exceed 50ms; a memory pool technology is used to pre-allocate commonly used objects to reduce fragmentation and delay caused by dynamic memory allocation; the NEON SIMD instruction set is used to optimize the feature extraction and model inference process; the recognition parameters are automatically adjusted according to the environmental noise level and the user's speaking speed; a secure boot mechanism is implemented, a hardware trust root is used to verify the system integrity, all network communications are encrypted using the TLS 1.3 protocol, and access control and audit functions are provided when data is exported; I2C and SPI interfaces are reserved to support external connection of other sensors or modules; the software architecture adopts a modular design; the user's usage habits are recorded and the personalized recognition model is gradually optimized.

Images