Term speech recognition system and recognition method for coal mine

Abstract

The invention discloses a coal mine term speech recognition system and method, and belongs to the technical field of coal mine management, and the system comprises a data collection and processing module which is used for simulating an underground operation environment to carry out multi-scene recording collection; performing synthetic data enhancement in a mode of mixing real noise and synthetic speech, and constructing field customized data; according to the invention, through customized data acquisition, acoustic model optimization and language model adaptation, the recognition accuracy of the coal mine terminology is effectively improved; the system integrates an anti-noise algorithm, supports dialect recognition, can stably operate in complex noise and different dialect environments, and is high in environmental adaptability; the voice recognition processing module and the edge calculation module work cooperatively, so that rapid information transmission and abnormity monitoring are realized, and the working efficiency is improved; meanwhile, accurate instruction recognition and real-time abnormity monitoring are achieved, equipment misoperation is effectively avoided, safety accident risks are reduced, and coal mine production safety is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal mine management, and in particular relates to a coal mine terminology speech recognition system and a recognition method. Background Art

[0002] As the coal mining industry continues to advance informatization and automation, the efficiency and accuracy of equipment operation and command transmission are crucial. However, existing speech recognition technology faces numerous challenges when applied underground in coal mines. First, the underground environment is extremely complex, with noise from machinery and fans seriously interfering with speech signals. Second, coal mining has a wide variety of specialized terminology and numerous dialects, making it difficult for ordinary speech recognition systems to accurately identify it. Currently available speech recognition systems are not customized for the unique environment and specialized terminology of underground coal mines and may not meet the actual production needs of coal mines. Therefore, a speech recognition system that can accurately identify mining terminology is particularly important. Summary of the Invention

[0003] Problem to be solved

[0004] In response to the problems raised in the existing background technology, the present invention provides a coal mine terminology speech recognition system and recognition method.

[0005] Technical Solution

[0006] To solve the above problems, the present invention adopts the following technical solutions.

[0007] Coal mine terminology speech recognition system, including:

[0008] Data acquisition and processing module: used to simulate underground working environments to collect multi-scene recordings; and to enhance synthetic data by mixing real noise with synthetic speech to build domain-customized data;

[0009] Acoustic model module: pre-processes audio and integrates anti-noise algorithms; it also incorporates environmental robustness training into the model, including multi-condition training and adversarial training;

[0010] Language model module: Builds coal mine command grammar to constrain recognition results; has dynamic dictionary update function, can synchronize with the mining terminology database in real time, and automatically adjust the vocabulary based on user feedback or logs;

[0011] Speech recognition processing module: receives input speech, uses the data constructed by the data acquisition and processing module, the acoustic model adjusted by the acoustic model module, and the language model adapted by the language model module to perform speech recognition of coal mine professional terms, and outputs recognition results.

[0012] Preferably, in the data acquisition and processing module, when recording the simulated underground working environment, the simulated noise includes the roar of machines, the sound of fans, and the cutting sound of coal mining machines.

[0013] Furthermore, when the data collection and processing module performs audio recording, the collected content includes worker instructions and equipment operating terms.

[0014] Preferably, when the acoustic model module pre-processes the audio, the processing method adopts any one of spectral subtraction, Wiener filtering or deep learning noise reduction algorithm.

[0015] Furthermore, the speech recognition processing module includes:

[0016] Terminology parsing unit: uses natural language processing technology to identify mining terminology in input speech and calls a dedicated pronunciation dictionary to determine the correct pronunciation and semantics of the term;

[0017] Syntax matching unit: Based on the customized downhole command grammar, the unit performs grammatical matching on the recognized terms and compares the speech content with the grammatical rules. If the speech content conforms to the grammatical rules, it will be processed. If the speech content does not conform to the grammatical rules, it will output an error prompt or re-recognize.

[0018] Model recognition unit: inputs the speech data after term parsing and grammar matching into the adjusted acoustic model and adapted language model for recognition, and outputs the recognition results;

[0019] Result adjustment unit: adjusts the recognition results, including removing redundant information, correcting common errors, and adjusting the recognition content according to contextual semantics.

[0020] Furthermore, it also includes an edge computing module. In an underground environment with no network or a weak network, the speech recognition processing module runs on the edge computing module to realize fast text analysis at the edge and perform real-time analysis of abnormal keywords in the sensor log.

[0021] Furthermore, in the acoustic model module, when a deep learning noise reduction algorithm is used for audio preprocessing, the deep learning noise reduction algorithm adopts a noise reduction method based on a neural network, and the neural network includes a convolutional neural network, a recursive neural network and its variants such as a long short-term memory network, and a gated recurrent unit; the multi-condition training is to simulate a variety of different underground environmental conditions during the training process, including noise of different intensities, different tunnel echo effects, etc., so that the model learns speech features under various conditions; the adversarial training is to introduce adversarial samples, and enhance the model's resistance to noise and interference by generating an adversarial network or adversarial attack and defense mechanism.

[0022] A method for voice recognition of coal mine terminology, including a coal mine terminology voice recognition system, comprises the following steps:

[0023] Simulate the underground working environment, record worker instructions and equipment operating terms in different dialects, and collect real noise;

[0024] Mix the collected real noise with the synthesized speech to construct domain-customized data;

[0025] Use the acoustic model module to preprocess the collected audio;

[0026] Add multi-condition training and adversarial training to the acoustic model module;

[0027] Construct a coal mine command grammar, establish a dynamic dictionary update mechanism, synchronize the mining terminology database in real time, and automatically optimize the vocabulary through user feedback or logs;

[0028] After receiving the input speech and parsing the terminology and grammar matching, the system uses the adjusted acoustic model and adapted language model for recognition and outputs the recognition result.

[0029] Beneficial effects

[0030] Compared with the existing technology, the beneficial effects of the present invention are: through customized data collection, acoustic model optimization and language model adaptation, the recognition accuracy of coal mine professional terms is effectively improved; the system integrates anti-noise algorithms and supports dialect recognition, and can operate stably in complex noise and different dialect environments, with strong environmental adaptability; the speech recognition processing module and the edge computing module work together to achieve rapid information transmission and anomaly monitoring, thereby improving work efficiency; at the same time, accurate command recognition and real-time anomaly monitoring can effectively avoid equipment misoperation, reduce the risk of safety accidents, and ensure coal mine production safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or exemplary descriptions. Obviously, the drawings described below are only some embodiments of the present application and should not be regarded as limiting the scope. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0032] Figure 1 This is the identification flow chart of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0034] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0035] Example 1

[0036] like Figure 1 As shown, the coal mine terminology speech recognition system includes:

[0037] Data acquisition and processing module:

[0038] In a simulated underground working environment laboratory, a high-frequency recorder was used to record multiple scenes. The simulated noises included the roar of machinery, the hum of fans, and the sounds of shearers cutting coal. The recorded content included various worker commands, such as "start the ventilator" and "stop the shearer," as well as equipment operating terms such as "scraper conveyor" and "hydraulic support."

[0039] It should be noted that workers from different mining areas and speaking different dialects can also be invited to participate in the recording to ensure that the collected voice data covers multiple dialects.

[0040] Real noise collected from coal mines is mixed with synthesized speech generated by speech synthesis software in varying proportions to create domain-specific data. For example, fan noise is mixed with the synthesized speech of "starting a drill" in varying proportions, such as 20%, 30%, and 40%, to increase data diversity.

[0041] Acoustic model module:

[0042] Audio preprocessing:

[0043] The audio is preprocessed using a convolutional neural network-based noise reduction method from a deep learning noise reduction algorithm. The specific steps are as follows:

[0044] Data Preparation: The collected noisy audio data is divided into frames, with each frame length set to 25ms and a frame shift of 10ms. Each frame of audio is subjected to a Fast Fourier Transform (FFT) to convert it to the frequency domain and obtain the audio's spectral characteristics.

[0045] Model Construction: Build a multi-layer CNN model. The input layer receives the audio's spectral features. The convolution layer extracts local features using multiple convolution kernels. The pooling layer downsamples these features to reduce the data dimension. For example, three convolution layers are set up, each containing 32 convolution kernels of 3x3 size. The pooling layer uses max pooling with a pooling window size of 2x2. The fully connected layer integrates the extracted features, and the output layer outputs the denoised audio spectral features.

[0046] Model training: We used a large number of noisy audio samples and their corresponding clean audio samples as training data. We defined the mean squared error as the loss function and used the Adam optimizer for model training. The learning rate was set to 0.001, and the number of training epochs was 50.

[0047] Environment robustness training:

[0048] During training, we simulate a variety of underground environmental conditions. For example, we set noise levels of varying intensities, ranging from low (e.g., 50dB, similar to the noise level in underground ventilation tunnels) to high (e.g., 90dB, similar to the noisy environment near a coal mining machine). We also simulate different tunnel echo effects by adjusting the audio reflection coefficient and delay time. Audio data from these different conditions is fed into the acoustic model for training, enabling the model to learn speech characteristics under various conditions.

[0049] Adversarial examples are introduced to construct a generative adversarial network. The generator is responsible for generating adversarial examples that resemble real speech data, with specific interference patterns, such as random frequency interference and amplitude fluctuations. The discriminator is responsible for distinguishing between real speech samples and adversarial examples. During training, the acoustic model competes with the generator and discriminator. The generator continuously optimizes to produce more realistic adversarial examples, the discriminator continuously improves its distinguishing ability, and the acoustic model learns to identify and resist interference in adversarial examples, strengthening its resistance to noise and interference.

[0050] Language model module:

[0051] Construct a coal mine instruction grammar and use the open source natural language processing toolkit NLTK to parse and process the constructed grammar rules.

[0052] A dynamic dictionary update mechanism was established, using a Python script to establish a real-time connection with the mining terminology database. When new terms are added to the database or old terms are updated, the script automatically retrieves the updated information and synchronizes it with the speech recognition system's dictionary. Furthermore, the system regularly analyzes user feedback on recognition errors and frequently used terms in the system's operation logs. If a term is frequently misrecognized or a new high-frequency term appears, the system automatically optimizes the vocabulary by adding new terms and correcting the pronunciation and semantics of incorrect terms.

[0053] Speech recognition processing module:

[0054] Using natural language processing technology, the input speech is tagged with parts of speech and recognized by named entities to determine whether it is mining terminology. If it is recognized as a mining term, a dedicated pronunciation dictionary is used to determine its correct pronunciation and semantics. For example, for the term "anchor drill," the dictionary clearly defines its pronunciation and its meaning as coal mine support equipment.

[0055] According to the customized underground command grammar, the recognized terms are grammatically matched. The speech content is compared with the grammatical rules. If the speech content conforms to the grammatical rules, such as "Raise the hydraulic support 50cm", subsequent processing is carried out. If the speech content does not conform to the grammatical rules, such as "Raise the hydraulic support 50cm", an error message "Command grammar error, please re-enter" is output and re-recognition is required.

[0056] The speech data after term parsing and grammar matching is input into the adjusted acoustic model and adapted language model for recognition, and the recognition results are output.

[0057] Adjustments are made to the recognition results, including removing redundant information, correcting common errors, and adjusting the recognition content based on contextual semantics. For example, if the recognition result is "coal mining machine, coal mining machine started," the redundant information is removed and the output is "coal mining machine started." If the recognition result is "blasthole, purpose unknown," the result is corrected to "blasthole used for blasting" based on the context and pre-set rules related to blasting purposes, improving the accuracy and readability of the recognition results.

[0058] Edge computing module:

[0059] In underground environments with no or weak network connectivity, devices with edge computing capabilities, such as the NVIDIA Jetson Nano development board, are used. A trained speech recognition system is deployed on this device, and the speech recognition processing module runs locally. When sensors collect log data, the speech recognition system can quickly analyze abnormal keywords in the logs in real time. For example, if a sensor detects a voice message saying "abnormally high gas concentration," the speech recognition system quickly recognizes and interprets it at the edge, issuing a timely alarm signal and notifying relevant personnel to take action.

[0060] In summary, this application has the following advantages:

[0061] Improve recognition accuracy:

[0062] Data customization: The data acquisition and processing module simulates the underground working environment to collect multi-scene recordings, and mixes real noise with synthetic speech for data enhancement to construct domain-customized data. This data contains voice information from various complex environments in coal mines, enabling the acoustic model and language model to learn more accurate voice features and language patterns, thereby improving the recognition accuracy of coal mining professional terminology.

[0063] Acoustic model optimization: The acoustic model module uses a deep learning noise reduction algorithm for audio preprocessing and incorporates multi-condition training and adversarial training. The deep learning noise reduction algorithm can effectively remove noise interference in the underground environment. Multi-condition training simulates different underground environmental conditions. Adversarial training enhances the model's resistance to noise and interference.

[0064] Language Model Adaptation: The language model module constructs a coal mine command grammar, constrains recognition results, and features dynamic dictionary updates. The coal mine command grammar standardizes the expression of voice commands, ensuring that recognition results are more aligned with actual needs. The dynamic dictionary update mechanism synchronizes with the mining terminology database in real time and automatically adjusts the vocabulary based on user feedback and logs, ensuring that the language model can promptly adapt to new terminology and expressions, improving its recognition of specialized terminology.

[0065] Enhance environmental adaptability:

[0066] Improved noise resistance: The acoustic model module integrates an anti-noise algorithm. Through deep learning noise reduction algorithms and environmental robustness training, it effectively improves the system's resistance to underground noise. In noise environments of varying intensities, the system can accurately recognize voice signals, ensuring the reliability of voice recognition in complex underground operating environments.

[0067] Dialect recognition support: The data acquisition and processing module covers different dialects during recording, enabling the system to learn the phonetic characteristics of different dialects. The language model module uses a dynamic dictionary update mechanism to accurately identify and process mining terminology in different dialects. This enables the system to adapt to the dialect expression habits of workers in different mining areas, improving the system's versatility and adaptability.

[0068] Improve work efficiency:

[0069] Fast information transmission: The voice recognition processing module can receive and recognize workers' voice instructions in real time and convert them into text messages. This makes information transmission faster and more convenient, reducing the time cost of manual communication and recording.

[0070] Edge computing support: In underground environments with no network or weak network, the edge computing module can realize rapid text analysis at the edge, perform real-time analysis of abnormal keywords in sensor logs, and promptly detect and handle equipment failures and safety hazards, ensuring the continuity and safety of coal mine production.

[0071] Reduce security risks:

[0072] Accurate command execution: Due to the system's high recognition accuracy and environmental adaptability, it can accurately identify workers' voice commands, avoiding equipment misoperation and safety accidents caused by incorrect command transmission.

[0073] Real-time anomaly monitoring: The edge computing module analyzes sensor logs for abnormal keywords in real time, promptly identifying equipment failures and potential safety hazards. Once an anomaly is detected, the system quickly issues an alarm, notifying relevant personnel to take appropriate action, reducing the likelihood of safety incidents.

[0074] Example 2:

[0075] A method for voice recognition of coal mine terminology, comprising the following steps:

[0076] Data collection: Simulate the underground working environment, record worker instructions and equipment operating terms, cover different dialects, and collect real noise.

[0077] Data augmentation: Mix collected real noise with synthesized speech to build domain-customized data.

[0078] Audio preprocessing: The collected audio is preprocessed using the CNN-based deep learning noise reduction algorithm in the acoustic model module.

[0079] Environmental robustness training: Multi-condition training and adversarial training are added to the acoustic model module to simulate different underground environmental conditions and introduce adversarial examples to enhance the robustness of the model.

[0080] Language model construction: Build coal mine instruction grammar, establish a dynamic dictionary update mechanism, synchronize the mining terminology database in real time, and automatically optimize the vocabulary through user feedback or logs.

[0081] Speech recognition: Receives input speech, performs term parsing and grammar matching, and then uses the adjusted acoustic model and adapted language model for recognition, outputting the recognition results.

[0082] Result optimization: Adjust the recognition results to improve their accuracy and readability.

[0083] The above-described embodiments merely represent preferred embodiments of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous modifications, improvements, and substitutions without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A coal mine terminology speech recognition system, characterized in that: include: Data acquisition and processing module: used to simulate the underground operation environment and collect multi-scene recordings; And by mixing real noise and synthetic speech to enhance synthetic data, we can build domain-customized data; Acoustic model module: pre-processes audio and integrates anti-noise algorithms; it also incorporates environmental robustness training into the model, including multi-condition training and adversarial training; Language model module: Builds coal mine command grammar to constrain recognition results; has dynamic dictionary update function, can synchronize with the mining terminology database in real time, and automatically adjust the vocabulary based on user feedback or logs; Speech recognition processing module: receives input speech, uses the data constructed by the data acquisition and processing module, the acoustic model adjusted by the acoustic model module, and the language model adapted by the language model module to perform speech recognition of coal mine professional terms, and outputs recognition results.

2. The coal mine terminology speech recognition system according to claim 1, characterized in that: In the data acquisition and processing module, when recording the simulated underground working environment, the simulated noise includes the roar of machines, the sound of fans, and the cutting sound of coal mining machines.

3. The coal mine terminology speech recognition system according to claim 2, characterized in that: When the data collection and processing module collects audio recordings, the collected content includes worker instructions and equipment operating terms.

4. The coal mine terminology speech recognition system according to claim 1, characterized in that: When the acoustic model module preprocesses the audio, the processing method uses any one of spectral subtraction, Wiener filtering, or deep learning noise reduction algorithms.

5. The coal mine terminology speech recognition system according to claim 1, characterized in that: The speech recognition processing module includes: Terminology parsing unit: uses natural language processing technology to identify mining terminology in input speech and calls a dedicated pronunciation dictionary to determine the correct pronunciation and semantics of the term; Syntax matching unit: Based on the customized downhole command grammar, the unit performs grammatical matching on the recognized terms and compares the speech content with the grammatical rules. If the speech content conforms to the grammatical rules, it will be processed. If the speech content does not conform to the grammatical rules, it will output an error prompt or re-recognize. Model recognition unit: inputs the speech data after term parsing and grammar matching into the adjusted acoustic model and adapted language model for recognition, and outputs the recognition results; Result adjustment unit: adjusts the recognition results, including removing redundant information, correcting common errors, and adjusting the recognition content according to contextual semantics.

6. The coal mine terminology speech recognition system according to claim 5, characterized in that: It also includes an edge computing module. When there is no network or a weak network environment underground, the speech recognition processing module runs on the edge computing module to realize fast text analysis at the edge and perform real-time analysis of abnormal keywords in the sensor log.

7. The coal mine terminology speech recognition system according to claim 4, characterized in that: In the acoustic model module, when a deep learning noise reduction algorithm is used for audio preprocessing, the deep learning noise reduction algorithm adopts a noise reduction method based on a neural network, and the neural network includes a convolutional neural network, a recursive neural network and its variants such as a long short-term memory network and a gated recurrent unit; the multi-condition training is to simulate a variety of different underground environmental conditions during the training process, including noise of different intensities, different tunnel echo effects, etc., so that the model learns speech features under various conditions; the adversarial training is to introduce adversarial samples and enhance the model's resistance to noise and interference by generating an adversarial network or an adversarial attack and defense mechanism.

8. A method for voice recognition of coal mine terminology, comprising the coal mine terminology voice recognition system according to claim 6, characterized in that: Here are the steps: Simulate the underground working environment, record worker instructions and equipment operating terms in different dialects, and collect real noise; Mix the collected real noise with the synthesized speech to construct domain-customized data; Use the acoustic model module to preprocess the collected audio; Add multi-condition training and adversarial training to the acoustic model module; Construct a coal mine command grammar, establish a dynamic dictionary update mechanism, synchronize the mining terminology database in real time, and automatically optimize the vocabulary through user feedback or logs; After receiving the input speech and parsing the terminology and grammar matching, the system uses the adjusted acoustic model and adapted language model for recognition and outputs the recognition result.

Images