A method, device, medium, and product for voice recognition of train drivers.

Abstract

This application discloses a method, device, medium, and product for train driver speech recognition, relating to the fields of deep learning and speech recognition technology. The method includes acquiring the train driver's speech information; performing denoising, preprocessing, and fast Fourier transform on the speech information to obtain first spectral features; inputting the first spectral features into a proprietary vocabulary recognition model to obtain a proprietary vocabulary recognition result; inputting the first spectral features into a general vocabulary recognition model to obtain a general vocabulary recognition result; and fusing the proprietary and general vocabulary recognition results to obtain the train driver's speech recognition result. By fusing the recognition results of the proprietary and general vocabulary recognition models, the train driver's speech recognition result is obtained. This fusion process effectively compensates for the limitations of the general model in railway field recognition, thereby improving the accuracy of train driver speech recognition.

Description

Technical Field

[0001] This application relates to the fields of deep learning and speech recognition technology, and in particular to a method, device, medium and product for speech recognition of train drivers. Background Technology

[0002] Train drivers are a crucial link in the rail transit system, and their performance is closely related to train safety. Statistics on train accidents show that most are related to human factors. Currently, the monitoring and management of train drivers mainly focuses on driver fatigue and dangerous behaviors, lacking monitoring of skill-based errors when not fatigued. To prevent skill-based errors by train drivers, Chinese train drivers are required to perform call-and-response maneuvers outside of their driving duties to improve their focus. By employing voice recognition technology to identify the driver's voice during driving, it is possible to monitor skill-based errors when not fatigued, thereby improving train safety.

[0003] Speech recognition technology is responsible for converting spoken content into corresponding text information. With the development of machine learning and deep learning technologies, the accuracy and speed of speech recognition technology have been greatly improved. Currently, a large number of end-to-end speech recognition models have emerged in the domestic and international markets; however, these models are mainly applicable to general fields and have limitations in practical applications in the railway sector. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, medium and product for train driver voice recognition, which can effectively make up for the limitations of general models in recognition in the railway field.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a train driver voice recognition method, including:

[0007] Obtain the train driver's voice information;

[0008] The speech information is denoised and preprocessed to obtain the first speech information;

[0009] Perform a Fast Fourier Transform on the first speech information to obtain the first spectral features;

[0010] The first spectral feature is input into the proprietary vocabulary recognition model to obtain a proprietary vocabulary recognition result; the proprietary vocabulary recognition model includes a proprietary syllable recognition model and a syllable conversion model; the proprietary syllable recognition model is used to perform syllable recognition processing on the first spectral feature to obtain a proprietary syllable recognition result; the syllable conversion model is used to perform syllable conversion processing on the proprietary syllable recognition result to obtain the proprietary vocabulary recognition result;

[0011] The first spectral feature is input into the general vocabulary recognition model to obtain the general vocabulary recognition result; the proprietary vocabulary recognition result and the general vocabulary recognition result are fused to obtain the train driver's speech recognition result.

[0012] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the train driver voice recognition method described above.

[0013] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the train driver voice recognition method described above.

[0014] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the train driver voice recognition method described above.

[0015] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0016] This application provides a method, device, medium, and product for train driver speech recognition. The train driver speech recognition method includes: acquiring the train driver's speech information; performing noise reduction and preprocessing on the speech information to obtain first speech information; performing a Fast Fourier Transform on the first speech information to obtain first spectral features; inputting the first spectral features into a proprietary vocabulary recognition model to obtain a proprietary vocabulary recognition result; the proprietary vocabulary recognition model includes a proprietary syllable recognition model and a syllable conversion model; the proprietary syllable recognition model is used to perform syllable recognition processing on the first spectral features to obtain a proprietary syllable recognition result; the syllable conversion model is used to perform syllable conversion processing on the proprietary syllable recognition result to obtain a proprietary vocabulary recognition result; inputting the first spectral features into a general vocabulary recognition model to obtain a general vocabulary recognition result; and fusing the proprietary vocabulary recognition result and the general vocabulary recognition result to obtain the train driver's speech recognition result.

[0017] This application obtains the speech recognition results of train drivers by fusing the recognition results of proprietary vocabulary recognition models and general vocabulary recognition models. This fusion process effectively makes up for the limitations of general models in railway recognition, thereby improving the accuracy of train driver speech recognition. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a diagram illustrating the application environment of a train driver voice recognition method according to an embodiment of this application.

[0020] Figure 2 A flowchart illustrating a train driver voice recognition method according to an embodiment of this application;

[0021] Figure 3 This is a detailed flowchart illustrating the preprocessing and fast Fourier transform steps in one embodiment of this application.

[0022] Figure 4 This is a schematic diagram of the structure of a proprietary vocabulary recognition model in one embodiment of this application;

[0023] Figure 5 This is a schematic diagram illustrating the basic principle of soft voting in one embodiment of this application;

[0024] Figure 6 A schematic diagram of a train driver voice recognition system provided in one embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the user interaction layer in one embodiment of this application; Figure 7 (a) is the initialization interface; Figure 7 (b) Speech recognition task interface;

[0026] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Because train drivers' speech information contains not only general vocabulary but also a large number of railway-specific terms, existing speech recognition models have low accuracy when dealing with railway-specific terms, and are usually trained on English data, resulting in poor support for Chinese information. Therefore, this application provides a train driver speech recognition method.

[0030] The train driver voice recognition method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send the train driver's voice information to server 104. After receiving the train driver's voice information, server 104 performs denoising, preprocessing, and Fast Fourier Transform on the voice information to obtain a first spectral feature; inputs the first spectral feature into a proprietary vocabulary recognition model to obtain a proprietary vocabulary recognition result; inputs the first spectral feature into a general vocabulary recognition model to obtain a general vocabulary recognition result; and fuses the proprietary and general vocabulary recognition results to obtain the train driver's voice recognition result. Server 104 can then feed back the obtained train driver's voice recognition result to terminal 102. In addition, in some embodiments, the train driver voice recognition method can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly perform voice recognition processing on the train driver's voice information, or the server 104 can obtain the train driver's voice information from the data storage system and perform voice recognition processing on the train driver's voice information.

[0031] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0032] In one exemplary embodiment, such as Figure 2 As shown, a voice recognition method for train drivers is provided. This method is executed by a computer device, specifically a terminal or server, or both. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 206. Wherein:

[0033] Step 201: Obtain the train driver's voice information.

[0034] Step 202: Perform noise reduction and preprocessing on the speech information to obtain the first speech information.

[0035] The denoising process involves setting the denoising parameters in the Wiener filter according to the acquired current train operating status, and then inputting the voice information into the adjusted Wiener filter to obtain denoised voice information. The operating status includes four states: default, constant speed, deceleration, and stop. The preprocessing involves sequentially performing pre-emphasis, framing, and windowing operations on the denoised voice information to obtain first voice information, such as... Figure 3 As shown.

[0036] Step 203: Perform a Fast Fourier Transform on the first speech information to transform it from the time domain to the frequency domain, obtaining the first spectral features, including a spectrogram and a log-mel spectrogram (the result of taking the logarithm of the Mel spectrogram), such as... Figure 3 As shown.

[0037] Step 204: Input the first spectral feature into the proprietary vocabulary recognition model to obtain the proprietary vocabulary recognition result; the proprietary vocabulary recognition model includes a proprietary syllable recognition model and a syllable conversion model.

[0038] Furthermore, such as Figure 4 As shown, the step of inputting the first spectral feature into the proprietary vocabulary recognition model to obtain the proprietary vocabulary recognition result specifically includes:

[0039] Step 2041: Perform syllable recognition processing on the first spectral features using a proprietary syllable recognition model to obtain proprietary syllable recognition results; wherein, the determination process of the proprietary syllable recognition model specifically includes:

[0040] a. Construct a proprietary syllable training dataset; each sample in the proprietary syllable training dataset includes a second spectral feature and a corresponding syllable label; the second spectral feature is obtained by preprocessing and performing a fast Fourier transform on the standard call phrases of train drivers; the syllable label is obtained by annotating the second spectral feature.

[0041] b. Constructing a proprietary syllable recognition network; the proprietary syllable recognition network is a network structure based on the CNN-GRU-CTC framework, which includes several convolutional layers, gated recurrent units (GRU), fully connected layers, and a connection-temporal classifier (CTC) connected in sequence; firstly, several one-dimensional convolutional layers are used to extract the temporal features and temporal correlations in the second spectral features. For example, 12 one-dimensional convolutional layers are set, each with a kernel size of 3 and an activation function of ReLU. Considering the characteristics of train drivers' speech information—fast speech rate and short speech length—this application uses dilated convolutions in the last two convolutional layers to expand the network's receptive field; then, it can... A gated recurrent unit (GRU) with 128 hidden units is used as a temporal module to further extract deep temporal features of speech information. Finally, in the speech recognition task of train drivers, due to the inconsistency between the second spectral feature and the corresponding syllable label length, the deep temporal features output by the gated recurrent unit cannot be aligned with the manually labeled syllable labels. Therefore, this application adopts a connection-time classifier (CTC) as a forced alignment module. When the input sequence and output sequence of the network cannot form a simple one-to-one alignment relationship, the deep temporal features of train driver speech information and syllable labels are forcibly aligned by extending the label length and traversing all possible labels.

[0042] c. Input each sample of the proprietary syllable training dataset into the proprietary syllable recognition network, update the parameters of the proprietary syllable recognition network, and obtain the proprietary syllable recognition model. Moreover, in the application stage of the proprietary syllable recognition model, for a given first spectral feature, after feature extraction by CNN and GRU, CTC decoding, softmax mapping, and decoding search, the first spectral feature is mapped to the output probability of the proprietary syllable recognition result.

[0043] Step 2042: The syllable conversion model is used to perform syllable conversion processing on the proprietary syllable recognition result to obtain the proprietary vocabulary recognition result.

[0044] Furthermore, the syllable conversion model includes a pronunciation dictionary and an n-gram language model; performing syllable conversion processing on the proprietary syllable recognition result to obtain the proprietary vocabulary recognition result specifically includes: using the pronunciation dictionary to convert the proprietary syllable recognition result into all corresponding text sequences; using the n-gram language model to calculate the probability values of all text sequences to obtain the proprietary vocabulary recognition result; assuming the text sequence S = {w1, w2,..., w i}, according to the conditional probability, the occurrence probability of the text sequence S can be expressed as:

[0045] P(S) = P(w1, w2,..., w n ) = P(w1) * P(w2|w1) *... * P(w i |w1, w2,..., w i-1 );

[0046] where, w i is the i-th word in the text sequence, and the conditional probability P(w i |w i-n+1 ,..., w i-1 ) in the n-gram language model is usually estimated by counting the occurrence frequency of a certain word in a large-scale corpus. Specifically:

[0047]

[0048] where, count represents the number of times a certain word appears in the train driving corpus.

[0049] For example, assuming that the obtained proprietary syllable recognition result is "zhuyi xian su liu shi gong li", by referring to the pronunciation dictionary, a possible corresponding text sequence "注意限速60公里" can be found from the corpus. The following is the process of calculating the output probability of this text sequence using the 3-gram model:

[0050] P(注意限速60公里) = P(注意) * P(限速|注意) * P(60|注意限速) * P(公里|注意限速60);

[0051] where, P(注意) is the conditional probability of the word "注意", P(限速|注意) is the conditional probability of the word "限速", P(60|注意限速) is the conditional probability of the word "60" appearing, and P(公里|注意限速60) is the conditional probability of the word "公里". The specific calculation methods of each conditional probability are as follows:

[0052]

[0053] Step 205: Input the first spectral feature into the general vocabulary recognition model to obtain the general vocabulary recognition result.

[0054] The process of determining the general vocabulary recognition model specifically includes:

[0055] a. Constructing a general vocabulary recognition network; the general vocabulary recognition network is built based on the Whisper framework. Specifically, the Whisper network is a speech recognition network open-sourced by OpenAI in 2022, using Transformer as its basic architecture; this network takes a log-Mel spectrum with a time step of 3000 and 80 channels as input. The main structure of the network consists of an input layer composed of two one-dimensional convolutional networks and an encoder-decoder layer based on Transformer; the two convolutional layers of the input layer are similar to a pyramid structure, and their main function is to reduce the dimensionality of the data, reduce the temporal resolution, and extract local information to facilitate subsequent feature extraction; the multi-layer stacked multi-head self-attention mechanism and multi-layer perceptron in the encoder are used to capture the correlation between various positions in the input sequence without being limited by the sequence length. The deep features are concatenated in the last encoding layer and used as the query and value input of the multi-head self-attention mechanism of the decoder; in addition, the decoder also receives information from the encoder to transform and generate the target sequence, and finally outputs a coherent text sequence.

[0056] b. Construct a general vocabulary training dataset; each sample in the general vocabulary training dataset includes a third spectral feature and a corresponding text label; the third spectral feature is obtained by preprocessing and performing a fast Fourier transform on Chinese speech data; the text label is obtained by annotating the third spectral feature.

[0057] c. Based on transfer learning technology, the samples in the general vocabulary training dataset are input into the general vocabulary recognition network, and the parameters of the general vocabulary recognition network are updated so that the network parameters are shifted towards the Chinese speech recognition task, thus obtaining the general vocabulary recognition model.

[0058] Furthermore, the general vocabulary recognition network includes a preset layer (e.g., a fully connected layer), and the network parameters of the Whisper model are fine-tuned using the LoRa algorithm (Low-Rank Adaptation) in a low-resource environment to obtain the general vocabulary recognition model. Specifically:

[0059] The preset layer is initialized; the initialization is used to represent the original pre-trained weight matrix W0 of the preset layer as the product of two trainable low-rank matrices A and B, where the dimension of A is d×r and the dimension of B is r×d (r<<d, where d is the dimension of the original pre-trained weight matrix).

[0060] Perform the following iterative operation on the samples in the general vocabulary training dataset:

[0061] The current sample of the general vocabulary training dataset is input into the general vocabulary recognition network corresponding to the current iteration number to obtain the initial general vocabulary recognition result corresponding to the current sample; the current sample of the general vocabulary training dataset is a sample that has not been input into the general vocabulary recognition network corresponding to the current iteration number; the general vocabulary recognition network corresponding to the current iteration number is the general vocabulary recognition network initialized by a preset layer or the general vocabulary recognition network updated after the previous iteration number.

[0062] Calculate the loss value between the initial general vocabulary recognition result and the corresponding text label for the current sample. Update the parameters of the trainable low-rank matrices A and B through backpropagation to obtain the updated trainable low-rank matrices A and B. Combine the updated trainable low-rank matrices A and B with the original pre-trained weight matrix W0 to obtain the updated weight matrix W for the current iteration, as follows:

[0063] W = W0 + BA.

[0064] Determine whether the loss value meets the stopping condition. If it does, determine the general vocabulary recognition model based on the weight matrix W updated after the current iteration number. Otherwise, update the general vocabulary recognition network corresponding to the current iteration number based on the weight matrix W updated after the current iteration number, and return "Input the current sample of the general vocabulary training dataset into the general vocabulary recognition network corresponding to the current iteration number to obtain the initial general vocabulary recognition result corresponding to the current sample".

[0065] Since Whisper is a network trained on multilingual data, and the general vocabulary recognition task in the train driver speech information recognition scenario mainly uses Chinese vocabulary, fine-tuning of the existing Whisper network is necessary. The most basic fine-tuning method is fine-tuning, which directly trains the network using the target dataset without changing the network structure. Fine-tuning requires freezing the parameter values ​​of the original model and adding new layers to learn new features. The parameters of the unfrozen network layers and the newly added network layers are then trained to achieve parameter transfer. However, traditional fine-tuning methods suffer from severe computational resource waste. Therefore, this application uses the LoRa algorithm to fine-tune the model, training only the introduced low-rank matrices A and B without updating the original pre-trained weight matrix W0. Since the number of parameters in the low-rank matrices A and B is much smaller than that in the original pre-trained weight matrix W0, the computational cost and storage requirements for fine-tuning are significantly reduced. The fine-tuning results are shown in Table 1 below.

[0066] Table 1. Fine-tuning effect table

[0067] Fine-tuning algorithm Word Error Rate (WER) Training parameter proportion Training time Unadjusted 40.9％ - - fine-tuning 7.51％ 100％ 26 hours LoRa 10.8％ 2.89％ 18 hours

[0068] Step 206 involves fusing the proprietary vocabulary recognition results and the general vocabulary recognition results to obtain the train driver's speech recognition results, specifically including:

[0069] Step 2061: Perform hot word weighting on the proprietary vocabulary recognition results to obtain weighted proprietary vocabulary recognition results.

[0070] Step 2062: Perform hot word weighting on the general vocabulary recognition results to obtain the weighted general vocabulary recognition results.

[0071] Step 2063: Using a soft voting method, the weighted proprietary vocabulary recognition results and the weighted general vocabulary recognition results are fused and calculated to obtain the predicted speech recognition results.

[0072] Step 2064: Based on the predicted speech recognition result, obtain the speech recognition result of the train driver.

[0073] It should be noted that, as Figure 5 As shown, the proprietary vocabulary recognition model and the general vocabulary recognition model are regarded as two pre-trained classifiers h1 and h2. Each classifier makes a judgment for the classification task Q. The judgment results of each classifier are combined to make a decision and output the final judgment result. Assume that the class labels of the classification task Q are {q1,q2,...,q}. j If the input sample is x, then the output results of the two classifiers above can be expressed as:

[0074]

[0075] in, This indicates that for input sample x, classifier h i Get class label q j The predicted output probability.

[0076] In the soft voting process, the weighted average of the output probabilities of the two classifiers is used as the final output probability value for each class label. Finally, the class label with the highest probability is selected as the final output result, as shown below:

[0077]

[0078] Among them, h j (x) represents the class label q j The weighted average probability is given by h(x), which represents the set of final output probability values ​​for all class labels. The class label with the highest probability value is selected as the final output result y.

[0079] Furthermore, considering that train drivers primarily use specialized vocabulary and secondarily use general vocabulary when making calls, this means that the voting decision should mostly shift towards the specialized vocabulary recognition model. Therefore, a hot word weighting module was added after probability calculation and before soft voting decision. This module uses the weight value of hot words as the metric for voting decision, and its core function is to assign higher parameter weights to specific words in specific scenarios.

[0080] This application assigns weights to hot words based on the frequency of occurrence of each proper noun in the train driver call phrase text database. The words in the text database are divided into four categories according to word frequency: high-frequency words, mid-to-high-frequency words, mid-frequency words, and low-frequency words. The division method and weights are shown in Table 2 below:

[0081] Table 2. Division Methods and Weights

[0082] proper nouns word frequency Weight High-frequency words >70 4 High-frequency words 40～70 3 Mid-frequency words 10～40 2 low frequency words <10 1

[0083] After obtaining the hot word weights λ∈{1,2,3,4} for proprietary vocabulary, the soft voting method can be adjusted to amplify the output probability of specific hot words in the proprietary vocabulary recognition model, i.e.:

[0084]

[0085] in, This indicates that after hot word weighting, the proprietary vocabulary recognition model h1 targets the class label q. j The output probability, λ j Indicates belonging to the class label q j Keyword weighting. j′ (x) represents the class label q j The weighted average probability after hot word weighting, h′(x) represents the set of final output probability values ​​of all class labels after hot word weighting, and the class label with the largest probability value is selected as the final output result y.

[0086] For example, suppose that for a certain train driver speech recognition task Q, the class labels include {"main segment", "segment by segment"}. The recognition results of the proprietary vocabulary recognition model h1 and the general vocabulary recognition model h2 are shown below:

[0087]

[0088] Following the traditional soft voting method, the prediction results after model fusion are as follows:

[0089]

[0090] Obviously, the "segment-by-segment" result, which has a higher probability of being output, is chosen as the final prediction result.

[0091] However, considering that train drivers use more specialized vocabulary in their driving scenarios, and to shift the model integration towards a specialized vocabulary recognition model while reducing interference from general vocabulary, this application uses hot word weighting to assign higher parameter weights to specific words in specific scenarios. In the aforementioned recognition task, "main segment" is a mid-to-high frequency specialized word in the corpus, with a weight of 3; while "segment by segment" is a general word, with a weight of 1. After hot word weighting, the output of the specialized vocabulary recognition model h1 is:

[0092]

[0093] Therefore, the output probability of the model after hot word weighting is:

[0094]

[0095] Clearly, after hot word weighting, the recognition effect of proper vocabulary is enhanced, and the final output of the fused model is "main segment".

[0096] This application also provides an application scenario in which the above-described train driver speech recognition method is applied. Specifically, the train driver speech recognition method provided in this embodiment can be applied in a train driver speech recognition scenario. The train driver speech recognition scenario includes a train driver speech recognition stage and a speech recognition result display stage; the train driver speech recognition method provided in this embodiment belongs to the train driver speech recognition stage.

[0097] Based on the same inventive concept, this application also provides a train driver voice recognition system for implementing the train driver voice recognition method described above. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations of one or more train driver voice recognition system embodiments provided below can be found in the limitations of the train driver voice recognition method described above, and will not be repeated here.

[0098] In one exemplary embodiment, such as Figure 6 As shown, a train driver voice recognition system is provided, including:

[0099] User interaction layer; the train driver inputs raw voice information into the system through the user interaction layer, supporting both microphone and local WAV file voice input modes to ensure that the train can perform both on-the-go and offline voice recognition. The user interaction layer interface in this application is as follows: Figure 7 As shown, it includes an initialization interface and a speech recognition task interface.

[0100] Business logic layer; the business logic layer passes voice information to the voice denoising module.

[0101] The basic service layer consists of a speech denoising module that denoises the speech signal, and a speech recognition module that receives the output from the speech denoising module and recognizes the denoised speech information. After preprocessing and feature extraction, the denoised speech information is input into a dual-model integrated proprietary vocabulary recognition model and a general vocabulary recognition model to perform proprietary vocabulary recognition tasks and general vocabulary recognition tasks, respectively. After recognition and voting, the speech recognition result of the train driver is output, displayed on the user interaction layer interface, and the output result after speech recognition is recorded and saved as a recognition log.

[0102] This application utilizes Python on the PyCharm platform to implement the overall system design and development. It is mainly divided into graphical interface design and business module design. The graphical interface design of the speech recognition system is primarily based on the PyQt framework. The business modules can be divided into three modules: speech denoising, proprietary syllable recognition, and general syllable recognition. The training and development of the proprietary syllable recognition network based on CNN-GRU-CTC is mainly implemented using the Keras deep learning framework, while the training and development of the general syllable recognition network based on transfer learning is mainly implemented using the PyTorch and Transformer machine learning libraries. Speech signal preprocessing and denoising are mainly implemented using the librosa library. The deep learning training system is Ubuntu 22.04.2LTS, and the processor is [processor details missing]. Xeon(R)w5-3423 CPU @ 2.10GHz x 12, with an NVIDIA RTX 6000 Ada Generation graphics card with 48GB of VRAM used for training. The graphical user interface development system is Windows 10, and the processor is... Core(TM)i7-6700K CPU@4GHz×4.

[0103] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database is used for train driver voice processing data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a train driver voice recognition method.

[0104] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0105] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0106] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0107] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0109] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for recognizing train driver speech, characterized in that, The train driver voice recognition method includes: Obtain the train driver's voice information; The speech information is denoised and preprocessed to obtain the first speech information; Perform a Fast Fourier Transform on the first speech information to obtain the first spectral features; The first spectral feature is input into the proprietary vocabulary recognition model to obtain a proprietary vocabulary recognition result; the proprietary vocabulary recognition model includes a proprietary syllable recognition model and a syllable conversion model; the proprietary syllable recognition model is used to perform syllable recognition processing on the first spectral feature to obtain a proprietary syllable recognition result; the syllable conversion model is used to perform syllable conversion processing on the proprietary syllable recognition result to obtain the proprietary vocabulary recognition result; The first spectral feature is input into the general vocabulary recognition model to obtain the general vocabulary recognition result; The proprietary vocabulary recognition results and the general vocabulary recognition results are fused together to obtain the train driver's speech recognition results; The process of determining the proprietary syllable recognition model specifically includes: A proprietary syllable training dataset is constructed; each sample in the proprietary syllable training dataset includes a second spectral feature and a corresponding syllable label; the second spectral feature is obtained by preprocessing and performing a fast Fourier transform on the standard call phrases of train drivers; the syllable label is obtained by annotating the second spectral feature; A proprietary syllable recognition network is constructed; the proprietary syllable recognition network includes several convolutional layers, gated recurrent units, fully connected layers and a connection-temporal classifier connected in sequence; wherein, the last two convolutional layers in the several convolutional layers are dilated convolutional layers; Each sample from the proprietary syllable training dataset is input into the proprietary syllable recognition network, and the parameters of the proprietary syllable recognition network are updated to obtain the proprietary syllable recognition model.

2. The train driver voice recognition method according to claim 1, characterized in that, The process of determining the general vocabulary recognition model specifically includes: A general vocabulary recognition network is constructed; the general vocabulary recognition network is built based on the Whisper framework. A general vocabulary training dataset is constructed; each sample in the general vocabulary training dataset includes a third spectral feature and a corresponding text label; the third spectral feature is obtained by preprocessing and performing a fast Fourier transform on Chinese speech data; the text label is obtained by annotating the third spectral feature; The samples in the general vocabulary training dataset are input into the general vocabulary recognition network, the parameters of the general vocabulary recognition network are updated, and the general vocabulary recognition model is obtained.

3. The train driver voice recognition method according to claim 1, characterized in that, The general vocabulary recognition network includes a preset layer; the step of inputting samples from the general vocabulary training dataset into the general vocabulary recognition network, updating the parameters of the general vocabulary recognition network, and obtaining the general vocabulary recognition model specifically includes: The preset layer is initialized; the initialization is used to represent the original pre-trained weight matrix W0 of the preset layer as the product of two trainable low-rank matrices A and B; Perform the following iterative operation on the samples in the general vocabulary training dataset: The current sample of the general vocabulary training dataset is input into the general vocabulary recognition network corresponding to the current iteration number to obtain the initial general vocabulary recognition result corresponding to the current sample; the current sample of the general vocabulary training dataset is the sample that has not been input into the general vocabulary recognition network corresponding to the current iteration number; the general vocabulary recognition network corresponding to the current iteration number is the general vocabulary recognition network initialized by a preset layer or the general vocabulary recognition network updated after the previous iteration number. Calculate the loss value between the initial general vocabulary recognition result and the corresponding text label for the current sample, update the parameters of the trainable low-rank matrices A and B through backpropagation to obtain the updated trainable low-rank matrices A and B, and combine the updated trainable low-rank matrices A and B with the original pre-trained weight matrix W0 to obtain the updated weight matrix W for the current iteration number. Determine whether the loss value meets the stopping condition. If it does, determine the general vocabulary recognition model based on the weight matrix W updated after the current iteration number. Otherwise, update the general vocabulary recognition network corresponding to the current iteration number based on the weight matrix W updated after the current iteration number, increment the current iteration number by 1, and return "Input the current sample of the general vocabulary training dataset into the general vocabulary recognition network corresponding to the current iteration number to obtain the initial general vocabulary recognition result corresponding to the current sample".

4. The train driver voice recognition method according to claim 1, characterized in that, The syllable conversion model includes a pronunciation dictionary and an n-gram language model. The syllable conversion model performs syllable conversion processing on the proprietary syllable recognition results to obtain the proprietary vocabulary recognition results, specifically including: Using the pronunciation dictionary, the specific syllable recognition results are converted into all corresponding text sequences; The probability values ​​of all text sequences are calculated using an n-gram language model to obtain the proprietary vocabulary recognition results.

5. The train driver voice recognition method according to claim 1, characterized in that, The process of fusing the specialized vocabulary recognition results and the general vocabulary recognition results to obtain the train driver's speech recognition results specifically includes: The proprietary vocabulary recognition results are processed by hot word weighting to obtain weighted proprietary vocabulary recognition results; The general vocabulary recognition results are processed by hot word weighting to obtain the weighted general vocabulary recognition results. A soft voting method is used to fuse the weighted proprietary vocabulary recognition results and the weighted general vocabulary recognition results to obtain the predicted speech recognition results. Based on the predicted speech recognition results, the speech recognition results of the train driver are obtained.

6. The train driver voice recognition method according to claim 1, characterized in that, The step of performing noise reduction and preprocessing on the speech information to obtain the first speech information specifically includes: Based on the current train operating status, the range of denoising parameters in the Wiener filter is set accordingly, and the voice information is input into the Wiener filter after the settings are adjusted to obtain the denoised voice information; the operating status includes four states: default, constant speed, deceleration, or stop. The denoised speech information is preprocessed to obtain the first speech information; the preprocessing includes pre-emphasis, framing, and windowing.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the train driver voice recognition method according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the train driver voice recognition method according to any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the train driver voice recognition method according to any one of claims 1-6.

Images