Intelligent voice recognition method and system for power dispatching scene

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a power dispatching scene intelligent voice recognition method and system, and the method comprises the steps: obtaining a Chinese mandarin public data set as a basic audio data set; constructing a speech recognition model based on deep learning; inputting the basic audio data set into a speech recognition model; acquiring to-be-tested power dispatching audio data as a real audio data set; converting the trained power dispatching voice recognition model into a deployment model and deploying the deployment model into a computer of an electric power bureau; the invention further discloses a system applied to the intelligent voice recognition method for the power dispatching scene. According to the method, the defects caused by a traditional manual recording mode are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and in particular to an intelligent speech recognition method and system for power dispatching scenarios. Background Art

[0002] In the current power dispatching scenario, the recording of various work contents mainly relies on traditional manual methods; during busy work, dispatchers need to manually and quickly record key data such as incoming call information, call content, and fault reports.

[0003] However, the problem with using traditional manual recording methods in power dispatching scenarios is that as time goes by and the workload increases, manual recording methods will inevitably lead to deviations in the recorded content, which may in turn cause certain safety hazards; in addition, traditional manual recording methods are not conducive to accurate and convenient query and tracing of work logs in the future. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an intelligent speech recognition method and system for power dispatching scenarios, which solves the drawbacks brought about by traditional manual recording methods.

[0005] In the first aspect, the present invention discloses an intelligent speech recognition method for power dispatching scenarios, the steps of which are as follows: S1, obtain a public Mandarin Chinese dataset as a basic audio dataset, and determine a training and test dataset based on the audio sample data; S2, build a speech recognition model based on deep learning, which includes feature extraction layer, convolution layer, linear layer, activation layer, core module layer, and output layer. The core module layer includes a feedforward network module, a multi-head attention mechanism module, a convolution module, a residual connection, and a layer standard normalization module; S3, inputting the basic audio data set into the speech recognition model, jointly training the speech recognition model based on the cross entropy loss function and the output layer CTC loss function to obtain a basic speech recognition model; S4, obtaining the power dispatching audio data to be tested as a real audio data set, using the basic speech recognition model to recognize the real audio data set to be tested, updating the parameters of the basic speech recognition model according to the result, and iteratively training to obtain the power dispatching speech recognition model; S5, convert the trained power dispatch speech recognition model into a deployment model, deploy it to the power bureau computer, build the front-end and back-end framework and combine it with the telephone recorder to form the final system. After the dispatcher answers the call, the back-end module of the system converts the working audio into text in real time, and then transmits it to the front-end page for the dispatcher to view and save locally.

[0006] The specific steps of S1 are: Step 1: Obtain the Mandarin Chinese public datasets AISHELL-1 and Thchs30, which consist of cleaned and annotated audio data labels and the text labels corresponding to the audio, as the basic audio datasets; Step 2: extract audio sample data of the power dispatching scenario, split the long audio samples into multiple short audio samples, and annotate them to obtain audio files and text labels corresponding to the audio files as the real audio dataset; Step 3: Construct training and test datasets based on the basic audio dataset, the real audio dataset, and the corresponding text labels. The ratio of the training dataset to the test dataset is divided according to the deep learning training and test data standards.

[0007] In step 4, the basic audio dataset is augmented to obtain an augmented basic audio dataset. The audio file labels in the augmented basic audio dataset are adjusted according to the augmentation processing method and added to the training and test datasets.

[0008] Specifically, the convolution kernel size of the convolution layer in the speech recognition model is set to 15; the output feature dimension of the linear layer is 256; the Conformer network modules in the core module layer contain 12, and the output dimension of each module is 256; the dimension of the feedforward network module is set to 2048; and the number of attention heads of the multi-head attention mechanism module is set to 4.

[0009] The specific steps of S3 are: Step 1: Perform preprocessing such as segmentation and labeling on the original audio data to generate the data format required for model training; Step 2: Input the preprocessed audio sample data into the audio feature extraction layer for feature extraction; Step 3: Extract features from the audio data through the audio feature extraction layer to obtain feature information of the audio data, input it into the linear layer for integration, and then transmit it to the activation layer; Step 4: Activate the audio feature information through the activation layer to obtain the audio feature tensor required for training, and transmit the audio tensor to the acoustic model encoder; Step 5: The acoustic model encoder performs a series of operations including feedforward layer processing, multi-head attention layer processing, layer normalization processing, etc. on the input audio feature tensor, so that the input feature vector is concatenated with the processed feature vector to obtain a new audio feature tensor, which is transmitted to the output layer. In the sixth step, supervised training is performed jointly based on the cross entropy loss function and the CTC loss function. The loss value is obtained according to the training results. The model parameters are continuously updated until the loss value converges. The training is stopped to obtain a trained basic speech recognition model.

[0010] The specific steps of S4 are: Step 1: Obtain the real audio data set to be tested and the trained basic speech recognition model; Step 2: Select the audio sample data to be tested from the test set and input it into the basic speech recognition model; Step 3: The model extracts the audio sample feature sequence and inputs it into the audio encoder. After being processed by various modules of the encoder, the audio feature tensor information of the test sample is obtained. Step 4: Input the audio feature tensor information and the text result output at the previous moment into the decoder for joint decoding. The encoder goes through a series of processes including the feedforward network module, the multi-head attention mechanism module, the convolution module, the layer normalization module, and the output layer to obtain the predicted output probability. Step 5: The final text sequence result is obtained by integrating the predicted probabilities of the output layer and the linear layer. The steps are repeated until the end marker appears to obtain the complete recognition result of the test set. In step 6, based on the pre-labeled test set text labels, the word error rate of the test set audio decoding results is tested to verify whether the model is effective, and the power dispatching speech recognition model is obtained by continuous optimization and fine-tuning.

[0011] The specific steps of S5 are: Step 1: Convert the trained power dispatching speech recognition model into a deployment model and prepare the hardware equipment such as servers, telephone recorders, monitors, and other equipment external connection cables and power supplies required for deployment; Step 2: Generate the deployment model into the structure format required for deployment through the deep learning framework to meet the basic requirements for deployment in the server Linux environment, and build a telephone recorder used in the work by combining the front-end and back-end frameworks; Step 3: Deploy the deployment model, backend framework, and frontend framework to the server system, connect the telephone recorder, and form an intelligent voice recognition deployment system for power dispatching scenarios; Step 4: Start the hardware and software required for the system for testing, load the working audio data in the telephone recorder through the entire deployment system, call the deployment model and the back-end framework to transcribe the working audio data in real time, and transmit the results to the front-end framework for display; In step 5, the front-end framework of the deployed system displays the text results transmitted from the back-end framework in real time on the dispatcher's work computer page for the dispatcher to view and modify, and finally save to the local computer.

[0012] In a second aspect, the present invention discloses an intelligent speech recognition system for power dispatching scenarios, which is applied to an intelligent speech recognition method for power dispatching scenarios, including: A power dispatching scenario voice data set determination module, used to obtain audio sample data; The power dispatching scenario speech recognition model construction module is used to build a speech recognition model based on deep learning; The power dispatching scenario speech recognition model training module is used to input the training data set into the deep learning-based speech recognition model for training to obtain a basic speech recognition model; The power dispatching scenario speech recognition model testing module is used to iteratively train the basic speech recognition model based on the real audio data set to obtain the power dispatching speech recognition model; The power dispatching scenario speech recognition model deployment module is used to convert the power dispatching speech recognition model into a deployment model and perform actual system deployment.

[0013] The beneficial effects of the present invention are: First, compared with the prior art, the present invention avoids deviations in recorded content by combining an intelligent voice recognition method for power dispatching scenarios, and is also conducive to accurate and convenient query and tracing of work logs in the future.

[0014] Second, compared with the prior art, the advantage of the intelligent speech recognition method for power dispatching scenarios of the present invention is that a speech recognition model based on the Conformer network is constructed, and the structure and parameters of the Conformer network are optimized and adjusted to make the model more effective; the network model is trained by combining the cross entropy loss function and the output layer CTC loss function to improve the model accuracy and generalization ability, and at the same time, to a certain extent, it solves the problem that a single loss function cannot accurately and quickly make the model training effectively converge, thereby improving the model training speed; in order to accurately identify the professional terminology of the power dispatching scenario, a professional terminology training module is constructed, and combined with the basic speech recognition model to form a power dispatching speech recognition model with professional terminology recognition ability, thereby effectively improving the model's recognition ability of professional terminology. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of the steps of the intelligent speech recognition method for power dispatching scenarios.

[0016] Figure 2 Optimize the flowchart for step one of the intelligent speech recognition method for power dispatching scenarios.

[0017] Figure 3 Another flowchart of step 1 of the intelligent speech recognition method for power dispatching scenarios.

[0018] Figure 4 This is a flowchart of step two of the intelligent speech recognition method for power dispatching scenarios.

[0019] Figure 5 This is a flowchart of step three of the intelligent speech recognition method for power dispatching scenarios.

[0020] Figure 6 This is a flowchart of step four of the intelligent speech recognition method for power dispatching scenarios.

[0021] Figure 7 This is a flowchart of step five of the intelligent speech recognition method for power dispatching scenarios.

[0022] Figure 8 It is a schematic diagram of the overall structure of the speech recognition model of the present invention.

[0023] Fig. 9 This is a flow chart of the professional terminology module training of the present invention.

[0024] Fig.10 This is a structural diagram of the intelligent speech recognition system for power dispatching scenarios.

[0025] Fig.11 A schematic diagram of the connection structure of hardware devices.

[0026] In the figure, 1101 is a landline telephone; 1102 is a recording device; 1103 is a server host; 1104 is a dispatcher display. DETAILED DESCRIPTION

[0027] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively below in conjunction with the specific embodiments. Preferred embodiments of the present invention are provided in the specific embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0029] Example 1 The present invention provides an intelligent speech recognition method for power dispatching scenarios, which can be applied to terminals or servers, and can also be software running in terminals or servers. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, set-top box, etc.; the server can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the intelligent speech recognition method for power dispatching scenarios, but is not limited to the above forms.

[0030] like Figure 1 , which is a flowchart of the steps of the intelligent speech recognition method for power dispatching scenarios. As can be seen from the figure, it specifically includes the following steps: Step 1: Obtain a public Mandarin Chinese dataset as a basic audio dataset, and determine the training and test datasets based on the audio sample data.

[0031] More specifically, the principle of the speech recognition model is mainly to transcribe audio data in life into text data. Therefore, public audio data containing Mandarin Chinese and real audio data of power dispatching scenarios are collected to form audio sample data, which can be specifically derived from public Internet data, manually annotated data and speech synthesis data.

[0032] like Figure 2 , which is a flowchart for optimizing step 1 of the intelligent speech recognition method for power dispatching scenarios. As can be seen from the figure, the audio data set includes a basic audio data set and a real audio data set. The audio sample data is obtained, and the training and test data sets are determined according to the audio sample data, including: Step 1: Obtain the Mandarin Chinese public datasets AISHELL-1 and Thchs30, which are composed of the cleaned and annotated audio data sets and the text labels corresponding to the audio, as the basic audio data sets; Step 2: extract audio sample data of the power dispatching scenario, split the long audio samples into multiple short audio samples, and annotate them to obtain audio files and text labels corresponding to the audio files as the real audio dataset; Step 3: Construct training and test datasets based on the basic audio dataset, the real audio dataset, and the corresponding text labels. The ratio of the training dataset to the test dataset is divided according to the deep learning training and test data standards.

[0033] More specifically, the text label is the audio data label in the basic audio data set corresponding to the text with accurate annotation of the audio. The embodiment of the present invention obtains the audio data set through two methods: first, obtains the public audio sample data of Mandarin Chinese to form a basic audio data set, including AISHELL-1 and Thchs30. The basic audio data set has been cleaned and annotated by a specialized organization, and the audio data labels in the basic audio data set and their corresponding text labels can be directly obtained; second, extracts the audio sample data of the power dispatching scene, and has professionals perform segmentation and annotation to obtain the audio data labels in the real audio data set and their corresponding text labels.

[0034] like Figure 3 , which is another flowchart of step 1 of the intelligent speech recognition method for power dispatching scenarios, Figure 1Add step 4 based on steps 1-3 in the optimization flow chart, including: Step 4: Perform data augmentation processing on the basic audio data set to obtain an augmented basic audio data set. According to the augmentation processing method, adjust the audio file labels in the augmented basic audio data set and add them to the training and test data sets. More specifically, the augmentation processing includes at least one of audio segmentation, time extension, pitch shifting, and noise addition.

[0035] More specifically, augmentation processing is used to expand the acquired basic audio data set to obtain richer and more diverse audio sample data; to modify the speaking speed of the processed data (audio) in the processed data set, increase the number of training sets, and make the data more diverse; to randomly add reverberation to the processed data in the processed data set to achieve the purpose of simulating a variety of sound scenes; to randomly add noise to the processed data in the processed data set to increase data complexity; and to use Specaugment technology (a technology used for speech recognition enhancement) to modify the audio spectrum of the processed data in the processed data set to improve the quality of audio features.

[0036] Step 2: Build a speech recognition model based on deep learning. The model includes feature extraction layer, convolution layer, linear layer, activation layer, core module layer, and output layer. The core module layer includes feedforward network module, multi-head attention mechanism module, convolution module, residual connection, and layer standard normalization module.

[0037] More specifically, based on the traditional Conformer network model, the embodiment of the present invention optimizes and adjusts the structure and parameters of the traditional Conformer model in consideration of the particularity of the power dispatching scenario, and constructs a new power dispatching speech recognition model.

[0038] like Figure 4 , which is a flowchart of step 2 of the intelligent speech recognition method for power dispatching scenarios. The input feature of the speech recognition model based on deep learning is a high-dimensional Fbank (FilterBank) feature. This feature extraction method can retain more original audio information. The model is mainly composed of a feature extraction layer, a convolution layer, a linear layer, an activation layer, multiple core module layers and an output layer. Each core module layer is composed of a feedforward network module, a multi-head attention mechanism module, a convolution module, a residual connection and a layer standard normalization module. Adjacent modules in the core module layer all have residual connections. Compared with the traditional Conformer network structure, the convolution layer of the present invention uses a causal convolutional neural network to capture local audio feature information, combined with the powerful context modeling capability of the multi-head attention mechanism, making the feature expression capability more powerful.

[0039] More specifically, the convolution kernel size of the speech recognition model is set to 15; the linear layer output feature dimension is 256; the core module contains 12 Conformer network modules, each of which has an output dimension of 256; the feedforward network module dimension is set to 2048; and the number of attention heads of the multi-head attention mechanism module is set to 4.

[0040] Step three, input the basic audio data set into the speech recognition model, and jointly train the speech recognition model based on the cross entropy loss function and the output layer CTC loss function to obtain a basic speech recognition model.

[0041] like Figure 5 , which is a flowchart of step 3 of the intelligent speech recognition method for power dispatching scenarios. The training data set is input into the speech recognition model based on deep learning. The initial speech recognition model is jointly trained based on the cross entropy loss function and the output layer CTC loss function to obtain a trained basic speech recognition model, including: Step 1: Perform preprocessing such as segmentation and labeling on the basic audio data set to generate the data format required for model training; Step 2: Input the preprocessed basic audio data set into the audio feature extraction layer for feature extraction; Step 3: The feature extraction layer extracts features from the audio data in the basic audio data set to obtain feature information of the audio data, which is then input into the linear layer for integration and then transmitted to the activation layer. Step 4: Activate the audio feature information through the activation layer to obtain the audio feature tensor required for training, and transmit the audio tensor to the acoustic model encoder; Step 5: The acoustic model encoder performs a series of operations including feedforward layer processing, multi-head attention layer processing, layer normalization processing, etc. on the input audio feature tensor, so that the input feature vector is concatenated with the processed feature vector to obtain a new audio feature tensor, which is transmitted to the output layer. In the sixth step, supervised training is performed jointly based on the cross entropy loss function and the CTC loss function. The loss value is obtained according to the training results. The model parameters are continuously updated until the loss value converges. The training is stopped to obtain a trained basic speech recognition model.

[0042] More specifically, traditional speech recognition models are usually trained using a cross-entropy loss function. The cross-entropy loss function has the advantage of fast convergence, but it focuses too much on minimizing the features between the predicted results and the true labels during the training process, that is, it focuses too much on the information in the features themselves and ignores the information between different categories to a certain extent. Based on this, the present invention proposes a joint cross-entropy loss function and a CTC loss function to supervise the training of a speech recognition model. The CTC loss function is based on a dynamic programming algorithm and is a loss function widely used in sequence recognition tasks, especially in the fields of speech recognition and handwriting recognition. The main advantage of the CTC loss function is that it can handle the problem of inconsistency between the input sequence length and the label sequence length without alignment, which can alleviate the problems caused by the cross-entropy loss function to a certain extent, thereby improving the accuracy and generalization ability of the speech recognition model.

[0043] Step 4: Obtain the power dispatching audio data to be tested as a real audio data set, use the basic speech recognition model to recognize the real audio data set to be tested, update the parameters of the basic speech recognition model according to the results, and iteratively train to obtain the power dispatching speech recognition model.

[0044] like Figure 6 , which is a flowchart of step 4 of the intelligent speech recognition method for power dispatching scenarios. The real audio data set to be tested is input into the trained basic speech recognition model for multiple iterative training to obtain the text results corresponding to the audio, and the model parameters are continuously updated according to the results. The final step is to obtain the power dispatching speech recognition model, including: Step 1: Get the real audio data set for testing and the trained basic speech recognition model; Step 2: Select audio data from the real audio data set to be tested from the test set and input it into the basic speech recognition model; Step 3: The model extracts the audio sample feature sequence and inputs it into the audio encoder. After being processed by various modules of the encoder, the audio feature tensor information of the test sample is obtained. Step 4: Input the audio feature tensor information and the text result output at the previous moment into the decoder for joint decoding. The encoder obtains the predicted output probability through a series of processes including the feedforward network module, the multi-head attention mechanism module, the convolution module, the layer normalization, and the output layer. Step 5: The final text sequence result is obtained by integrating the predicted probabilities of the output layer and the linear layer. The steps are repeated until the end marker appears to obtain the complete recognition result of the test set. In step 6, based on the pre-labeled test set text labels, the word error rate of the test set audio decoding results is tested to verify whether the model is effective, and the power dispatching speech recognition model is obtained by continuous optimization and fine-tuning.

[0045] Step 5: Convert the trained power dispatch speech recognition model into a deployment model and deploy it to the power bureau computer, build a front-end and back-end framework and combine it with the telephone recorder to form the final system. After the dispatcher answers the call, the back-end module of the system converts the working audio into text in real time, and then transmits it to the front-end page for the dispatcher to view and save locally.

[0046] like Figure 7 , which is the flowchart of step 5 of the intelligent speech recognition method for power dispatching scenarios. The trained power dispatching speech recognition model is converted into a deployment model and deployed to the computer of the power bureau to ensure that the model trained based on the deep learning framework can be applied to the system, including: Step 1: Convert the trained power dispatching speech recognition model into a deployment model and prepare the hardware equipment such as servers, telephone recorders, monitors, and other equipment external connection cables and power supplies required for deployment; Step 2: Generate the deployment model into the format required for final deployment through the deep learning framework to meet the basic requirements for deployment in the server Linux environment, and build a telephone recorder used in the work by combining the front-end and back-end frameworks; Step 3: Deploy the deployment model, backend framework, and frontend framework to the server system, connect the telephone recorder, form an intelligent voice recognition deployment system for the power dispatching scenario, and start the hardware and software required by the system for testing; Step 4: Load the working audio data in the telephone recorder through the entire deployment system, call the internal power dispatching speech recognition model and the back-end framework to transcribe the working audio data in real time, and transmit the results to the front-end framework for display; In step 5, the front-end framework of the deployed system displays the text results transmitted from the back-end framework in real time on the dispatcher's work computer page for the dispatcher to view and modify, and finally save to the local computer.

[0047] like Figure 8 , which is a schematic diagram of the overall structure of the basic speech recognition model of the present invention, and the model structure parts required for training the basic speech recognition model include: a preprocessing part, an acoustic model part and an output layer part.

[0048] Specifically, the preprocessing part includes: feature extraction layer, convolution layer, linear layer and activation layer, etc., which mainly processes the input raw audio sample data into the data format required for speech recognition model training; Specifically, the acoustic model part includes: an acoustic model encoder and an acoustic model decoder. The encoder mainly includes multiple Conformer network blocks, and the decoder mainly includes multiple Transformer network blocks and a CTC decoding network. It mainly encodes the preprocessed audio sample data into high-dimensional audio feature information, and then decodes the information through the decoder to obtain the corresponding decoding information; Specifically, the output layer mainly includes a Softmax output network, and the network structure mainly includes an input layer, a hidden layer, and an output layer. It mainly classifies and outputs the decoded information output by the decoder, outputs different probabilities of each feature information, and finally obtains the text result corresponding to the audio sample data. Among them, the Softmax output converts the output of the network into a probability distribution, so that the probability value of each category is between 0 and 1, and the sum of the probability values ​​of all categories is 1.

[0049] like Fig. 9 , is a flowchart of the professional terminology module training of the present invention, including: Based on the basic audio data set, the real audio data set and the trained electric power dispatching speech recognition model, a professional terminology module is trained to obtain a trained professional terminology module; Specifically, the trained professional terminology module is combined with the power dispatching speech recognition model architecture to form a power dispatching speech recognition model with professional terminology recognition capability; Specifically, the acoustic model includes an audio encoder based on the Conformer network structure, a professional terminology text encoder based on the LSTM network structure, and a decoder based on the Transformer network structure in the basic speech recognition model; Specifically, the audio encoder parameters are frozen, the speech recognition text result is used as the input of the text encoder, the professional terminology module is trained, and then the result is combined with the output of the audio encoder and sent to the decoder. The entire acoustic model training parameters and process refer to step S103; Specifically, the decoder output is transmitted to the Softmax output layer, and the text results and professional terms corresponding to the audio samples are output.

[0050] The embodiment of the present invention introduces the cross entropy loss function and the output layer CTC loss function to jointly train the speech recognition model, so that the speech recognition model can better capture the similarities and differences between training samples, and alleviate the problem that only using the cross entropy loss function will cause the model to pay more attention to the features themselves and ignore the associations between training samples of different categories, so that the speech recognition model can learn more discriminative feature representations, thereby improving the generalization ability of the speech recognition model, and also improving the accuracy and recognition efficiency of speech recognition in power dispatch scenarios. In addition, the embodiment of the present invention also provides a professional terminology module training method, which is combined with the speech recognition model architecture to form a power dispatch speech recognition model with professional terminology recognition capabilities, so that the power dispatch speech recognition model can more effectively identify professional terms in power dispatch scenarios, thereby improving the professional capabilities of the speech recognition model.

[0051] Compared with the prior art, the intelligent speech recognition method for power dispatching scenarios of the present invention has the advantages of constructing a speech recognition model based on the Conformer network, optimizing and adjusting the Conformer network structure and parameters to make the model more effective; combining the cross entropy loss function with the output layer CTC loss function to train the network model, improving the model accuracy and generalization ability, and at the same time solving the problem that a single loss function cannot accurately and quickly make the model training converge effectively to a certain extent, thereby improving the model training speed; in order to accurately identify the professional terminology of the power dispatching scenario, a professional terminology training module is constructed, and combined with the basic speech recognition model to form a power dispatching speech recognition model with professional terminology recognition ability, thereby effectively improving the model's recognition ability of professional terminology.

[0052] Example 2 The present invention provides an intelligent speech recognition system for power dispatching scenarios, such as Fig.10 , which is a schematic diagram of the structure of the intelligent speech recognition system for power dispatching scenarios, including: A power dispatching scenario voice data set determination module is used to obtain audio sample data and determine a training data set and a test data set based on the audio sample data; The power dispatching scenario speech recognition model construction module is used to build a speech recognition model based on deep learning. The model includes a feature extraction layer, a convolution layer, a linear layer, an activation layer, a core module layer, and an output layer. The core module layer includes a feedforward layer module, a multi-head attention mechanism module, a convolution layer module, and a layer standard normalization module. The core module layer and the output layer are the core parts of the acoustic model encoder and decoder. The power dispatching scenario speech recognition model training module is used to input the training data set into the deep learning-based speech recognition model for training to obtain a basic speech recognition model; The power dispatching scenario speech recognition model test module is used to perform inference recognition on the real audio data set to be tested according to the basic speech recognition model, obtain the corresponding speech recognition text result, and update the basic speech recognition model according to the result to obtain the power dispatching speech recognition model; The power dispatching scenario speech recognition model deployment module is used to convert the power dispatching speech recognition model into a deployment model and perform actual system deployment. According to the back-end and front-end frameworks, the audio data in the dispatching work is recognized and transcribed into text and saved locally on the computer.

[0053] The contents in Example 1 are all applicable to the embodiment. The functions specifically implemented by the system of this embodiment are the same as the method in Example 1, and the beneficial effects achieved are also the same as the beneficial effects achieved by the above method embodiment.

[0054] Example 3 The present invention also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the above-mentioned intelligent speech recognition method for power dispatching scenarios is realized.

[0055] like Fig.11 , is a schematic diagram of the connection structure of hardware devices, the electronic equipment includes: Fixed telephone 1101 can be a common household fixed telephone, which includes basic functions such as normal dialing, connecting, and hands-free. The telephone line interface of the fixed telephone needs to match the junction box of RJ11 interface type; The recording device 1102 may be a recorder capable of recording fixed-line telephones, including basic recording, connection reminder, hang-up feedback, record preservation and other functions. The recording device interface also needs to match the RJ11 interface type. In addition, the recording device needs to have an RJ45 network cable interface to ensure that it can be connected to the server host. The server host 1103 may be any type of server with processor, memory, graphics card, power supply and other related hardware, and needs to be able to meet the basic conditions for deploying the speech recognition model and to process multiple fixed-line telephone input audio data in parallel; The dispatcher display 1104 may be a common display device with basic display functions and an HDMI interface to ensure that it can be connected to the server host; The communication and power supply of the above devices need to meet their own hardware conditions, and are mainly connected and communicated through wired means, and the technical solutions provided in the embodiments of this specification are implemented through software or firmware.

[0056] Example 4 The present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned intelligent speech recognition method for power dispatching scenarios.

[0057] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network.

[0058] The embodiment of the present invention also discloses a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions.

Claims

1. An intelligent speech recognition method for power dispatching scenarios, characterized by: Here are the steps: S1, obtain a public Mandarin Chinese dataset as a basic audio dataset, and determine a training and test dataset based on the audio sample data; S2, build a speech recognition model based on deep learning, which includes feature extraction layer, convolution layer, linear layer, activation layer, core module layer, and output layer. The core module layer includes a feedforward network module, a multi-head attention mechanism module, a convolution module, a residual connection, and a layer standard normalization module; S3, inputting the basic audio data set into the speech recognition model, jointly training the speech recognition model based on the cross entropy loss function and the output layer CTC loss function to obtain a basic speech recognition model; S4, obtaining the power dispatching audio data to be tested as a real audio data set, using the basic speech recognition model to recognize the real audio data set to be tested, updating the parameters of the basic speech recognition model according to the result, and iteratively training to obtain the power dispatching speech recognition model; S5, converts the trained power dispatching speech recognition model into a deployment model and deploys it to the computer of the power bureau, builds the front-end and back-end framework and combines it with the telephone recorder to form the final system; after the dispatcher answers the call from the agent, the back-end module of the system converts the working audio into text in real time, and then transmits it to the front-end page for the dispatcher to view and save locally.

2. The intelligent speech recognition method for power dispatching scenarios according to claim 1 is characterized by: The specific steps of S1 are: Step 1: Obtain the Mandarin Chinese public datasets AISHELL-1 and Thchs30, and use the audio data labels in the cleaned and labeled basic audio datasets as the basic audio datasets; Step 2: extract audio sample data of the power dispatching scenario, split the long audio samples into multiple short audio samples, and annotate them to obtain audio files and text labels corresponding to the audio files as the real audio dataset; Step 3: Construct training and test datasets based on the basic audio dataset, the real audio dataset, and the corresponding text labels. The ratio of the training dataset to the test dataset is divided according to the deep learning training and test data standards.

3. The intelligent speech recognition method for power dispatching scenarios according to claim 2 is characterized by: In step 4, the basic audio dataset is augmented to obtain an augmented basic audio dataset. The audio file labels in the augmented basic audio dataset are adjusted according to the augmentation processing method and added to the training and test datasets.

4. The intelligent speech recognition method for power dispatching scenarios according to claim 1 is characterized by: The convolution kernel size of the convolution layer in the speech recognition model is set to 15; the output feature dimension of the linear layer is 256; the Conformer network modules in the core module layer contain 12, and the output dimension of each module is 256; the feedforward network module dimension is set to 2048; and the number of attention heads of the multi-head attention mechanism module is set to 4.

5. The intelligent speech recognition method for power dispatching scenarios according to claim 1 is characterized by: The specific steps of S3 are: Step 1: Perform preprocessing such as segmentation and labeling on the original audio data to generate the data format required for model training; Step 2: Input the preprocessed audio sample data into the audio feature extraction layer for feature extraction; Step 3: Extract features from the audio data through the feature extraction layer to obtain feature information of the audio data, input it into the linear layer for integration, and then transmit it to the activation layer; Step 4: Activate the audio feature information through the activation layer to obtain the audio feature tensor required for training, and transmit the audio tensor to the acoustic model encoder; Step 5: The acoustic model encoder performs a series of operations including feedforward layer processing, multi-head attention layer processing, layer normalization processing, etc. on the input audio feature tensor, so that the input feature vector is concatenated with the processed feature vector to obtain a new audio feature tensor, which is transmitted to the output layer. In the sixth step, supervised training is performed based on the cross entropy loss function and the output layer CTC loss function. The loss value is obtained according to the training results, and the model parameters are continuously updated until the loss value converges. The training is stopped to obtain a trained basic speech recognition model.

6. The intelligent speech recognition method for power dispatching scenarios according to claim 1 is characterized by: The specific steps of S4 are: Step 1: Obtain the real audio data set to be tested and the trained basic speech recognition model; Step 2: Select the audio sample data to be tested from the test set and input it into the basic speech recognition model; Step 3: The model extracts the audio sample feature sequence and inputs it into the audio encoder. After being processed by various modules of the encoder, the audio feature tensor information of the test sample is obtained. Step 4: Input the audio feature tensor information and the text result output at the previous moment into the decoder for joint decoding. The encoder goes through a series of processes including the feedforward network module, the multi-head attention mechanism module, the convolution module, the layer normalization module, and the output layer to obtain the predicted output probability. Step 5: The final text sequence result is obtained by integrating the predicted probabilities through the output layer and the linear layer. The steps are repeated until the end marker appears to obtain the complete recognition result of the test set. In step 6, based on the pre-labeled test set text labels, the word error rate test is performed on the test set audio decoding results to verify whether the model is effective, and the power dispatching speech recognition model is continuously optimized and fine-tuned.

7. The intelligent speech recognition method for power dispatching scenarios according to claim 1 is characterized by: The specific steps of S5 are: Step 1: Convert the trained power dispatching speech recognition model into a deployment model and prepare the hardware equipment such as servers, telephone recorders, monitors, and other equipment external connection cables and power supplies required for deployment; Step 2: Generate the deployment model into the required format through the deep learning framework to meet the basic requirements of deployment in the server Linux environment, and build a telephone recorder used in the work by combining the front-end and back-end frameworks; Step 3: Deploy the deployment model, backend framework, and frontend framework to the server system, connect the telephone recorder, and form an intelligent voice recognition deployment system for power dispatching scenarios; Step 4: Start the hardware and software required for the system for testing, load the working audio data in the telephone recorder through the entire deployment system, call the deployment model and the back-end framework to transcribe the actual working audio data in real time, and transmit the results to the front-end framework for display; In step 5, the front-end framework of the deployed system displays the text results transmitted from the back-end framework in real time on the dispatcher's work computer page for the dispatcher to view and modify, and finally save to the local computer.

8. An intelligent speech recognition system for power dispatching scenarios, characterized by: The intelligent speech recognition method for electric power dispatching scenarios as described in any one of claims 1 to 6 comprises: A power dispatching scenario voice data set determination module, used to obtain audio sample data; The power dispatching scenario speech recognition model construction module is used to build a deep learning speech recognition model; The power dispatching scenario speech recognition model training module is used to input the training data set into the deep learning-based speech recognition model for training to obtain a basic speech recognition model; The power dispatching scenario speech recognition model testing module is used to iteratively train the basic speech recognition model based on the real audio data set to obtain the power dispatching speech recognition model; The power dispatching scenario speech recognition model deployment module is used to convert the power dispatching speech recognition model into a deployment model and perform actual system deployment.

Images