Electrical system operation instruction ticket identification method based on secondary voice verification

Abstract

The invention discloses an electric system operation instruction ticket identification method based on secondary voice verification, and belongs to the field of electric power system operation. Comprising the steps that the mobile terminal determines whether step verification is consistent with an original verification sequence; the user reads the first item of the power transformation switching operation ticket operation item; judging whether the characters read by the user are consistent with the characters corresponding to the power transformation switching operation ticket; if yes, reading the same content once by the auditor, and judging again; if not, immediately prompting the user that the reading content is not matched with the operation item; after matching of one operation item is completed, traversing the whole operation power transformation switching operation ticket item, and performing reading identification and matching by the user in sequence until reading of the whole power transformation switching operation ticket is completed, thereby completing the operation; through a secondary voice verification mode, verification of retelling instruction identification information is completed; and after the operation auditing is completed, the information of the operation ticket operation is synchronously filled into the operation ticket document, so that the correctness of the switching operation is ensured, and the flow control of the switching operation ticket operation is realized in a wireless network environment.

Description

Technical Field

[0001] The invention belongs to the field of electric power operation control, and in particular relates to a voice instruction recognition method for electric system operation instruction tickets. Background Art

[0002] With the growth of electricity consumption, the power system is facing the continuous increase in the amount of information and data in all links of generation, transmission, transformation and distribution, and its requirements for the transmission, analysis and processing speed of information data are also increasing.

[0003] Under the traditional method, the execution process of various dispatching work orders (maintenance orders, operation tickets) relies entirely on manual follow-up, reminders, and supervision. The timeliness of the entire operation process relies on manual supervision and telephone reminders. With the continuous growth of business volume, the labor intensity of personnel is high, and the pressure on monitoring the dispatching work order process is very high, which can easily lead to problems such as untimely monitoring and error-prone dispatching work order processes.

[0004] Artificial intelligence technology widely involves many fields such as human-computer mode interaction and program automation execution. It has extremely strong information processing capabilities and can effectively realize the automation, programming and intelligence of power working modes with the help of modern science and technology. Among them, the combination of voice interaction technology and traditional control work has also become an important research topic to improve operational efficiency and further ensure the safe and stable electricity use of all citizens.

[0005] Voice is the most natural means of input. In recent years, voice recognition has been widely used in smart homes, car navigation and other occasions. As the basis of production and life, it is inevitable for electricity to absorb voice processing technology.

[0006] The invention patent with the authorization announcement date of 2021.12.28 and the authorization announcement number CN 110808040 B discloses a "system and method for interlocking work tickets and operation ticket processes based on voice control", including a first mobile terminal for use by operators, a second mobile terminal for use by work supervisors, and a third mobile terminal for use by work permit holders, as well as a progress reporting module and a scheduling module. The progress reporting module is used for on-site personnel to send on-site progress confirmation information to the scheduling module through the first mobile terminal or the second mobile terminal. The scheduling module is used to receive each on-site progress confirmation information sent by the progress reporting module. When the on-site progress confirmation information of all on-site personnel in the operation ticket is received, the next scheduling task is allowed to be executed. The method includes S1 progress reporting and S2 scheduling steps; it realizes effective control of the power operation process through the first mobile terminal, the second mobile terminal, the third mobile terminal, the progress reporting module and the scheduling module.

[0007] The invention patent with the authorization announcement date of 2021.07.30 and the authorization announcement number CN 112990791 B discloses "a dispatch work order process monitoring method and system based on intelligent voice". It identifies the current operation link and operation order of the maintenance operation ticket generated by the maintenance order, and judges whether the time difference of the operation order has timed out. When a timeout occurs, the warning message is transmitted to the browser through the voice function for broadcast and content display, thereby monitoring the process of each operation link, and providing voice reminders and supervision on the timeliness of the business process. Early warning can be achieved, which improves the timeliness and interactivity of monitoring, solves the technical problems of untimely monitoring and error-prone dispatch work order processes, and effectively improves the efficiency of power dispatching work in daily work, especially in emergency fault conditions.

[0008] The technical solution disclosed in the above patent focuses on the operation process of work tickets, operation tickets or process monitoring of scheduling work orders, and uses voice technology to remind or confirm information, but does not involve "how to improve the accuracy of voice recognition", or in other words, it does not involve how to improve the accuracy of voice recognition, that is, how to ensure the accuracy, real-time and robustness of voice command recognition, thereby improving the reliability and stability of the system.

[0009] At present, the research on speech recognition technology in the power field is relatively weak. Among them, the invention patent application with the application publication date of 2021.05.18 and the application publication number of CN 112820282 A discloses a "power grid dispatching system based on speech recognition and natural language processing", including a language recognition unit, a dispatching system and a playback system. The dispatching system includes a language receiving module, a processing module 1, a data acquisition module, a data analysis module, a big data module, a scheme optimization module, an interactive module 1, a risk assessment module and an alarm module. When the data of the power grid is abnormal, it can transmit the information to the interactive module 2 through the interactive module 1, and the processing module 2 broadcasts the information through the broadcast module, so as to remind the dispatcher. The technical solution can collect the language instructions of the dispatcher through the language recognition unit, and send them to the dispatching system after processing, thereby issuing instructions to the processing module 1, and the processing module 1 processes the abnormal information of the power grid according to the instructions, which effectively reduces the workload of the dispatcher.

[0010] However, in actual applications, it is found that during the inspection of the power system, factors such as high noise on site and inspection by non-specific personnel pose challenges to the accurate recognition of speech.

[0011] Therefore, an intelligent voice recognition mode suitable for the power system needs to have a strong suppression capability against on-site electromagnetic noise and be able to adapt to the inspection environment of different sites.

[0012] In the power system, voice command recognition is a very practical technology. However, due to the particularity of the power system, voice command recognition in the power system faces many challenges and difficulties.

[0013] First, the complexity of voice commands makes the processing of voice signals very complicated. Voice signals are subject to noise, accents, speaking speed, intonation and other interferences, and it is very difficult to extract useful information from them. Secondly, the complexity of the power system itself increases the difficulty of voice command recognition. There are various devices, control commands and state changes in the power system. How to identify these commands and accurately operate the equipment requires professional knowledge and high technology. In addition, the environment of the power system is also very complex. For example, in harsh environments such as high temperature, low temperature, and noise, the accuracy of voice commands will be greatly affected. Finally, when doing voice command recognition, safety is a very critical issue. If someone maliciously attacks the system and identifies the wrong command to control the equipment, it will have very serious consequences for the power system.

[0014] In order to solve these problems and challenges, it is necessary to develop a power system voice command recognition method with secondary voice verification function. This algorithm needs to have advanced voice signal processing, feature extraction, model training and other capabilities. At the same time, considering the particularity of the power system, it is necessary to optimize and adjust the algorithm in a targeted manner based on the characteristics of the power system. Only in this way can the accurate recognition and control of power system commands be achieved. Summary of the invention

[0015] The technical problem to be solved by the present invention is to provide a method for recognizing electric system operation instruction tickets based on secondary voice verification. It recognizes the voice instructions of the electric system operation ticket to assist operators in operating the switching operation ticket operation instructions, and adopts the mode of secondary voice verification to verify the voice instructions in the process of executing the switching operation ticket; its support vector machine classifier performs classification, and realizes voice instruction recognition based on Gaussian mixture model by performing voice instruction recognition in the electric system voice control scenario, which has high recognition accuracy and real-time performance in the electric system, good applicability and robustness, and can meet the requirements of voice instruction recognition in complex environments in the electric system.

[0016] The technical solution of the present invention is to provide a method for identifying an electric system operation instruction ticket based on secondary voice verification, including receiving a substation switching operation ticket at a mobile terminal, which is characterized by comprising the following steps:

[0017] 1) The PC sends the identified substation switching operation ticket file data to the server / mobile terminal;

[0018] 2) After the mobile terminal obtains the data, it performs a step verification on it;

[0019] 3) Determine whether the step verification is consistent with the original verification sequence. If it is confirmed, the next step is to select the substation switching operation ticket. If it is not confirmed, manual step deployment is performed;

[0020] 4) After selecting the substation switching operation ticket, the user needs to read aloud the first item of the operation;

[0021] 5) The program determines whether the text read aloud by the user is consistent with the text corresponding to the serial number on the substation switching operation ticket; if they are consistent, the next operation is executed, and the auditor reads aloud again to make a judgment; if they are inconsistent, the user is immediately prompted that "the reading content does not match the operation item, please check and read again";

[0022] 6) After one operation item is matched, the entire substation switching operation ticket items are traversed, and the user reads, identifies and matches them one by one until the entire substation switching operation ticket is read and the operation is completed. The user can exit the program or select another substation switching operation ticket again;

[0023] 7) Through the secondary voice verification mode, the voice content recognition and review function is completed;

[0024] 8) The voice command recognition of the power system operation ticket is used to assist the operator in operating the switching operation ticket operation instructions, verify the voice commands in the process of executing the switching operation ticket, so as to realize the voice command recognition based on the Gaussian mixture model and meet the requirements of voice command recognition in complex environments in the power system.

[0025] Specifically, in the step 1), the Scoket mode is adopted to establish communication, and the output stream of the communication is used to send the substation switching operation ticket file to the mobile terminal.

[0026] Furthermore, in the step 1), SFTP communication is established, the PC uploads the file to the server for storage, and the mobile terminal obtains the switching operation ticket file from the server.

[0027] Furthermore, in the step 1), SFTP communication is established, the PC uploads the file to the server for storage, and the mobile terminal obtains the switching operation ticket file from the server, which at least includes the following steps:

[0028] a) Get the transfer file input stream object;

[0029] b) reading the incoming stream data of the transmission file;

[0030] c) Upload to the cloud server using FTP tool.

[0031] Specifically, in the step 3), if it is not possible to determine whether the step verification is consistent with the original verification sequence, then manually review whether the operation steps of the substation switching operation ticket are correct, and reselect the switching operation ticket.

[0032] Specifically, in the steps 4) and 5), the system obtains a passage of language read aloud by the user, obtains the language instructions of the substation switching operation ticket, and uploads them to the Tencent Cloud server for online voice recognition; the system calls back the online voice recognition results of Tencent Cloud to determine whether the voice recognition results are consistent with the content of the corresponding operation items of the substation switching operation ticket.

[0033] Specifically, in the above 5), if it is determined that the voice recognition result is inconsistent with the content of the corresponding operation item of the substation switching operation ticket, the system issues an "error prompt" to indicate which line has a problem and marks the operation instruction content of the substation switching operation ticket of the voice recognition;

[0034] If it is judged that the voice recognition result is consistent with the content of the corresponding operation item of the substation switching operation ticket, the voice match of the single substation switching operation ticket is complete, and the system returns to the step of "selecting the substation switching operation ticket" again.

[0035] Specifically, the voice content recognition and review function includes converting a voice signal into corresponding text information; the system includes at least four parts: feature extraction, acoustic model, language model, and dictionary and decoding; the collected sound signal is preprocessed including filtering and framing to extract the signal to be analyzed from the original signal; then, the feature extraction work converts the sound signal from the time domain to the frequency domain to provide a suitable feature vector for the acoustic model; the acoustic model then calculates the score of each feature vector on the acoustic feature based on the acoustic characteristic; and the language model calculates the probability of the sound signal corresponding to a possible phrase sequence based on relevant linguistic theories; finally, the phrase sequence is decoded according to the existing dictionary to obtain the final possible text representation.

[0036] Furthermore, the relationship between the acoustic model and the language model is expressed by the Bayesian formula:

[0037]

[0038] Among them, P(X|W) is called the acoustic model, and P(W) is called the language model.

[0039] Furthermore, the language model is used to process the text sequence, and combined with the output of the acoustic model, a text sequence with the highest probability is given as a speech recognition result.

[0040] Compared with the prior art, the advantages of the present invention are:

[0041] 1. The technical solution of the present invention successfully proposes a command recognition method based on voice signal processing according to the characteristics of voice command recognition of power system operation tickets; the method has a secondary voice verification function, can quickly and accurately recognize the commands spoken by the user, and realize process control in a wireless network environment, and has high practical value;

[0042] 2. The technical solution of the present invention uses a support vector machine classifier for classification, and realizes voice command recognition based on a Gaussian mixture model by performing voice command recognition in a power system voice control scenario. It has high recognition accuracy and real-time performance in the power system, good applicability and robustness, and can meet the requirements of voice command recognition in a complex environment in the power system;

[0043] 3. The technical solution of the present invention uses the voice command recognition of the power system operation ticket to assist the operator in operating the switching operation ticket operation instructions, verifying and checking the voice commands in the process of executing the switching operation ticket, and prompting or warning the wrong voice commands to further improve the accuracy of the voice commands and reduce the probability of wrong operations, thereby ensuring the correctness of the "switching" operation;

[0044] 4. The technical solution of the present invention extracts commonalities from the rich resources of mainstream languages, trains a publicly available model, and on this basis establishes data suitable for the recognition of voice commands in the power system. Through a machine learning algorithm, the recognition training of voice commands in the power system is performed, thereby improving the accuracy of the recognition of voice commands in the power system.

[0045] 5. The technical solution of the present invention has a secondary voice verification function, and users and staff can perform secondary instruction repetition and verification operations on the software, reducing time and labor costs; the verification of the repetition instruction recognition information only requires the user to go through the system, and it can be completed with one-click operation; the recognition process is intelligent, and real-time recognition can be achieved; the operation content during the instruction recognition process, after the operation review is completed, the information of the operation ticket operation will be filled into the operation ticket document, reducing the steps of manually filling in the operation information of the operation ticket, thereby improving work efficiency, and can also provide document downloads for completed operation ticket instruction recognition and verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the overall structure of the power system voice command recognition method based on machine learning of the present invention;

[0047] Figure 2 It is a schematic diagram of the feature extraction process of MFCC of the present invention;

[0048] Figure 3 It is a schematic diagram of the hidden Markov model;

[0049] Figure 4 This is a schematic diagram of the acoustic model based on GMM-HMM;

[0050] Figure 5 It is a schematic diagram of the overall flow of the method for recognizing voice instructions of an operation ticket of an electric power system according to the present invention;

[0051] Figure 6 It is a schematic diagram of the speech recognition method in the technical solution of the present invention;

[0052] Figure 7 It is a schematic diagram of the software architecture of speech recognition of the technical solution of the present invention;

[0053] Figure 8 It is a control flow block diagram of speech recognition of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings.

[0055] Figure 1 In the figure, a schematic diagram of the overall structure of the method for recognizing electric system operation instruction tickets based on secondary voice verification according to the technical solution of the present invention is given. The fundamental purpose is to provide a voice recognition method and mode, including on-site voice input, on-site voice preprocessing, and feature extraction of on-site voice; from a pre-constructed voice database, feature extraction of the pre-constructed voice database is performed, and acoustic model training is performed to obtain an acoustic model; a text database of power system operation commands is used to train a language model to obtain a language model; the feature extraction of the on-site voice, the acoustic model, the pronunciation dictionary and the language model are comprehensively processed, and the final text output is obtained through voice decoding and search algorithms (collectively referred to as "voice instruction recognition method").

[0056] The voice command recognition method comprises the steps of voice signal enhancement, voice frame segmentation, spectrum smoothing and the like.

[0057] The preprocessing of on-site speech is based on the collection and preprocessing of speech signals, acoustic feature extraction, feature processing and dimensionality reduction speech recognition. The speech command recognition algorithm based on GMM-HMM, DNN, CNN and RNN is used to implement the power system speech command recognition method. The power system related voice commands are collected for algorithm training and testing, thereby improving the recognition rate of power system voice commands.

[0058] Since the power system involves more professional knowledge and terminology, compared with commonly used voice recognition, the power system professional terminology and voice commands need to establish a higher correlation to ensure the accuracy of voice command recognition.

[0059] The basic idea to solve this problem can be summarized as extracting commonalities from the rich resources of mainstream languages ​​to train a publicly available model, and on this basis, establishing data suitable for the recognition of voice commands in power systems. Through machine learning algorithms, recognition training is performed on power system voice commands to improve the accuracy of voice command recognition in power systems.

[0060] In the technical solution of the present invention, in the scenario application of the power system voice command recognition method in the execution of the switching operation ticket, a verification function is provided for the original voice command operation mode or process to ensure the accuracy of the voice command and reduce the probability of erroneous operation.

[0061] The technical solution of the present invention applies the power system voice command recognition technology to the relevant operating systems in the power system, combines the technology with application software, and uses the power system operation ticket voice command recognition to assist operators in operating the switching operation ticket operation instructions, verify and check the voice instructions in the process of executing the switching operation ticket, in order to further improve the accuracy of voice instructions and reduce the probability of misoperation, prompt or alarm for erroneous voice instructions to ensure the correctness of the switching operation.

[0062] The technical solution of the present invention uses voice recognition technology to enable the operator to repeat the operation ticket instructions twice. Only when the two repetitions are both "repetitions are successful" will the relevant instructions of the operation ticket be executed.

[0063] The voice command recognition of the power system operation ticket needs to be applied to the relevant operation ticket system. The application software uses the work ticket for entering the switching operation as the interface, and the system needs to recognize the operation steps in the work ticket.

[0064] Through voice recognition technology, the switching instructions in the operation ticket are converted into voice, thereby realizing functions such as verification of whether the switching operation task is consistent with the dispatching instruction, whether the operation ticket is filled out correctly and completely, whether the ticket is clear, and whether the listed safety measures meet the requirements of the work ticket.

[0065] In addition, the power system voice command recognition technology can also verify that the issuer, recipient, operation start and end time on the switching operation ticket are consistent with the actual situation, thereby improving the execution efficiency and accuracy of the operation ticket.

[0066] Its specific workflow is that the power system operation ticket voice command recognition system needs to log in with a username and password on the login page. Only by entering a valid username and password can the login be successful. This method ensures the reliability and security of the system.

[0067] Enter the operation ticket and obtain a set of substation switching operation tickets. First, the system sorts the switching operation tickets into the database, and manually checks whether the switching operation ticket steps are correct. If correct, directly enter the operation ticket selection step. If incorrect, manually sort and update the database operation ticket information.

[0068] After a single operation ticket is successfully selected in the operation ticket, start the voice recognition rehearsal operation. The actual operation can only begin after the rehearsal is successful at least once.

[0069] During the voice recognition process, the voice instructions for the operation items on the operation ticket are obtained, the voice instruction audio is obtained through the microphone voice instruction, and then the voice instruction is recognized through voice recognition technology, and the voice instruction repetition of the operation items on the switching operation ticket is voice reviewed and verified; if the voice review and verification is successful, the second voice instruction repetition can be entered; if the voice review and verification fails, the content message "Repetition failed" is prompted and the voice instruction needs to be repeated from the first time.

[0070] Only when the second voice command repetition is successful, the single switching operation ticket is matched. You can select another switching operation ticket again to perform voice command repetition verification.

[0071] The electric power system voice command recognition system based on machine learning constructed by the technical solution of the present invention adds intelligent voice recognition auxiliary correction processing on the basis of voice recognition, so that the recognition efficiency in the electric power system voice command recognition process is further improved, and an auxiliary voice recognition library related to electric power system voice commands is formed.

[0072] The advantage of the technical solution of the present invention is that the efficiency of voice command recognition in electrical system operation voice command recognition is greatly improved. Through the auxiliary recognition of intelligent voice recognition, the recognition ability of basic voice recognition for some electrical-related special terms can be continuously improved, thereby greatly improving the recognition ability of basic voice recognition for electrical-related voice commands.

[0073] Traditionally, operation instruction repetition is to repeat the instructions manually to perform the operation. After adopting the technical solution of the present invention, the electronic operating system can effectively improve its work efficiency. On this basis, intelligent voice instruction recognition and verification are added to reduce misoperation, thereby greatly improving work efficiency.

[0074] After adopting the technical solution of the present invention, users and staff can perform instruction repetition and verification operations on the software of the wireless communication terminal (for example, handheld wireless communication tools such as mobile phones), reducing time costs, labor costs, etc.

[0075] The verification and checking of the identification information of the repetition instruction only requires the user to go through the system and complete it with one-click operation.

[0076] The technical solution of the present invention makes the recognition process intelligent and can realize real-time recognition. After the operation review is completed, the operation content in the instruction recognition process will fill the operation information of the operation ticket into the operation ticket document, reducing the steps of manually filling in the operation information of the operation ticket, thereby improving work efficiency, and also providing a convenient function to download the document after the operation ticket instruction recognition and verification is completed.

[0077] The technical solution of the present invention is helpful to build a "one-click invoicing" system and provide technical support for intelligent voice command recognition for the digital "switching" operation mode.

[0078] Specifically, the technical solution of the present invention successfully proposes a command recognition method based on voice signal processing based on the characteristics of voice command recognition of power system operation tickets. The algorithm can quickly and accurately recognize the commands spoken by the user and realize process control in a wireless network environment, which has high practical value.

[0079] The power system voice command recognition method is a voice signal processing method based on deep learning to improve the recognition rate and accuracy of voice commands. In the research process, a variety of deep learning models and algorithms were used, such as convolutional neural network (CNN), long short-term memory network (LSTM) and attention mechanism.

[0080] The technical solution of the present invention preprocesses the speech signal, including operations such as denoising, downsampling and normalization, so as to reduce the interference of signal noise on the recognition effect; secondly, regularization methods such as dropout and L2 regularization are added to the model training to prevent the model from overfitting; during the training process, the technical solution of the present invention uses a deep neural network-hidden Markov model to train the acoustic model, and adopts an adaptive learning rate adjustment strategy based on the Adam optimization algorithm to improve the model performance and training efficiency; at the same time, the weights and biases of the model are initialized to ensure that the model can converge and optimize quickly.

[0081] Figure 2 In the method, the MFCC (Mel-frequency cepstral coefficients) feature extraction process is a speech feature extraction process, thereby realizing speech feature scalarization, and then the power system voice command recognition training is performed through a machine learning algorithm to improve the accuracy of power system voice command recognition.

[0082] The power system voice command recognition method described in the technical solution of the present invention provides a verification function for the original voice command operation mode or process in the scenario application of executing the switching operation ticket, thereby ensuring the accuracy of the voice command and reducing the probability of erroneous operation.

[0083] The overall process of voice command recognition for power system operation tickets is to use the power system operation ticket voice command recognition software system. Before using the system, you need to install the MySQL database in the window and the JDK operating environment for running the JAVA program.

[0084] The voice command recognition software system for operating the power system operation ticket is installed on the window system or server, and the operation ticket is generated by the relevant operation task ticket system, and the voice command recognition software system is provided for command verification and checking.

[0085] The electrical operation instruction ticket is identified from the electrical operation ticket, and the identified operation ticket is sent to the mobile terminal. The staff operates the mobile terminal to select the matching operation ticket, and reads the operation item according to the assessment and verification steps based on the content of the operation ticket. The software verifies the information read by the staff. If the verification passes, the auditor needs to perform a second reading verification. After the two people pass the verification, jump to the next operation item; if an error occurs in the reading, the software needs to actively prompt the user that the reading content does not match the operation item, issue a warning, and remind the staff to pay attention to the operation.

[0086] Machine learning is a part of artificial intelligence. It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. The ultimate goal of artificial intelligence is to enable computers to simulate human thinking and behavior.

[0087] The existing power speech recognition is a type of machine learning algorithm based on supervised learning. It labels the power system speech command data in detail, and then uses the labels of these data labels for training data to learn a function mapping from input variable X to input variable Y. From learning with category labels to training based on the input and output of training samples to obtain the optimal model, the model is then used to predict the output of new inputs, that is, the recognition result with unique characteristics that conforms to the power system speech command.

[0088] However, it is found in actual applications that this type of speech recognition algorithm has the defect of overfitting, that is, it performs well on the training data set but poorly on unknown data. The reason is that the voice instructions of the power system are different from ordinary vernacular. It has deep professionalism and complexity in the power industry, which makes the overall algorithm too complex. When encountering a new hypothesis model, it will be judged as not being able to reasonably exist or the conditions for the hypothesis to be established cannot be recognized. Based on the above defects, the technical solution of the present invention summarizes and reduces the dimension of complex power system voice instructions by acquiring and using more data (data set enhancement) and feature dimensionality reduction (reducing the dimension of features by artificially selecting retained features).

[0089] In the process of extracting features and reducing the dimensionality, a large number of power system voice instructions are subjected to secondary data processing by feature escape, and these instructions are secondary extracted and summarized by machine learning. Based on the above two methods, the machine learning model of the power system voice instruction recognition method described in the present invention can be reasonably constructed.

[0090] The so-called speech recognition is to convert a speech signal into corresponding text information. The system mainly includes four parts: feature extraction, acoustic model, language model, dictionary and decoding. In order to extract features more effectively, it is often necessary to perform preprocessing such as filtering and framing on the collected sound signal to extract the signal to be analyzed from the original signal; then, the feature extraction work converts the sound signal from the time domain to the frequency domain to provide a suitable feature vector for the acoustic model; the acoustic model then calculates the score of each feature vector on the acoustic feature based on the acoustic characteristic; and the language model calculates the probability of the sound signal corresponding to the possible phrase sequence based on the relevant linguistic theories; finally, according to the existing dictionary, the phrase sequence is decoded to obtain the final possible text representation.

[0091] Among them, the relationship between the acoustic model and the language model is expressed by the Bayesian formula:

[0092]

[0093] Among them, P(X|W) is called the acoustic model, and P(W) is called the language model. Most studies treat the acoustic model and the language model separately, and the differences in the acoustic models of different manufacturers' speech recognition systems are also reflected in the differences in the acoustic models. In addition, the end-to-end method based on big data and deep learning is also developing. It directly calculates P(X|W), that is, it treats the acoustic model and the language model as a whole.

[0094] The voice recognition operation is triggered by the interaction between the system and the user to verify the operation items of the substation switching operation ticket after the user selects the substation switching operation ticket and confirms the verification of the operation items in the selected substation switching operation ticket.

[0095] For recognition, it is necessary to accurately identify the words read by the user and then match them with the operation items on the substation switching operation ticket. For the machine learning algorithm mentioned in the project, the software will establish a semantic library dedicated to the user's working environment based on the results of the speech recognition algorithm. The number of uses, calls, verifications, etc. can represent the frequent use of the calibration, that is, put this phrase into the semantic library, and the phrase in the semantic library can be matched first during the operation of the speech recognition function.

[0096] The acoustic model is to link the observed features of the speech signal with the speech modeling unit of the sentence, that is, to calculate P(X|W). The Hidden Markov Model (HMM) is usually used to solve the indefinite length relationship between speech and text.

[0097] For example, in Figure 3 In the hidden Markov model shown in , the acoustic model is expressed as:

[0098] P(X|W)=P(w1)P(x1|w1)P(w2|w1)P(x2|w2)P(w2|w2)P(x3|w2)

[0099] Among them, the initial state probability P(w1) and state transition probability (P(w2|w1), P(w2|w2)) can be calculated by conventional statistical methods, and the emission probabilities (P(x1|w1), P(wx2|w2), P(x3|w2}) can be solved by the mixed Gaussian model GMM or the deep neural network DNN.

[0100] GMM-HMM-based speech recognition can improve recognition performance by utilizing the strong learning ability of DNN. The acoustic model diagram is as follows: Figure 4 as shown in .

[0101] The difference between GMM-HMM and DNN-HMM is that DNN is used to replace GMM to solve the emission probability P(xj|w i ), the advantage of the GMM-HMM model is that it has a small amount of calculation and good effect; the DNN-HMM model improves the recognition rate, but requires high computing power for the hardware.

[0102] The language model in speech recognition is also used to process text sequences. It combines the output of the acoustic model to give the text sequence with the highest probability as the speech recognition result.

[0103] Since the language model represents the probability of a certain text sequence occurring, it is generally represented by the chain rule, such as W is composed of w1, w2, ...w n Composition, then P(W) can be expressed by the conditional probability related formula as follows:

[0104] P(W)=P(w1)P(w2|w1)P(w3|w1,w2)…P(w n |w1,w2,...w n-1 )

[0105] Because the conditions are too long, it becomes difficult to estimate the probability. A common practice is to assume that the probability distribution of each word only depends on the first few words that appear. Such a language model is called an n-gram model.

[0106] In the n-gram model, the probability distribution of each word depends only on the previous n-1 words. For example, in the trigram (n is 3) model, the above formula can be simplified to:

[0107] P(W)=P(w1)P(w2|w1)P(w3|w1,w2)…P(w n |w1,w2,...w n-1 )

[0108] =P(w1)P(w2|w1)P(w3|w1,w2)P(w4|w2,w3)P(w5|w3,w4)…P(w n |w n-2 , wn-1 )

[0109] The "switching" operation of the operation ticket refers to a management and supervision tool required for equipment inspection and maintenance during the industrial production process. The preparation and implementation of the operation ticket is an important means to ensure industrial production safety.

[0110] The preparation of the operation ticket requires a text description. For the personnel who perform the switching operation, they need to be trained before they can understand the meaning of the operation ticket. However, in the actual production environment, due to the high workload and the large number of operators, it is difficult to verify the signature of the operation ticket and the consistency of the operation task with the dispatching instruction. In addition, there are many problems in filling out the operation ticket, such as incomplete filling, unclear ticket, and safety measures that do not meet the requirements.

[0111] To solve the above problems, the power system operation ticket voice command recognition technology came into being. This technology can convert the switching instructions in the operation ticket into voice through voice recognition technology, so as to verify whether the switching operation task is consistent with the dispatching instruction, whether the operation ticket is filled out correctly and completely, the ticket is clear, and whether the listed safety measures meet the requirements of the work ticket. In addition, the power system voice command recognition technology can also verify whether the issuer, recipient, operation start and end time on the switching operation ticket are consistent with the actual time, thereby improving the execution efficiency and accuracy of the operation ticket.

[0112] For switching operations, the power system operation ticket voice command recognition is used to assist operators in operating the switching operation ticket operation instructions. Verify and check the voice instructions in the process of executing the switching operation ticket. In order to further improve the accuracy of voice instructions and reduce the probability of misoperation, prompt or alarm the wrong voice instructions to ensure the correctness of the switching operation. Use voice recognition technology to enable the operator to repeat the operation ticket instructions twice. Only when the two repetitions are successful can the relevant instructions of the operation ticket be executed. The power system operation ticket voice command recognition needs to be applied to the relevant operation ticket system. The application software uses the work ticket for entering the switching operation as the interface, and the system needs to recognize the operation steps in the work ticket. In the process of executing the switching operation, the power system operation ticket voice command recognition needs to identify the corresponding voice operation instructions according to the steps and processes, provide the correctness verification of the voice instructions for the switching operation process, and provide corresponding process guidance and instructions to ensure that the staff's switching operation steps are consistent with the work ticket.

[0113] Specifically, Figure 5 As shown in the figure, the electric system operation instruction ticket is identified from the substation switching operation ticket, and the identified substation switching operation ticket is sent to the mobile terminal. The staff operates the mobile terminal to select the matching substation switching operation ticket, and reads the operation items according to the content of the substation switching operation ticket according to the assessment and verification steps. The software verifies the information read by the staff. If the verification passes, the auditor needs to perform a second reading verification. After the two people pass the verification, jump to the next operation item; if an error occurs in the reading, the software needs to actively prompt the user that the reading content does not match the operation item, issue a warning, and remind the staff to pay attention to the operation.

[0114] The voice recognition operation is triggered by the interaction between the system and the user to verify the operation items of the substation switching operation ticket after the user selects the substation switching operation ticket and confirms the verification of the operation items in the selected substation switching operation ticket.

[0115] For speech recognition, that is, decoding the audio file, and then matching the pronunciation dictionary, language model, etc., and finally the one that meets the matching conditions will be output as the result.

[0116] Speech recognition mechanism:

[0117] The voice recognition operation is triggered by the interaction between the system and the user to verify the operation items of the substation switching operation ticket after the user selects the substation switching operation ticket and confirms the verification of the operation items in the selected substation switching operation ticket.

[0118] For speech recognition, that is, decoding the audio file, and then matching the pronunciation dictionary, language model, etc., and finally the one that meets the matching conditions will be output as the result.

[0119] For recognition, it is necessary to accurately identify the words read by the user and then match them with the operation items on the substation switching operation ticket. For the machine learning algorithm mentioned in the project, the software will establish a semantic library dedicated to the user's working environment based on the results of the speech recognition algorithm. The number of uses, calls, verifications, etc. can represent the frequent use of the calibration, that is, put this phrase into the semantic library, and the phrase in the semantic library can be matched first during the operation of the speech recognition function.

[0120] like Figure 6 As shown in , for recognition, it is necessary to accurately recognize the words read by the user and then match them with the operation items on the substation switching operation ticket. For the machine learning algorithm (or machine learning method) mentioned in this technical solution, the software will establish a set of semantic libraries dedicated to the user's working environment based on the results of the speech recognition algorithm. The number of uses, calls, verifications, etc. can represent the frequent use of the calibration, that is, put this phrase into the semantic library, and the phrase in the semantic library can be matched preferentially during the operation of the speech recognition function.

[0121] Specifically, in the technical solution of the present invention, the speech recognition method includes the following steps:

[0122] 1) Obtain the voice of the substation switching operation ticket and upload it to Tencent Cloud online voice recognition.

[0123] 2) Tencent Cloud online speech recognition succeeds, and the callback returns the speech recognition result.

[0124] 3) Determine whether the voice recognition result is consistent with the content of the substation switching operation ticket operation instruction; if the content is consistent, return the executable substation switching operation ticket operation instruction; if the content is inconsistent, mark the substation switching operation ticket operation instruction content of the voice recognition, and re-upload the operation voice instruction content.

[0125] like Figure 7 As shown in the figure, the software architecture of the technical solution of the present invention adopts the development technology webapp that can be loaded on the mobile terminal for the actual working environment. Its core functions for the business layer are: 1. Obtaining the substation switching operation ticket; 2. Function of verifying the steps of the operation items; 3. Function of verifying the operation items by voice recognition; 4. Function of warning prompt when the content read aloud by the user is inconsistent with the name of the item on the substation switching operation ticket.

[0126] The display layer of the system architecture adopts an APP program to provide staff with a more intuitive interactive page; the business layer implements data processing, processing, and other business-level operations; the service layer provides an interface to realize data collection to the server and other services; the data layer uses the MySQL database as the underlying library to configure the configuration file; the basic layer is the information operating system, network services, etc. that support the entire project.

[0127] The control process of speech recognition is as follows Figure 8 As shown in:

[0128] 1) The PC sends the identified substation switching operation ticket file data to the server / mobile terminal;

[0129] 2) After the mobile terminal obtains the data, it performs a step verification on it;

[0130] 3) Manually determine whether the step verification is consistent with the original verification sequence. If it is confirmed, the next step is to select the substation switching operation ticket. If it is not sure, manual step deployment can be carried out;

[0131] 4) After selecting the substation switching operation ticket, the user needs to read aloud the first item of the operation;

[0132] 5) The program determines whether the text read by the user is consistent with the text corresponding to the serial number on the substation switching operation ticket; if they are consistent, the next operation is executed and the auditor reads again to make a judgment; if they are inconsistent, the program immediately prompts the user that the content read aloud does not match the operation item and asks the user to check and read again;

[0133] 6) After one operation item is matched, the entire substation switching operation ticket items are traversed, and the user reads, identifies and matches them one by one until the entire substation switching operation ticket is read and the operation is completed. The program can be exited or other substation switching operation tickets can be selected again.

[0134] The PC end transfers files to the mobile end, including establishing SFTP communication, uploading files from the PC end to the server for storage, and obtaining the switching operation ticket file from the mobile end to the server:

[0135] a) Get the transfer file input stream object

[0136] b) Read the incoming stream data of the transferred file

[0137] c) Upload to the cloud server using FTP tool.

[0138] The implementation methods of switching operation ticket-sequencing include:

[0139] (2.1) Query the database to obtain the initial switching operation ticket number;

[0140] (2.2) Sorting method of switching operation ticket number:

[0141] (2.2.1) System sorting:

[0142] The switching operation tickets are sorted according to the order of the switching operation ticket information storage database, and the switching operation ticket serial number is determined whether it is sorted from small to large or is empty. If the storage order does not correspond to the serial number or the serial number is empty, the serial number is modified according to the switching operation ticket number, otherwise no modification is required.

[0143] (2.2.2) Manual sorting:

[0144] The sorted serial number passed by the method is compared with the initial serial number. If there is a change in the switching operation ticket serial number, the serial number is modified according to the switching operation ticket number, otherwise no modification is required.

[0145] (Note: Sorting method, 0 is system sorting, 1 is manual sorting)

[0146] Switching Operation Ticket - Selected implementation methods include:

[0147] (2.1) First, check whether there is a selected switching operation ticket based on the selected switching operation ticket ID. If there is, first modify the previously selected switching operation ticket ID to 0. Otherwise, no modification is required.

[0148] (2.2) According to the selected switching operation ticket number passed by the method, query the selected switching operation ticket information, and then modify the selected switching operation ticket identifier to 1 according to the selected switching operation ticket number.

[0149] (Note: Select the switch operation ticket ID, 0 is not selected, 1 is selected)

[0150] The implementation methods of the switching operation ticket-preview diagram include:

[0151] (a) Switching operation ticket - operation items - content review and sequence review;

[0152] (b) Error information detection: sequence and content.

[0153] The implementation methods of switching operation ticket-command voice recognition review include:

[0154] (2.1) First retelling:

[0155] a. Obtain a passage of language read aloud by the user (switching operation ticket instruction voice), upload the voice file for voice recognition, and if the voice recognition is successful and the voice recognition result is returned, the operation instruction item will be checked and reviewed; if the voice instruction recognition fails, the voice instruction will be repeated again.

[0156] b. Compare the operation instruction items of the switching operation ticket with the switching operation ticket instructions recognized by voice recognition. If the verification is successful, the information read aloud by the user is consistent with the items of the substation switching operation ticket, and a second retelling can be performed; if the verification fails, mark the unmatched voice content information and save it to the database, and then repeat the voice instruction again.

[0157] (2.2) Second retelling:

[0158] a. Obtain a passage of language read aloud by the user (the voice of the switching operation ticket instruction), upload the voice file for voice recognition, and if the voice recognition is successful and the voice recognition result is returned, the operation instruction item will be checked and reviewed; if the voice instruction recognition fails, the voice instruction will be repeated again.

[0159] b. Compare the operation instruction items of the switching operation ticket with the switching operation ticket instructions recognized by voice. If the verification is successful, the information read aloud by the user is consistent with the items of the substation switching operation ticket, and the switching operation ticket instruction items can be executed; if the verification fails, mark the unmatched voice content information and save it to the database, and then re-upload the switching operation ticket voice instructions for voice recognition.

[0160] (2.3) After two repetitions and verifications are successful, execute the switching operation ticket operation instructions.

[0161] The implementation methods of switching operation ticket-review-check include:

[0162] (1) User operation - Read - Switching operation ticket - Operation item - Information, then tick after the sequence number;

[0163] (2) User Operation - Voice Review Success - Switching Operation Ticket - Operation Item - Information, then check the corresponding checkbox;

[0164] (3) Return the switching operation ticket word document data.

[0165] Implementation method of switching operation ticket-equipment-QR code-recognition:

[0166] Scan the device QR code to confirm whether the switching operation device corresponds to the device on the operation ticket.

[0167] The implementation methods of switching operation ticket-review-information-entry include:

[0168] (1) Obtain user operation-switching operation ticket-operation-information, such as issuer, inspector, etc.;

[0169] (2) Enter the switching operation ticket-operation-information into a word document;

[0170] (3) Return the switching operation ticket word document data.

[0171] Furthermore, before performing voice recognition on the device, it is necessary to scan the operating device for identification. If the device information data parsed through the QR code on the device is consistent with the device information data on the operation ticket, it means that the operating device is correct and the operation can be performed; if the data recognized through the QR code is inconsistent, it means that the operating device is not the same device and needs to be rescanned. Only when the device information is consistent can the voice recognition review work be performed on this operation ticket.

[0172] The innovation of this technical solution is:

[0173] 1. Preprocess the voice commands of the power system to make speech recognition more suitable for the power system: In the voice command recognition algorithm of the power system, speech signal preprocessing is very important. It is necessary to collect and preprocess the speech signal to facilitate the subsequent acoustic feature extraction and feature processing;

[0174] This technical solution firstly ensures the high quality and good clarity of the collected voice signal through professional recording equipment; the preprocessing of the voice signal includes noise removal, signal enhancement, voice segmentation, etc., and the preprocessed voice signal is normalized, and the amplitude value of the preprocessed voice signal is scaled to a fixed range, so that signals of different amplitudes can be compared and processed.

[0175] In addition, the speech signal is segmented and divided into meaningful speech segments to facilitate subsequent feature extraction and recognition.

[0176] Based on the above preprocessing operations, this method is more suitable for the complex operating environment of the power system and can more accurately identify professional power system voice command information.

[0177] 2. This technical solution adopts acoustic feature extraction technology: Acoustic feature extraction is a key link in speech recognition, and its purpose is to extract information useful for speech recognition from acoustic signals.

[0178] Commonly used acoustic features include short-time energy, short-time average amplitude difference, Mel-scale Frequency Cepstral Coefficients (MFCC), linear predictive coding (LPC), etc. The speech recognition algorithm of this technical solution adopts MFCC acoustic feature extraction technology; first, in order to strengthen the high-frequency components, pre-emphasis processing is required; then the signal is divided into several sub-segments with a length of the frame length, and a windowing function is applied to each sub-segment to reduce the influence of spectrum leakage; then FFT calculation is performed to convert the time domain signal into a frequency domain signal; in order to be more in line with the auditory characteristics of the human ear, a Mel filter bank is used for filtering processing to convert the spectrum into a Mel spectrum; finally, the Mel spectrum is subjected to DCT transformation (Discrete Cosine Transform), and the first N coefficients are taken as MFCC features.

[0179] The technical solution of the present invention provides a voice command recognition algorithm based on a Gaussian mixture model, and adopts a support vector machine classifier for classification. By performing voice command recognition in a power system voice control scenario, the recognition accuracy, real-time and robustness are improved, and a voice command recognition algorithm based on a Gaussian mixture model is realized. The algorithm has high recognition accuracy and real-time performance in the power system, good applicability and robustness, and can meet the requirements of voice command recognition in complex environments in the power system.

[0180] The present invention can be widely used in the field of voice command recognition for power system operation, operation and maintenance management, and dispatching management.

Claims

1. A method for identifying an electric system operation instruction ticket based on secondary voice verification, comprising receiving a substation switching operation ticket at a mobile terminal, characterized in that The following steps are involved: 1) The PC sends the identified substation switching operation ticket file data to the server / mobile terminal; 2) After the mobile terminal obtains the data, it performs a step verification on it; 3) Determine whether the step verification is consistent with the original verification sequence. If it is confirmed, select the substation switching operation ticket for the next step. If it is not confirmed, perform manual step deployment; 4) After selecting the substation switching operation ticket, the user needs to read aloud the first item of the operation; 5) The program determines whether the text read aloud by the user is consistent with the text corresponding to the serial number on the substation switching operation ticket; if they are consistent, the next step is executed, and the auditor reads aloud again to make a judgment; if they are inconsistent, the user is immediately prompted "the content read aloud does not match the operation item, please check and read aloud again"; 6) After one operation item is matched, the entire substation switching operation ticket items are traversed, and the user reads, identifies and matches them one by one until the entire substation switching operation ticket is read and the operation is completed. The user can exit the program or select another substation switching operation ticket again; 7) Through the secondary voice verification mode, the voice content recognition and review function is completed; 8) The voice command recognition of the power system operation ticket is used to assist the operator in operating the switching operation ticket operation instructions, verify the voice commands in the process of executing the switching operation ticket, so as to realize the voice command recognition based on the Gaussian mixture model and meet the requirements of voice command recognition in complex environments in the power system.

2. The method for identifying an electric system operation instruction ticket based on secondary voice verification according to claim 1, characterized in that In the step 1), the Scoket mode is adopted to establish communication, and the output stream of the communication is used to send the substation switching operation ticket file to the mobile terminal.

3. The method for recognizing electric system operation instruction tickets based on secondary voice verification according to claim 2, characterized in that In the step 1), SFTP communication is established, the PC uploads the file to the server for storage, and the mobile terminal obtains the switching operation ticket file from the server.

4. The method for recognizing electric system operation instruction tickets based on secondary voice verification according to claim 3, characterized in that In the step 1), SFTP communication is established, the PC uploads the file to the server for storage, and the mobile terminal obtains the switching operation ticket file from the server, which at least includes the following steps: a) Get the transfer file input stream object; b) reading the incoming stream data of the transmission file; c) Upload to the cloud server using FTP tool.

5. The method for recognizing electric operation instruction tickets based on secondary voice verification according to claim 1, characterized in that In the step 3), if it is not possible to determine whether the step verification is consistent with the original verification sequence, then manually review whether the operation steps of the substation switching operation ticket are correct, and reselect the switching operation ticket.

6. The method for recognizing electric system operation instruction tickets based on secondary voice verification according to claim 1, characterized in that In the steps 4) and 5), the system obtains a passage of language read aloud by the user, obtains the language instructions of the substation switching operation ticket, and uploads them to the Tencent Cloud server for online voice recognition; the system calls back the online voice recognition results of Tencent Cloud to determine whether the voice recognition results are consistent with the content of the corresponding operation items of the substation switching operation ticket.

7. The method for recognizing electric system operation instruction tickets based on secondary voice verification according to claim 1, characterized in that In the above 5), if it is determined that the voice recognition result is inconsistent with the content of the corresponding operation item of the substation switching operation ticket, the system issues an "error prompt" to indicate which line has a problem and marks the content of the substation switching operation ticket operation instruction of the voice recognition; If it is judged that the voice recognition result is consistent with the content of the corresponding operation item of the substation switching operation ticket, the voice match of the single substation switching operation ticket is complete, and the system returns to the step of "selecting the substation switching operation ticket" again.

8. The method for identifying an electric operation instruction ticket based on secondary voice verification according to claim 1, characterized in that The voice content recognition and review function includes converting a voice signal into corresponding text information; the system includes at least four parts: feature extraction, acoustic model, language model, and dictionary and decoding; the collected sound signal is preprocessed including filtering and framing to extract the signal to be analyzed from the original signal; then, the feature extraction work converts the sound signal from the time domain to the frequency domain to provide a suitable feature vector for the acoustic model; the acoustic model then calculates the score of each feature vector on the acoustic feature based on the acoustic characteristic; and the language model calculates the probability of the sound signal corresponding to a possible phrase sequence based on relevant linguistic theories; finally, the phrase sequence is decoded according to the existing dictionary to obtain the final possible text representation.

9. The method for recognizing electric operation instruction tickets based on secondary voice verification according to claim 8, characterized in that The relationship between the acoustic model and the language model is expressed by the Bayesian formula: Among them, P(X|W) is called the acoustic model, and P(W) is called the language model.

10. The method for identifying an electric system operation instruction ticket based on secondary voice verification according to claim 8, characterized in that The language model is used to process the text sequence, and combined with the output of the acoustic model, a text sequence with the highest probability is given as a speech recognition result.

Images