Test instrument voice instruction recognition method based on multi-model fusion

Through multi-model fusion and feature library construction, the problem of difficulty in identifying stability of speech recognition technology and keyword extraction in the test instrument is solved, and more efficient and accurate voice command recognition is achieved. It is suitable for different test instruments and improves recognition speed and accuracy.

CN119007712APending Publication Date: 2024-11-22THE 41ST INST OF CHINA ELECTRONICS TECH GRP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410994982.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing speech recognition technology has problems such as inability to accurately identify pronunciation of specific fields and confusion of pronunciation in test and measurement instruments, resulting in insufficient recognition stability, difficulty in extracting keywords and high recognition ambiguity, which limits its application.

Method used

Using a multi-model fusion method for voice command recognition of test instruments, including speech recognition, keyword extraction, text feature extraction and semantic similarity matching models, to build a dedicated instruction feature library for test instruments, and use the Paraformer model to add time stamp prediction and hot word customization. Ability, combining the HMM model to perform word segmentation and semantic feature calculations, and realize accurate matching and recognition of voice commands.

Benefits of technology

Improve the accuracy and robustness of the voice command recognition of the test instrument, solve the problems of keyword extraction and semantic feature matching in specific fields, and achieve faster and more accurate speech recognition, which is suitable for the deployment of different test instruments, improving the flexibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007712A_ABST
    Figure CN119007712A_ABST
Patent Text Reader

Abstract

The invention discloses a test instrument voice instruction recognition method based on multi-model fusion, and belongs to the field of voice recognition. According to the method, researches in the aspects of control instruction voice recognition, semantic similarity calculation, statement keyword extraction and the like are developed, a special instruction feature library for a test and measurement instrument is constructed, accurate matching recognition of test instrument instructions is realized, and a foundation is laid for future intelligent instrument development; according to the scheme of the invention, a timestamp prediction function and a hot word customization capability are added on the basis of a standard Paraformer model, the recognition accuracy is further improved, and the problem of robust speech recognition of a speech instruction of a test instrument is solved; the keyword extraction model solves the problem of quantitative numerical value and key parameter extraction in the voice instruction of the test instrument; the semantic feature similarity matching model solves the problem of accurately issuing the corresponding program control code after the test instruction is identified. According to the method, the recognition speed is higher, the accuracy is better, meanwhile, the voice recognition scheme can be deployed for different test instruments, and the flexibility is higher.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of speech recognition, and in particular relates to a test instrument speech instruction recognition method based on multi-model fusion. Background Art

[0002] With the development of cutting-edge science and technology and high-risk engineering, the requirements for the intelligence of electronic measurement technology are constantly increasing. Using artificial intelligence technology to empower traditional test and measurement instruments, improve the intelligence of instruments, and expand the intelligent interaction capabilities of instruments is a key step in the development of intelligent instruments. At present, the interactive interface of instrument operation mainly has two forms: panel keyboard operation and outsourced computer operation. It is necessary to explore new methods of interaction between people and instruments. The emergence of intelligent speech recognition technology has become a link for people and instruments to use language communication, realize human-computer voice command recognition, make some originally complex and tedious tasks simple, and greatly simplify the steps of instrument use. The test instrument voice command recognition technology based on multi-model fusion lays the foundation for the development of future intelligent instruments.

[0003] At present, speech recognition technology is usually based on experimental environment. When it comes to application-oriented implementation, especially in the recognition of test and measurement instrument instructions, there are problems such as inability to accurately recognize specific field proper nouns and confusion of similar-sounding words. These problems greatly limit the application of speech recognition technology in test and measurement instruments. The present invention takes spectrum measurement instruments as typical verification objects, integrates speech recognition technology to carry out models such as test instrument instruction speech recognition, semantic similarity calculation, and semantic keyword extraction, and realizes the intelligent recognition function of test instrument voice instructions.

[0004] Patent "A voice control method and system based on program-controlled instructions" (CN202111129109.1), the invention provides a voice control method and system based on program-controlled instructions, the method described includes obtaining user voice, voice recognition and control program to complete the conversion of recognition results to SCPI instructions by SCPI instruction mapping program; instrument control is completed according to SCPI instructions. The present invention adopts voice recognition technology, has excellent compatibility and flexibility, and can be applied to signal / spectrum analyzers, signal analysis software and any test instruments and equipment compatible with SCPI program-controlled instructions.

[0005] The patent "A measuring instrument test system and method based on voice control" (CN201911088746.1) discloses a measuring instrument test system and method based on voice control, including: a voice recognition model client, the voice recognition model communicates with the test instrument through the client, the client receives the command of the voice recognition model and parses it, and then configures the corresponding test instrument to perform related operations. The measurement results are displayed through the client and sent to the voice recognition model, and the voice recognition model synthesizes and broadcasts the feedback results and performs the next command operation. Based on the voice control design of the measuring instrument, the voice interactive configuration and use of the test instrument can be realized, and the voice guidance of the test business logic can be realized to replace manual operation.

[0006] At present, speech recognition technology is usually based on experimental environment. When it comes to application implementation, especially in test and measurement instruments, it has shortcomings such as inability to accurately identify specific domain proper nouns and confusion of similar-sounding words. These shortcomings greatly limit the application of speech recognition technology in test and measurement instruments, as shown in the following aspects:

[0007] First, the stability of existing speech recognition technology cannot meet the needs of voice command recognition of test instruments. The accuracy of speech recognition under ideal conditions has reached a certain level, but in some complex speech environments, speech recognition tasks are difficult to complete;

[0008] Second, the keywords of test instruments are highly similar and data resources are scarce. Acoustic modeling needs to use insufficient data resources to train and obtain as many acoustic features as possible. It is urgent to build a corpus and sentence feature library for the test instrument field;

[0009] Third, the current speech recognition technology has the defect of recognition ambiguity. There are words with similar pronunciations in various command statements of the test instrument. The voice interaction commands of the test instrument contain a large number of professional nouns, floating point decimal type words, and unit nouns, making it difficult to build a model for a single accent. For this reason, it is urgent to build a keyword extraction model in the field of test instruments, build a keyword library dedicated to test instruments, and combine the acoustic model and keyword recognition classifier through multi-task learning to accurately identify keywords in the test field and extract key parameters. Summary of the invention

[0010] In view of the above-mentioned technical problems existing in the prior art, the present invention proposes a test instrument voice command recognition method based on multi-model fusion, which has a reasonable design, overcomes the shortcomings of the prior art, and has good effects.

[0011] In order to achieve the above object, the present invention adopts the following technical solution:

[0012] A test instrument voice command recognition method based on multi-model fusion includes the following steps:

[0013] Step 1: Voice input;

[0014] Step 2: Speech preprocessing;

[0015] Step 3: Voice recognition;

[0016] Perform speech recognition through a speech recognition model;

[0017] Step 4: Keyword extraction;

[0018] Extract key parameters of instructions through keyword extraction model;

[0019] Step 5: Text feature extraction;

[0020] Extract text features through a text feature extraction model;

[0021] Step 6: semantic similarity matching;

[0022] The semantic similarity of the instruction text is calculated through the semantic similarity matching model to perform semantic similarity matching;

[0023] Step 7: Program control instruction output.

[0024] Preferably, the construction of the speech recognition model in step 3 specifically includes the following steps:

[0025] Step S01: voice database;

[0026] Step S02: feature extraction;

[0027] Step S03: Acoustic model training.

[0028] Preferably, step 3 specifically includes the following steps:

[0029] Step 3.1: Voice message queue;

[0030] Step 3.2: Real-time monitoring of endpoints;

[0031] Step 3.3: Real-time speech recognition;

[0032] Timestamp prediction and hot word customization;

[0033] Step 3.4: Get the text result message.

[0034] Preferably, in step 3, the speech recognition model includes five modules, namely, an encoder, a predictor, a sampler, a decoder and a loss function.

[0035] Preferably, the construction of the text feature extraction model in step 5 specifically includes the following steps:

[0036] Step S11: program control command text data;

[0037] Step S12: feature extraction;

[0038] Step S13: Obtain text features.

[0039] Preferably, step 5 specifically includes the following steps:

[0040] Step 5.1: Keyword extraction;

[0041] Step 5.2: Extract keywords to form a parameter list.

[0042] Preferably, the construction of the feature matching model in step 6 specifically includes the following steps:

[0043] Step S21: loading feature database;

[0044] Step S22: instruction similarity matching;

[0045] Step S23: extract the instruction with the highest confidence.

[0046] Preferably, step 6 specifically includes the following steps:

[0047] Step 6.1: Load the program control instruction library;

[0048] Step 6.2: Query the program control code according to the matched instruction;

[0049] Step 6.3: Fill the keyword parameter list into the program code.

[0050] Beneficial technical effects brought by the present invention:

[0051] The present invention proposes a test instrument voice command recognition technology based on multi-model fusion, carries out research on control command voice recognition, semantic similarity calculation, sentence keyword extraction, etc., builds a special command feature library for test and measurement instruments, realizes accurate matching and recognition of test instrument commands, and lays the foundation for the development of future intelligent instruments; the scheme of the present invention adds timestamp prediction function and hot word customization ability on the basis of the standard Paraformer model, further improves the recognition accuracy, and solves the problem of robust voice recognition of test instrument voice commands; the keyword extraction model solves the problem of quantitative numerical value and key parameter extraction in test instrument voice commands; the semantic feature similarity matching model solves the problem of accurately issuing corresponding program control codes after test command recognition. The best existing technical solution focuses on the construction of the entire process framework. The present invention applies for research based on the test instrument voice recognition, realizes the perfect integration of multiple current mainstream voice recognition models, realizes the recognition of test instrument commands from the semantic level, and has faster recognition speed and better accuracy. At the same time, the voice recognition solution can be deployed for different test instruments, with higher flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of a multi-model fusion speech recognition method;

[0053] Figure 2 This is a flow chart of the speech recognition model;

[0054] Figure 3 This is the structural diagram of the speech recognition model;

[0055] Figure 4 Flow analysis diagram for keyword extraction model;

[0056] Figure 5 It is a schematic diagram of the word segmentation method based on the HMM model;

[0057] Figure 6 Flowchart for semantic feature calculation and similarity matching;

[0058] Figure 7 A general roadmap for multi-model testing instrumented speech recognition methods. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0060] The present invention uses multi-model fusion intelligent speech recognition technology to empower test and measurement instruments, conducts research on new technologies and methods such as speech recognition, semantic similarity calculation, sentence keyword extraction, etc. for test and measurement instruments, and builds a special vocabulary library and a semantic feature library for test instruments. With spectrum analyzers as typical verification objects, intelligent speech interaction technology as knowledge drive, and machine self-learning, induction, and reasoning, the machine decision-making process at the artificial intelligence level is completed, providing an efficient, direct, and universal new model for intelligent testing work. The technical solution of the present invention is as follows: Figure 1 shown.

[0061] 1. Test instrument command speech recognition model

[0062] The accurate recognition of speech is the basis for accurate recognition of test instrument instructions. The present invention trains speech recognition models based on large-scale industrial corpora, completes the migration training of special speech recognition models, and can deploy them to applications. The speech recognition model process is as follows: Figure 2 shown.

[0063] The core model of the speech recognition function is Paraformer, a non-autoregressive end-to-end speech recognition model trained on a manually annotated Mandarin speech recognition dataset containing 60,000 hours of speech data. Paraformer is the first model that can match the performance of autoregressive models and has a 10x improvement in inference speed over autoregressive models on large-scale datasets. The Paraformer model framework consists of five modules: encoder, predictor, sampler, decoder, and loss function. The structure is as follows: Figure 3 shown.

[0064] A total of three losses, MWER, MAE and CE, are defined, and joint training is performed during training:

[0065]

[0066] The predictor consists of two convolutional layers, which predict the length of the output label by accumulating weights. The MAE loss is defined as:

[0067]

[0068] For MWER, the specific calculation formula is as follows:

[0069]

[0070] During inference, the sampler module is inactive and the bidirectional parallel decoder only uses the acoustic vector E aThe final prediction result is output through one round of latent vector H. Although the decoder performs forward inference twice during each training, the computational complexity is not increased because it is only performed once during inference. At the same time, in order to improve the performance of Paraformer, the speech recognition model of the present invention adds timestamp prediction and hot word customization capabilities on the basis of the standard Paraformer, further improving the recognition accuracy.

[0071] 2. Keyword extraction model in the field of testing instruments

[0072] There is no space between words in Chinese to indicate word boundaries, so Chinese word segmentation is the basic module and the first step in many natural language processing systems. After completing the first step of speech recognition to obtain the voice command text, sentence keyword extraction becomes the key. The keyword extraction process is as follows: Figure 4 shown.

[0073] In the keyword extraction process, the HMM-based word segmentation method belongs to the word segmentation method based on character formation, which converts the word segmentation problem into a character classification problem (sequence labeling problem). The word formation method based on characters does not rely on a pre-compiled vocabulary, but still requires a training corpus of segmented words. The five-tuple in the HMM model represents: observation sequence, hidden state sequence, state initial probability, state transition probability, state emission probability, and the model is expressed as:

[0074]

[0075] Where X is the observation sequence, Y is the hidden state sequence, P(y0) is the initial probability of the state, and P(y t |y t-1 ) is the state transition probability, P(x t |y t ) is the state emission probability.

[0076] There are three problems in the use of HMM model: probability calculation, model learning, and predictive decoding. The five-tuple of HMM, calculate the probability of the occurrence of the observation sequence X given the hidden sequence Y under the model; the observation sequence {X}, the hidden sequence {Y} are known, and the model state initial probability, state transition probability and state emission probability are estimated to make the probability of the observation sequence X as large as possible under the model; the model state initial probability, state transition probability, state emission probability and the observation sequence X are known, and the maximum probability hidden sequence Y is solved. The word segmentation method based on the HMM model is as follows Figure 5 shown.

[0077] 3. Semantic feature calculation and semantic feature similarity matching model

[0078] The semantic similarity calculation method has a wide range of applications in natural language processing fields such as information retrieval, word sense disambiguation, and machine translation. The present invention calculates the semantic features of all program-controlled instructions of the spectrum analyzer to form a semantic feature library file of the program-controlled instructions of the spectrum analyzer. In the process of speech recognition, the sound instruction features of real-time speech recognition are matched with the library file, and the program-controlled instruction output with the maximum confidence is screened out. The semantic feature calculation and similarity matching process is as follows: Figure 6 shown.

[0079] The text content of sentences recognized by the speech recognition model is used to extract and match the semantic features of control instructions, mainly based on distance and content.

[0080] (1) Distance-based semantic similarity calculation

[0081] The distance-based semantic similarity calculation method is to measure the position of two concept nodes in the ontology hierarchy tree and reflect the difference in the form of path length. The shorter the path, the greater the similarity; the longer the path, the smaller the similarity. The distance between two concept words in the ontology hierarchy is used to represent the semantic distance between them. The calculation formula is as follows:

[0082]

[0083] (2) Content-based semantic similarity calculation

[0084] The content-based method combines the calculation of information entropy with ontology relationships. The basic idea is: the more information is shared between concepts, the smaller the entropy and the greater the similarity. The shared information content is expressed by calculating the amount of information of the common parent node. Because in the ontology, the child node is often a refinement of the parent node on the previous level, the information content of any child node in the entire tree structure can reflect the information content of all its ancestor nodes. The calculation formula for the information content of a single concept node is as follows:

[0085]

[0086] The formula for calculating the similarity between any two nodes in the ontology is as follows:

[0087]

[0088] The present invention aims to solve the problem of single interaction mode of test instrument operation and explores new methods of human-instrument interaction. Research on control command voice recognition, semantic similarity calculation, semantic keyword extraction, etc. is carried out to build a special statement instruction feature library for test and measurement instruments. Breakthroughs in key technologies such as accurate recognition of test instrument voice instructions, accurate extraction of key statement parameters, sentence semantic feature similarity matching and special program control framework control are made, laying the foundation for the future intelligent development of test instruments. The overall technical route of the invention is as follows: Figure 7 shown.

[0089] The key points and protection points of the present invention are as follows:

[0090] (1) The present invention proposes a special domain speech recognition method based on small sample learning of test instrument voice commands. By introducing the transfer learning method, after the initial training of the model on the industrial voice data set, the transfer learning test instrument special voice library is trained, and the data expansion is performed for homophones, near-phonetic words, and polysemous words, so as to construct a high-precision speech recognition model for the special domain of intelligent test instruments.

[0091] (2) The present invention proposes a text semantic similarity matching model based on the semantic fusion model of the test instrument instruction. A semantic feature extraction network is designed and developed to extract features from key verb-object structures in sentences, construct a multi-dimensional semantic feature vector, and combine it with a parallel similarity calculation module to efficiently complete the instruction library voice matching.

[0092] (3) The present invention proposes a method for extracting keywords from test instrument voice instructions by word formation and fusion language model. By constructing an efficient keyword retrieval model, the retrieval of keywords in the special field of test instruments is realized, and the accurate extraction of program-controlled input parameters and accurate recognition of program-controlled measurement units are realized, providing necessary data input for the universal platform of intelligent speech recognition.

[0093] (4) The present invention proposes a method for designing a universal platform architecture for intelligent speech recognition of test instruments. According to the program control process of the test instrument, a universal method suitable for speech recognition of the test instrument is summarized, and a speech recognition model dedicated to the test instrument is established. By separating the front and back ends, the modules such as speech recognition, keyword extraction, feature extraction, and feature matching are software-reconfigurable, and a test instrument speech command recognition technology based on multi-model fusion is proposed.

[0094] The present invention proposes a test instrument voice command recognition technology based on multi-model fusion, conducts research on control command voice recognition, semantic similarity calculation, semantic keyword extraction, etc., constructs a special sentence feature library for test and measurement instruments, realizes accurate matching and recognition of test instrument commands, and lays the foundation for the development of future intelligent instruments. This solution adds timestamp prediction function and hot word customization capability on the basis of the standard Paraformer model, further improves the recognition accuracy, and solves the problem of robust voice recognition of test instrument voice commands; the keyword extraction model solves the problem of strong differences in test instrument voice commands and lack of data sample resources; the semantic feature similarity matching model solves the ambiguity problem of test command voice. The best existing technical solution focuses on the construction of the entire process framework. The present invention conducts research on the basis of test instrument voice recognition, realizes the perfect integration of multiple current mainstream voice recognition models, realizes the recognition of test instrument commands from the semantic level, and has faster recognition speed and better accuracy. At the same time, the voice recognition solution can be deployed for different test instruments, with higher flexibility.

[0095] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A test instrument voice command recognition method based on multi-model fusion, characterized by: The steps include: Step 1: Voice input; Step 2: Speech preprocessing; Step 3: Voice recognition; Perform speech recognition through a speech recognition model; Step 4: Keyword extraction; Extract key parameters of instructions through keyword extraction model; Step 5: Text feature extraction; Extract text features through a text feature extraction model; Step 6: semantic similarity matching; The semantic similarity of the instruction text is calculated through the semantic similarity matching model to perform semantic similarity matching; Step 7: Program control instruction output.

2. The test instrument voice command recognition method based on multi-model fusion according to claim 1 is characterized in that: The construction of the speech recognition model in step 3 specifically includes the following steps: Step S01: voice database; Step S02: feature extraction; Step S03: Acoustic model training.

3. The test instrument voice command recognition method based on multi-model fusion according to claim 1, characterized in that: Step 3 specifically includes the following steps: Step 3.1: Voice message queue; Step 3.2: Real-time monitoring of endpoints; Step 3.3: Real-time speech recognition; Timestamp prediction and hot word customization; Step 3.4: Get the text result message.

4. The test instrument voice command recognition method based on multi-model fusion according to claim 1, characterized in that: In step 3, the speech recognition model includes five modules: encoder, predictor, sampler, decoder and loss function.

5. The method for voice command recognition of a test instrument based on multi-model fusion according to claim 1, characterized in that: The construction of the text feature extraction model in step 5 specifically includes the following steps: Step S11: program control command text data; Step S12: feature extraction; Step S13: Obtain text features.

6. The method for voice command recognition of a test instrument based on multi-model fusion according to claim 1, characterized in that: Step 5 specifically includes the following steps: Step 5.1: Keyword extraction; Step 5.2: Extract keywords to form a parameter list.

7. The method for voice command recognition of a test instrument based on multi-model fusion according to claim 1, characterized in that: The construction of the feature matching model in step 6 specifically includes the following steps: Step S21: loading feature database; Step S22: instruction similarity matching; Step S23: extract the instruction with the highest confidence.

8. The method for voice command recognition of a test instrument based on multi-model fusion according to claim 1, characterized in that: Step 6 specifically includes the following steps: Step 6.1: Load the program control instruction library; Step 6.2: Query the program control code according to the matched instruction; Step 6.3: Fill the keyword parameter list into the program code.