Voice instruction recognition and generation system, method and equipment

Abstract

The invention discloses a voice instruction recognition and generation system, method and device, and relates to the technical field of voice recognition and human-computer interaction. The system performs multi-thread concurrent processing through module decoupling and zero copy message communication, and comprises a voice acquisition module, a voice endpoint detection module, a voice recognition module, a text instruction analysis module and an instruction sending module, the voice acquisition module acquires a user voice signal; the voice endpoint detection module judges a voice starting point and a voice ending point and intercepts a voice sequence; the voice recognition module transwrites the voice sequence into a text instruction string; the voice recognition module is compatible with an AIS mode and an ONNX mode at the same time; the text instruction analysis module performs semantic understanding on the text instruction string to generate a control instruction; and the instruction sending module sends out the control instruction. According to the invention, the accuracy, real-time performance and compatibility of voice instruction recognition can be improved.

Description

Technical Field

[0001] This application relates to the fields of speech recognition and human-computer interaction technology, and in particular to a speech command recognition and generation system, method and device. Background Technology

[0002] Voice, as a natural and intuitive means of interaction, has been widely used in human-computer interaction scenarios such as smart homes and in-vehicle voice assistants. A typical voice interaction system usually includes modules such as voice wake-up, speech recognition, natural language understanding (NLU), and command execution. In recent years, the development of deep learning technology has greatly promoted the performance improvement of Automatic Speech Recognition (ASR), enabling end-to-end models to achieve significant results in multilingual and multi-scenario environments. In particular, the emergence of advanced acoustic models such as Transformer and Conformer has improved recognition accuracy in accented and noisy environments. Existing research shows that end-to-end ASR models have gradually surpassed traditional hybrid systems of acoustic and language models, achieving lower error rates in standard evaluations.

[0003] However, several challenges remain in human-computer interaction applications using voice command recognition. First, high real-time performance is required: the system needs to convert user speech into operation commands with the lowest possible latency, but complex models often introduce significant computational overhead and latency. Second, robustness and accuracy are crucial: environmental noise or accents / dialects can affect recognition performance, especially in multi-user or far-field scenarios, leading to misidentification. To improve robustness, some studies have introduced speaker-customized and personalized acoustic models, such as personalized VAD (Voice Activity Detection) or endpoint detection algorithms, to accurately detect the target speaker's speech activity in multi-speaker environments. Third, system integration is complex: the text output by speech recognition often requires further semantic intent parsing before mapping to specific commands, which typically involves natural language understanding techniques such as intent classification and slot extraction. Traditional approaches often employ cascaded ASR+NLU models, resulting in complex inter-module interfaces and a tendency for errors to cascade and amplify. Furthermore, when running independently on devices, lightweight models and framework compatibility must be considered to reduce hardware dependence and ensure real-time operation. Some existing solutions attempt to reduce model size through methods such as model pruning and quantization, enabling deployment on embedded devices. For example, the PersonalVAD2.0 system proposed by Ding et al. uses 8-bit quantization and a streaming Conformer network to reduce the model size by approximately 75%, making it more suitable for device operation while maintaining performance. However, how to further improve real-time performance and compatibility with multiple inference frameworks while ensuring recognition accuracy remains a key challenge for current voice interaction technology. Summary of the Invention

[0004] The purpose of this application is to provide a voice command recognition and generation system, method, and device that can improve the accuracy, real-time performance, and compatibility of voice command recognition.

[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a voice command recognition and generation system. The system performs multi-threaded concurrent processing through module decoupling and zero-copy message communication. The system includes: a voice acquisition module, a voice endpoint detection module, a voice recognition module, a text command parsing module, and a command sending module. The voice acquisition module is used to acquire the user's voice signal; The speech endpoint detection module is used to determine the start and end points of speech and extract speech sequences; The speech recognition module is used to transcribe speech sequences into text command strings; the speech recognition module is compatible with both AIS mode and ONNX mode. The text instruction parsing module is used to understand the semantics of text instruction strings and generate control instructions; The command sending module is used to send out control commands.

[0006] Secondly, this application provides a voice command recognition and generation method, which performs multi-threaded concurrent processing through step decoupling and zero-copy message communication, and the method includes: Acquire user voice signal; Determine the start and end points of the speech and extract the speech sequence; Transcribe speech sequences into text instruction strings; The semantics of the text instruction string are understood to generate control instructions; Send out control commands.

[0007] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the above-described voice command recognition and generation system.

[0008] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a voice command recognition and generation system, method, and device that integrates voice recognition and text command parsing, optimizes the voice command processing flow, and improves the accuracy and real-time performance of voice command processing. The system achieves multi-threaded concurrent processing through module decoupling and zero-copy message communication, and supports dual-framework compatible deployment in AIS mode and ONNX mode, thereby improving the overall recognition efficiency and compatibility of the system. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This application provides a schematic diagram of the functional modules of a voice command recognition and generation system. Figure 2 This is a schematic diagram of the end-to-end closed-loop and decoupling architecture provided in an embodiment of this application; Figure 3 This is a schematic diagram of the VAD dual-threshold process provided in an embodiment of this application; Figure 4 A schematic diagram of the CTC+Attention hybrid architecture of WeNet provided in the embodiments of this application; Figure 5 A breakdown diagram of the total system latency provided in this application embodiment; Figure 6 A flowchart illustrating the voice command recognition and generation method provided in this application embodiment; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] In one exemplary embodiment, such as Figure 1 As shown, a voice command recognition and generation system is provided. This system is run by a computer device, specifically a terminal or server running independently, or both. In this embodiment, the system uses module decoupling and zero-copy message communication for multi-threaded concurrent processing. The system includes: a voice acquisition module, a voice endpoint detection module, a voice recognition module, a text command parsing module, and a command sending module. The voice acquisition module acquires user voice signals; the voice endpoint detection module determines the start and end points of the voice and extracts the voice sequence; the voice recognition module transcribes the voice sequence into a text command string; the voice recognition module is compatible with both AIS and ONNX modes; the text command parsing module performs semantic understanding of the text command string and generates control commands; and the command sending module sends the control commands.

[0014] The processing procedures of the voice acquisition module, voice endpoint detection module, voice recognition module, text command parsing module, and command sending module constitute an end-to-end processing flow from voice input to command output. The system in this application aims to achieve efficient and robust voice command understanding and execution through process optimization and architectural innovation.

[0015] As an optional implementation, the system architecture is optimized through modular decoupling: This application adopts a modular decoupling and multi-threaded parallel system architecture, dividing each functional module in the above process into independent threads or processes, and using ZeroMQ message queues to achieve asynchronous communication and data transfer between modules, thereby improving system throughput and stability. Loosely coupled communication between modules avoids blocking caused by direct function calls, improving parallel efficiency and fault tolerance. Each thread employs a circular buffer and queue mechanism, enabling smooth processing of continuous voice command input in high-concurrency scenarios.

[0016] As an optional implementation, the AIS / ONNX dual inference framework is compatible with lightweight deployment: Considering the different requirements of various application environments for model inference frameworks and hardware, the ASR module of this application is compatible with both AIS chip-accelerated inference and general ONNXRuntime inference modes (i.e., AIS mode and ONNX mode). In AIS mode, the offline model format (.om) of AI-accelerated hardware (such as the Huawei Ascend chip) is utilized, enabling deep learning models to run on embedded devices with extremely low latency. In ONNX mode, the open-source ONNXRuntime is used to execute .onnx models on general CPU / GPU platforms to ensure cross-platform compatibility. Furthermore, this application optimizes thread scheduling and priority allocation for embedded deployment, enabling each module to efficiently utilize multi-core processor resources and achieve low-power real-time speech recognition on the edge.

[0017] As an optional implementation method, such as Figure 2 As shown, in acquiring user voice signals, the voice acquisition module includes: acquiring the user's original voice signal; and performing gain control or noise reduction on the original voice signal to obtain the user's voice signal. Specifically, a high signal-to-noise ratio microphone is used to acquire the user's voice signal, and gain control or noise reduction preprocessing is performed when necessary to improve the effective signal components for subsequent recognition.

[0018] As an optional implementation, Voice Activity Detection (VAD) uses an improved endpoint detection algorithm to determine the start and end points of speech in real time, filtering out silent segments and avoiding truncating valid commands. VAD's speed and accuracy reduce invalid data processing and improve system response speed. The principle and modeling of voice endpoint detection are as follows: Speech endpoint detection (VAD) is used to determine the start and end times of speech signals in an audio stream, and is the first step in speech command processing. The classic VAD method is based on simple features such as Short-Time Energy (STE): it divides continuous audio into frames, calculates the energy of each frame, and compares it with a preset threshold to determine the presence or absence of speech. Frame energy calculation: For a length of... The Frame signal Short-time energy is often defined as the average of the squares of the sampling amplitudes: .

[0019] in This reflects the energy level within a speech frame. If Higher than the set threshold If the value is above a certain threshold, the frame is considered to contain speech; otherwise, it is considered silent. Threshold It can adaptively adjust based on background noise to improve detection accuracy in different environments. In practical applications, a dual-threshold strategy is often used to increase robustness: a lower threshold is set to detect the start of speech and a higher threshold to detect the end of speech, combined with features such as zero-crossing rate to avoid false positives. This dual-threshold VAD process is as follows: Figure 3 As shown, the start and end points of speech segments can still be found relatively reliably even when the background noise is close to the speech energy.

[0020] However, the simple energy method is prone to misclassifying noise as speech in noisy environments. To improve accuracy, modern VADs incorporate deep learning modeling, treating endpoint detection as a frame-level binary classification problem. A typical approach is to first extract a series of features from the audio (such as Mel-frequency cepstral coefficients (MFCC) or short-time Fourier transform spectrum) as feature vectors for each frame. Then by deep neural networks Predict the probability that this frame is speech: .here A value close to 1 indicates a speech frame, and a value close to 0 indicates a non-speech frame. (Model) Structures such as multilayer perceptrons, convolutional neural networks, or long short-term memory networks can be used to model adjacent frames to utilize temporal context information.

[0021] During training, labeled data is used to optimize the model parameters with cross-entropy loss, so that the output probability... As close as possible to the actual annotation After training, the output probabilities for consecutive frames are calculated during inference. Smoothing filters are applied, and a threshold (e.g., 0.5) is set to determine the speech activity region. Deep learning-based VADs can detect subtle differences in energy patterns, achieving higher accuracy than fixed energy threshold methods in complex environments. In summary, the VAD module achieves fast, coarse detection using energy thresholding and can be combined with deep models to further improve the accuracy and robustness of speech endpoint determination.

[0022] As an optional implementation, Automatic Speech Recognition (ASR) uses end-to-end models such as WeNet to transcribe speech into text command strings. This application utilizes the unified streaming / non-streaming model provided by WeNet to achieve low latency while maintaining high recognition accuracy. Considering the characteristics of the Chinese language scenario, a hybrid CTC / Attention architecture can be used, and a custom language model can be loaded to reduce accent ambiguity.

[0023] Automatic Speech Recognition (ASR) Module Principle: Hybrid Decoding of CTC and Attention, detailed as follows: The speech recognition module converts the extracted speech features into a text sequence. This application adopts the Unified Two-Stage (U2) end-to-end architecture from the WeNet toolkit. Its core is to connect the CTC decoder and attention decoder in parallel after a shared encoder, achieving a combination of high-speed decoding and high-precision recognition. The encoder consists of multiple layers of Transformers or Conformers, encoding the acoustic feature sequence of the input speech into a high-dimensional representation. ( (This refers to the number of time frames). The encoder output is simultaneously fed into two decoding branches: CTC decoding and Attention decoding. The first stage uses the CTC (Connected Temporal Classification) decoder for streaming decoding, and the second stage uses the Attention (AED) decoder to re-evaluate candidate results, thus balancing real-time performance and accuracy. The entire model is optimized during training by weighting and combining the CTC loss and attention loss. The WeNet CTC+Attention hybrid architecture is as follows: Figure 4 As shown, the shared encoder is connected to both the CTC decoder and the attention decoder; the shared encoder is used to encode and represent the speech sequence, and simultaneously feeds it to the CTC decoder and the attention decoder for decoding to obtain the text instruction string.

[0024] CTC Principle and Forward Algorithm: The CTC decoder estimates the alignment probability between speech and text sequences without requiring manual frame-by-frame alignment. For a given encoder output sequence... The CTC definition includes the "blank" symbol. The extended label sequence, after inserting blanks, yields a length of... Alignment path CTC assumes that each frame is output independently, and the path... The probability is the product of the output probabilities of each frame: .

[0025] target word sequence The probability of is the sum of the probabilities of all aligned paths: .

[0026] in To remove consecutive repeating symbols and whitespace, the CTC loss is... Since the number of alignment paths is exponential, the forward-backward algorithm is used for efficient computation. .definition In time Frame in extended sequence The Middle Each state (corresponding to) or If the cumulative probability of ) is given, then the initial value for forward recursion is... , ,other Its recursive relation has two cases: if White space or with If they are the same, then: .

[0027] otherwise( Not empty and not equal to ),but: .

[0028] The above dynamic programming is recursively applied to... and (The extended sequence terminator) can be used to obtain... The CTC decoder shares the same encoder representation with the attention decoder during training, allowing them to complement each other's strengths: CTC focuses on fast matching of the overall sequence, providing initial results; the attention decoder focuses on inter-character dependencies, compensating for the shortcomings of CTC's independent frame assumption. During training, a joint loss function is used to weightedly sum the objectives of both decoders. .

[0029] in These are CTC loss and attention model loss, respectively. is a weight hyperparameter used to balance the proportion of the two loss components in the overall objective. Through joint training, the encoder learns feature representations suitable for both frame-independent decoding and sequence-dependent decoding.

[0030] Attention Decoding and Weight Calculation: The introduction of the attention mechanism enables the decoder to align and generate text using relevant information from the encoder's output sequence. WeNet's attention decoder belongs to the Autoregressive Sequence-to-Sequence (AED) model, utilizing the hidden states of previous outputs at each decoding step. With the encoder's implicit representation of all frames Calculate attention weights and then generate the output for the current step. A typical formula for calculating attention weights is as follows: First, define the attention weight for the first... The output symbol and the first Match score represented by the frame encoder: in Content-based functions (such as feedforward networks or dot products) can be used: for example, Bahdanau additive attention employs... Then for each decoding step For all frames Calculate the Softmax value to obtain the normalized attention distribution: .

[0031] In summary, the ASR module utilizes a CTC+Attention hybrid architecture to achieve efficient and reliable end-to-end speech recognition: the CTC branch provides fast and stable alignment decoding, while the Attention branch provides fine-grained context modeling. The two work together to achieve a balance between high accuracy and low latency in real-time applications.

[0032] As an optional implementation, after obtaining the instruction text through speech recognition, its semantics need to be further parsed to generate corresponding operation commands. For the text instruction string output by ASR, semantic understanding is performed, including word segmentation and lexical analysis of Chinese instructions to extract key verbs, objects, etc.; then, intent classification is performed, categorizing the instructions into predefined intent categories (such as controlling devices, querying information, etc.); finally, instruction template matching is performed, matching the parsed intent and parameters to a standardized instruction format. For example, the user's voice "turn on the living room light" can be parsed as {intent: "turn on device", device: "living room light"}, and after matching the template <turn on X>, a control command is generated.

[0033] In terms of understanding the semantics of the text instruction string and generating control instructions, the text instruction parsing module can be replaced by the following steps 101-104: Step 101: Perform word segmentation and feature representation on the text instruction string to obtain a word sequence.

[0034] Word segmentation and feature representation: For languages ​​like Chinese that lack natural delimiters, the first step is to break down continuous sentences into meaningful word sequences. A traditional method is dictionary- and rule-based word segmentation algorithms: utilizing a pre-built dictionary, the optimal sentence segmentation is found through maximum matching or dynamic programming. For example, given a dictionary... and input sequence You can define the segmentation sequence. The score is the sum of the frequencies or probabilities of each word, and the segment with the highest total score is selected as the output. This method is simple and fast, but it relies on dictionary quality and struggles to handle out-of-vocabulary (OV) words. Another advanced approach utilizes the word segmentation mechanism of pre-trained models (such as BERT) or context vectors for word segmentation. BERT uses the WordPiece algorithm to break down rare words into smaller sub-word units, thus alleviating the OV problem while preserving the complete representation of common words. Alternatively, a neural network word segmentation model can be trained: taking the BERT vectors of each character in the sentence as input, the model outputs the probability of segmentation for each character, thereby delineating word boundaries. This BERT-based word segmentation can capture ambiguity using context, improving segmentation accuracy. Regardless of the strategy used, the result of word segmentation is to extract the input text... Convert to word sequence This provides the basic unit for subsequent intent and slot resolution.

[0035] Step 102: Represent the word sequence using Bag-of-Words or TF-IDF, and classify the intent using Softmax regression or SVM classifier to predict the intent category.

[0036] Intent Detection: The goal of intent detection is to determine which predefined intent a user's voice command belongs to, such as "check the weather" or "set an alarm." A common method is to represent the entire sentence as a vector feature and train a multi-class classifier to output the intent label. In traditional implementations, this can be done by analyzing the segmented text... Using Bag-of-Words or TF-IDF representations, the intent category is then predicted using Softmax regression or an SVM classifier. Alternatively, deep learning can be employed, utilizing a pre-trained language model like BERT: special markers "[CLS]" and "[SEP]" are added before and after the sentence, and the resulting data is encoded by BERT to obtain the sentence vector. Then, add a fully connected layer and a Softmax layer as the classifier. The Softmax classification formula is: ; in It is an input statement The vector representation of (e.g., BERT's [CLS] vector). For the number of intent categories, and For the first The weights and biases corresponding to each class. By training to maximize the probability of the correct class or minimize the cross-entropy loss, the above formula can learn the parameters that map different semantics to the corresponding intent classes. During inference, the weights and biases are... As the user intent to be identified.

[0037] Step 103: Based on the word sequence, perform slot extraction to obtain slot-value pairs.

[0038] Slot Filling: Slots typically refer to the parameter information carried in user commands, such as the city name and date in a weather query command, or the song name in a music play command. Slot filling essentially involves identifying several segments from a sentence and assigning them predefined semantic labels, which can be formalized as a sequence labeling problem. Mainstream methods fall into two categories: rule-based entity dictionary matching and statistical model-based sequence labeling. Rule-based and dictionary-based methods: For each slot, a corresponding trigger word list or regular expression pattern is built, and then the sentence is scanned for matching. For example, if there is a city slot "<city>", a city name dictionary can be built-in, and the word segmentation sequence... If a word appears in the list of cities, it is extracted and the slot is labeled "city". This method is simple to implement and easy to incorporate domain knowledge, but its ability to recognize flexible expressions is limited.

[0039] Sequence labeling models treat slot filling as a word-by-word classification task, often using the BIO (Block Identification and Omission) labeling scheme to represent slot boundaries. For example, word sequences... Corresponding slot label sequence , You can train a bidirectional LSTM+CRF or BERT+CRF model: First, obtain the context vector for each word using a bidirectional LSTM or BERT. Then, after linear transformation and softmax, the emission scores of each slot label are obtained. A Conditional Random Field (CRF) layer is then used to decode the optimal label sequence to ensure global validity (e.g., B / I / O order constraints for slot labels). For simplicity, each word can also be directly classified using independent softmax. ; in It is a word eigenvectors, It is a slot label The classification weights.

[0040] In practical systems, intent recognition and slot extraction are often jointly modeled to leverage the correlation between the two. For example, one approach is to share a coding layer to simultaneously predict the entire sentence intent and each word slot label, thus considering the global constraint of intent on slot discrimination. Regardless of whether it's a pipelined or joint model, the output of the semantic parsing module is a structured semantic frame containing the identified intent and its corresponding slot-value pairs.

[0041] Step 104: Based on the predicted intent category and slot-value pair, match the instruction template to generate structured control instructions.

[0042] The three main steps of word segmentation, intent classification, and slot (parameter) extraction can be implemented using rule-based methods or by using machine learning models to improve intelligence and robustness.

[0043] As an optional implementation, instruction generation and transmission involves generating machine-executable instruction messages (such as JSON or binary commands) based on the parsing results. These instructions are then sent to the target device or upper-level control system via a lightweight communication protocol (such as ZeroMQ publish / subscribe mode), ensuring reliable delivery and timely execution.

[0044] As an optional implementation, the sum of the audio acquisition time of the voice acquisition module, the endpoint detection time of the voice endpoint detection module, the recognition processing time of the voice recognition module, the text parsing time of the text command parsing module, and the command generation and issuance time of the command sending module is taken as the total delay of the sequential execution of the modules; the system response time is less than the total delay of the sequential execution of the modules. To measure the efficiency of this application from voice input to command output, the total system delay needs to be modeled. System response time Defined as the time interval from when the user begins speaking to when the system produces an execution result (or feedback), it can be decomposed into the sum of delays in multiple stages, such as... Figure 5 As shown.

[0045] Audio acquisition latency includes latency introduced by microphone hardware and audio buffers. For example, the system may process audio in frames with a frame length of 10ms to 20ms, which introduces a very short initial latency. Additionally, if voice processing is in the cloud, there is also network transmission latency in sending the audio stream to the server.

[0046] Endpoint detection latency: VAD needs to wait for a short period of silence after the speech ends to confirm that the user is no longer speaking. This silence threshold is typically several hundred milliseconds. For example, the system might require at least 0.5 seconds of silence to determine the end of the speech, thus introducing a tail endpoint latency of ~0.5 seconds. If streaming ASR is used, the VAD latency also determines the timing of real-time audio truncation before sending it to recognition; too short a latency may result in truncating the user's last speech, while too long a latency increases unnecessary waiting.

[0047] Recognition processing time: This refers to the time required for the ASR module to extract and decode features from the recording. On one hand, it is related to the length of the input speech: longer speech contains more frames, and the processing time increases approximately linearly. On the other hand, it is related to model complexity and computing power: inference on CPUs using models like the Transformer can take hundreds of milliseconds or even seconds. Recognition time can be significantly reduced through model optimization and hardware acceleration (such as GPUs / dedicated chips).

[0048] Text parsing time: This refers to the processing time for converting ASR output text into structured commands, including steps such as tokenization, intent classification, and slot extraction. Compared to deep ASR, this latency is typically small, in the millisecond range. Even when using models like BERT for intent slot parsing, inference only takes tens of milliseconds due to the short sentences (generally command length <20 characters). Therefore, the NLU parsing in this application can be considered as... The latency in this stage can be further reduced by appropriately simplifying the model or by using parallel computing.

[0049] Command generation and delivery latency: The time it takes to generate and execute device commands based on the parsed results. For local device control, the latency is primarily the application logic processing time, which is typically very short (within tens of milliseconds). If a network call to cloud services or remote devices is required, the latency depends on network round-trip latency and the processing time at the receiving end.

[0050] Adding all the above steps together gives the total system delay. It is important to note that in a streaming architecture, certain stages can be executed in parallel and overlapped, resulting in a perceived latency that is less than the sum of its parts. For example, ASR decoding can output partial results in real time during user speech, and NLU can preprocess parts of the text. However, for the sake of simplicity, we assume that each stage is completed sequentially. This application balances the performance and speed of each module in its design, ensuring that the total response time is kept within an acceptable range for the user (typical target <1 second, where the ideal interaction latency ≈200ms). In summary, the formula provides a quantitative analysis tool for the response time of the voice command system. The optimization of each module can be evaluated within this framework to assess its contribution to the overall latency, thereby guiding system improvements to achieve a smoother voice interaction experience.

[0051] Based on the latest end-to-end Automatic Speech Recognition (ASR) model, this method achieves a complete closed-loop process from voice input to command output. The innovations of this method are: First, it proposes an optimized voice command processing flow, including steps such as voice signal acquisition, endpoint detection (VAD), speech recognition, text command parsing (semantic segmentation, intent classification, and command template matching), and command generation and transmission. Second, in terms of system architecture, it achieves multi-threaded concurrent processing through module decoupling and zero-copy message communication (ZMQ), and supports dual-framework compatible deployment of AIS hardware inference and ONNX model inference to improve the overall recognition efficiency and robustness of the system. Experimental results show that the proposed method can efficiently and accurately convert user speech into structured commands. Compared with existing technologies, this application significantly improves the real-time performance and reliability of voice command recognition, and has broad application prospects in human-computer interaction scenarios such as smart homes and in-vehicle voice control.

[0052] In summary, this application proposes an innovative end-to-end solution to address the challenges of real-time performance, robustness, and system integration in the field of voice command recognition. Key contributions include: combining an improved voice endpoint detection algorithm with a hybrid CTC+Attention architecture ASR model to enhance recognition accuracy and timeliness; introducing a semantic parsing module to accurately understand command intent; and employing a zero-copy parallel architecture with module decoupling to ensure low-latency and stable system operation on terminal devices. This method is particularly suitable for scenarios requiring rapid voice response, such as smart homes and in-vehicle voice assistants, and has significant value and broad application prospects in improving user experience and system reliability.

[0053] This application achieves an efficient closed loop from voice command recognition to generation through process and architectural innovations. Compared with existing technologies, this application has the following advantages: (1) Process optimization improves the accuracy and speed of voice command processing, with clear decoupling and efficient collaboration among each link; (2) Modular architecture enhances the robustness and scalability of the system, and the use of ZMQ concurrent communication significantly reduces latency and blocking risks; (3) Dual-frame compatible design allows the system to be flexibly deployed on the cloud or various hardware platforms, achieving optimal performance on both ordinary CPUs and AI accelerators; (4) Lightweight strategy enables the model to run smoothly on resource-constrained devices, meeting the stringent requirements of real-time human-computer voice interaction. Therefore, this application has significant application value in the fields of smart home voice control, in-vehicle voice assistants, and smart security voice commands.

[0054] Based on the same inventive concept, this application also provides a voice command recognition and generation method executable by the aforementioned voice command recognition and generation system. The solution provided by this method is similar to the implementation scheme described in the above system; therefore, the specific limitations in one or more voice command recognition and generation method embodiments provided below can be found in the limitations of the voice command recognition and generation method system described above, and will not be repeated here.

[0055] In one exemplary embodiment, such as Figure 6 As shown, a voice command recognition and generation method is provided. This method performs multi-threaded concurrent processing through step decoupling and zero-copy message communication. The method includes the following steps 201 to 205.

[0056] Step 201: Obtain the user's voice signal.

[0057] Step 202: Determine the start and end points of the speech and extract the speech sequence.

[0058] Step 203: Transcribe the speech sequence into a text instruction string.

[0059] Step 204: Perform semantic understanding of the text instruction string and generate control instructions.

[0060] Step 205: Send out the control command.

[0061] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores control instructions. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a voice command recognition and generation system.

[0062] Those skilled in the art will understand that Figure 7 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0063] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0064] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A voice command recognition and generation system, characterized by, The system is multi-threaded and concurrent by module decoupling and zero-copy message communication, and comprises a voice collection module, a voice endpoint detection module, a voice recognition module, a text instruction analysis module and an instruction sending module. The voice collection module is used to acquire user voice signals. The voice endpoint detection module is used to determine voice start and end points and intercept voice sequences. The voice recognition module is used to transcribe voice sequences into text instruction strings; the voice recognition module is compatible with both AIS mode and ONNX mode. The text instruction analysis module is used to perform semantic understanding on the text instruction string and generate control instructions. The instruction sending module is used to send the control instructions.

2. The voice command recognition and generation system of claim 1, wherein, The system adopts a system architecture of module decoupling and multi-thread parallelism, and uses a circular buffer and a queue mechanism within each thread; The system uses a ZeroMQ message queue to realize asynchronous communication and data transmission between modules; The modules in the system use loose coupling communication.

3. The voice command recognition and generation system of claim 1, wherein, In terms of acquiring user voice signals, the voice collection module comprises: collecting original voice signals of a user; controlling the gain or reducing noise of the original voice signals to obtain user voice signals.

4. The voice command recognition and generation system of claim 1, wherein, The AIS mode uses an offline model format of AI acceleration hardware to run a deep model on an embedded device; The ONNX mode uses an open source ONNX Runtime to execute an.onnx model on a CPU / GPU platform.

5. The voice command recognition and generation system of claim 1, wherein, The voice recognition module comprises a shared encoder, a CTC decoder and an attention decoder; The shared encoder is connected with the CTC decoder and the attention decoder respectively; The shared encoder is used to encode voice sequences and feed them to the CTC decoder and the attention decoder for decoding to obtain text instruction strings.

6. The voice command recognition and generation system of claim 1, wherein, In terms of performing semantic understanding on the text instruction string and generating control instructions, the text instruction analysis module comprises: performing word segmentation and feature representation on the text instruction string to obtain a word sequence; using Bag-of-Words or TF-IDF representation on the word sequence and using a Softmax regression or SVM classifier to perform intent classification to predict an intent category; performing slot extraction according to the word sequence to obtain slot-value pairs; matching an instruction template according to the predicted intent category and the slot-value pairs to generate a structured control instruction.

7. The voice command recognition and generation system of claim 6, wherein, The slot extraction method comprises a rule-based entity dictionary matching method and a statistical model-based sequence labeling method.

8. The voice command recognition and generation system of claim 1, wherein, The sum of the audio collection time of the voice collection module, the endpoint detection time of the voice endpoint detection module, the recognition processing time of the voice recognition module, the text analysis time of the text instruction analysis module and the command generation and delivery time of the instruction sending module is the total delay of the module sequential execution; The response time of the system is less than the total delay of the module sequential execution.

9. A method for voice command recognition and generation, the method comprising: The method is multi-threaded and concurrent by step decoupling and zero-copy message communication, and comprises: acquiring user voice signals; determining voice start and end points and intercepting voice sequences; transcribing voice sequences into text instruction strings; performing semantic understanding on the text instruction string to generate control instructions; sending the control instructions.

10. A computer device comprising: A memory, a processor, and a computer program stored on the memory and loadable on the processor, characterized in that the processor executes the computer program to implement the voice instruction recognition and generation system of any one of claims 1-8.

Images