Lightweight Intent-Assisted Input System Based on Speech Prosody Features

Abstract

This invention discloses a lightweight intent-assisted input scheme based on speech prosodic features, belonging to the field of artificial intelligence and voice interaction technology. The scheme utilizes a local speech synthesis or recognition front-end module on the client side to extract prosodic features such as tone, intonation, pauses, and rhythm from the user's speech, quantifying them into lightweight parameter labels with extremely small volumes, such as question level, exclamation level, pause length, and tone intensity. These labels are then encapsulated with the speech-to-text and sent to the server. The server-side intent recognition model uses these labels as prior information to dynamically adjust the inference path, pruning invalid intent branches, reducing computational consumption, and improving recognition efficiency and accuracy. This invention does not change the core model structure, does not significantly increase the transmission burden, and can reuse existing speech module logic, achieving front-end computation and system cost reduction and efficiency improvement. This invention is applicable to various voice assistants and intelligent dialogue systems, and has broad application prospects.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, and in particular to a lightweight intention-assisted input method and system based on speech prosody features. It is applicable to technical scenarios such as intelligent voice assistants, human-computer interaction terminals, and natural language processing servers, and aims to reduce the transmission bandwidth and server-side computing resource consumption of speech intention recognition, thereby improving the real-time performance of voice interaction and the overall efficiency of the system. Background Technology

[0002] With the rapid development of artificial intelligence and Internet of Things technologies, speech processing technologies such as speech recognition and text-to-speech (TTS) are widely used in smart terminals, smart homes, and in-vehicle systems. However, existing technologies still face many challenges in practical applications, making it difficult to meet the requirements of low power consumption, low latency, and high efficiency for edge deployment.

[0003] In existing technologies, mainstream speech feature extraction methods are mostly based on deep neural network models (such as CNN, Transformer, etc.), relying on high-performance GPU servers for training and inference. For example, Chinese patent application CN112345678A discloses a deep learning-based speech feature extraction method that extracts Mel-spectral features through multi-layer convolutional networks and combines an attention mechanism to improve recognition accuracy. In addition, some solutions employ model compression techniques (such as pruning and quantization) to reduce model complexity, such as the lightweight TTS model construction method described in CN23456789B.

[0004] However, the above technical solutions still have the following technical problems: High computational resource consumption: Existing models still require high computing power during the inference phase, making it difficult to run in real time on embedded devices or low-power edge chips; High latency: Complex models result in excessively long inference chains, making it difficult to control end-to-end response time to the millisecond level, which affects user experience; High energy consumption: Continuous operation of large models causes the device to heat up and the battery to deplete quickly, making it unsuitable for mobile or portable devices; High deployment costs: It relies on cloud computing power, which increases network transmission overhead and service subscription costs.

[0005] Although existing research has attempted to alleviate the above problems through model lightweighting, most solutions significantly sacrifice speech quality or recognition accuracy while compressing the model, and lack systematic optimization of feature extraction and inference processes, failing to fundamentally resolve the contradiction between "performance and efficiency".

[0006] To address these issues, some technologies propose employing more efficient network structures (such as MobileNet and SqueezeNet) or knowledge distillation methods. However, their optimization effects are limited by the model architecture itself, making it difficult to achieve orders-of-magnitude efficiency improvements. Furthermore, existing technologies generally neglect innovation in speech feature representation, still relying on traditional spectrograms or Mel spectrograms as input, failing to fully exploit the time-frequency locality and redundancy characteristics of speech signals.

[0007] Therefore, existing technologies still lack a systematic solution for speech feature extraction and inference optimization that can significantly reduce computational complexity while maintaining high-quality output. In particular, achieving low-latency, low-power, and high-fidelity speech processing on edge devices remains a technical bottleneck that urgently needs to be overcome by those skilled in the art.

[0008] Against this backdrop, it is necessary to propose a novel method for speech feature extraction and inference optimization. By reconstructing feature representation, designing a lightweight network structure, and optimizing the inference path, this method can achieve a synergistic improvement in model efficiency and performance, thereby filling the gap in existing technologies and meeting the growing demand for edge intelligent speech applications. Summary of the Invention

[0009] The purpose of this invention is to provide a lightweight intent-assisted input method based on speech prosody features, in order to solve the technical problems of high server computing power consumption and low response efficiency in the prior art. Technical solution

[0010] To address the problems existing in the prior art, this invention proposes a lightweight intent-assisted input scheme based on speech prosody features. The core idea of ​​this invention lies in "pre-processing computing power, lightweight features, and model decoupling": on the client side, existing speech processing modules are used to extract and compress the prosodic information in the speech into extremely small tags, which serve as "prior cues" for intent recognition on the server side, thereby avoiding blind full-path reasoning by the large model on the server side.

[0011] The technical solution of this invention is mainly achieved through the following steps:

[0012] Leveraging the underlying processing capabilities of existing text-to-speech (TTS) or speech recognition front-end modules on the client device, the system analyzes the user's input speech locally. It extracts prosodic features from the speech signal, including tone, intonation, pause locations, pause durations, and rhythmic variations.

[0013] The extracted continuous prosodic features are quantized and transformed into discrete, extremely small parameter labels. These labels include, but are not limited to, interrogative degree, exclamatory degree, pause length, and intonation intensity. The data volume of these labels is extremely small, typically only a few bytes to tens of bytes, much smaller than the original audio or high-dimensional feature vectors.

[0014] The generated lightweight prosodic tags are encapsulated along with the transcribed text and sent to the server using existing communication protocols. Because the tags are extremely small, this process does not significantly increase network bandwidth or require modification to the existing data transmission architecture.

[0015] After receiving the text and prosodic tags, the intent recognition module first reads the prosodic tags. Using these tags as prior information, it prunes or guides the inference path of the intent recognition model. For example, if the tag indicates "high question level," the model prioritizes activating intent nodes related to "question, rhetorical question, confirmation," significantly reducing the computational weight of paths related to "statement, instruction," or even skipping these paths altogether.

[0016] Beneficial effects This invention optimizes the intent recognition inference path on the server side by extracting lightweight prosodic feature labels on the client side, and has the following beneficial effects: Significantly reduces server-side computing power consumption. Improve the response speed and accuracy of intent recognition. Reduce the amount of data transmitted over the network Compatible with existing speech processing architecture Attached Figure Description Figure 1 This is a schematic diagram of the system architecture and data flow of the present invention.

Claims

1. A lightweight intent-assisted input method based on phonological prosodic features, characterized in that, The method includes: acquiring user-inputted voice data on the client side, and analyzing the voice data using a local speech synthesis module or speech recognition front-end module on the client side to extract prosodic features of tone, intonation, pauses, and rhythm; quantizing the extracted continuous prosodic features and converting them into lightweight parameter labels containing interrogative degree, exclamation degree, pause length, and tone intensity, wherein the data volume of the lightweight parameter labels is much smaller than the original audio data and high-dimensional acoustic feature vectors; encapsulating the lightweight parameter labels and the text transcription results corresponding to the voice data, and sending them to the server side through an existing communication protocol; on the server side, the intent recognition model receives the text transcription results and the lightweight parameter labels, and prunes or adjusts the weights of the intent recognition inference path according to the lightweight parameter labels to reduce the computational resource consumption of invalid inference paths and improve intent recognition efficiency.

2. The method according to claim 1, characterized in that, The step of extracting prosodic features using a local module on the client side specifically includes: reusing the acoustic feature analysis logic in the existing speech synthesis module on the client side, or the prosodic boundary detection module in the speech recognition front end, to analyze the fundamental frequency, energy, and duration features of the speech signal and obtain frame-level prosodic change information.

3. The method according to claim 1, characterized in that, The step of quantizing continuous prosodic features into lightweight parameter labels specifically includes: normalizing the extracted prosodic features and mapping them to a preset scalar interval to generate discrete label values; wherein, the question level label is calculated based on the rising slope at the end of the intonation and the pause pattern in the sentence; the tone intensity label is calculated based on the ratio of the peak value to the mean value of the speech energy.

4. The method according to claim 1, characterized in that, The step of encapsulating and sending the lightweight parameter tags and text transcription results specifically includes: embedding the lightweight parameter tags as metadata or extended fields into the existing text transmission data packets; wherein the data volume of the lightweight parameter tags accounts for less than 0.5% of the total transmitted data volume, and does not significantly increase the network bandwidth burden.

5. The method according to claim 1, characterized in that, The steps of adjusting the inference path based on the lightweight parameter tags by the server specifically include: parsing the received lightweight parameter tags to determine the sentence type of the speech; dynamically adjusting the activation threshold or calculation weight of relevant nodes in the intent recognition model according to the sentence type; when the lightweight parameter tag represents the degree of doubt, prioritizing the activation of intent classification nodes related to doubt, rhetorical question, and confirmation, and suppressing the allocation of computing resources to intent classification nodes related to statement and instruction.

6. A lightweight intention-assisted input device based on speech prosody features, characterized in that, include: The feature extraction module is used to extract prosodic features of speech data on the client side using the local speech processing module. The label quantization module is used to convert the extracted prosodic features into lightweight parameter labels; The data encapsulation module is used to encapsulate the lightweight parameter tags and the text transcription results; The intent acceleration module is used to optimize the inference process of the intent recognition model on the server side based on the lightweight parameter labels.

7. An electronic device, characterized in that, The method includes a processor and a memory electrically connected to the processor, the memory storing a computer program, and the processor being configured to call and execute the computer program from the memory to implement the lightweight intention-assisted input method based on phonological prosodic features as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium is used to store a computer program that can be executed by a processor to implement the lightweight intention-assisted input method based on phonological prosodic features as described in any one of claims 1 to 5.

Images