Multielement intention recognition method and system based on sound
By converting user voice streams into speech text and using big model technology for multi-intention recognition, the linguistic diversity and data limitation problems existing in sound intention recognition in traditional NLP are solved, and higher intent recognition accuracy is achieved.
Patent Information
- Application Number
- CN202411599930.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-02-28
Smart Images
Figure CN119541473A_ABST
Abstract
Description
Background Art
[0002] Traditional intent recognition uses NLP to perform intent recognition. Using NLP for voice intent recognition has the following defects:
[0003] 1. Diversity and complexity of language:
[0004] 1.1 Accent and dialect issues: People in different regions have different accents, dialects, and language habits, which can lead to differences in pronunciation, intonation, and speech speed, increasing the difficulty of recognition. For example, some southern dialects differ greatly from Mandarin in pronunciation and word usage, which may cause the model to have a lower accuracy in recognizing the voice intent of users in these regions. For some niche dialects or language variants in specific regions, the model may not have enough training data to accurately understand and recognize them.
[0005] 1.2 Diversity of language expression: People may have many different ways of expressing the same intention, and language expression is often flexible and creative, which brings challenges to the learning and understanding of the model. For example, different expressions such as "I want to eat", "I'm hungry and want to eat something", and "I want to find something to eat" may all express the same intention of eating, but there are great differences in the text. The model needs to be able to accurately understand these different expressions and identify the intentions behind them.
[0006] 1.3 Difficulty in semantic understanding: There are complex semantic phenomena in language, such as polysemy, multiple words with one meaning, metaphor, irony, etc., which makes it difficult to accurately understand the true meaning of a sentence. For example, "I feel a little cold today" can simply mean that the body feels cold, or it can be a euphemism that implies the need for care or other implicit intentions. NLP models may make mistakes when understanding such ambiguous expressions.
[0007] 2. Limitations on data quality and quantity:
[0008] 2.1 Data Bias: The data used to train the NLP model may be biased, resulting in inaccurate intent recognition for certain groups or scenarios. For example, if the training data mainly comes from users in urban areas, the voice intent recognition effect for users in rural areas may not be good. In addition, some types of intent may be over-represented in the dataset, while other types of intent may be ignored, which will affect the generalization ability of the model.
[0009] 2.2 Accuracy of data annotation: Accurate annotation is crucial for the training of NLP models, but the data annotation process may contain errors or inconsistencies. Manually annotating data requires a lot of time and effort, and the subjective understanding and judgment of the annotators may also lead to differences in the annotation results. If the annotated data is inaccurate, the model will learn the wrong pattern, which will affect the accuracy of intent recognition.
[0010] 2.3 Insufficient data: For some specific fields or rare intents, there may be a lack of sufficient training data. NLP models require a large amount of data to learn language patterns and rules. If the amount of data is insufficient, the model cannot fully learn the relevant features, resulting in poor recognition of these rare intents. Summary of the invention
[0011] The present application provides a sound-based multi-intention recognition method and system to improve the accuracy of intention recognition in scenarios with complex language semantics.
[0012] In a first aspect, a method for multi-intention recognition based on sound is provided, comprising the following steps:
[0013] Convert the user's voice stream to obtain voice text;
[0014] Perform multi-intention recognition on the speech text to obtain a multi-intention recognition result.
[0015] In the above technical scheme, the user sound stream is converted to obtain voice text; the voice text is subjected to multi-intention recognition to obtain multi-intention recognition results; compared with the use of NLP for semantic intent recognition, the semantics are analyzed by multi-intention recognition according to classification models such as target intention, action intention, parameter intention, emotional intention, and potential intention by using large model technology, which can more accurately identify user intentions; the intention recognition large model technology solves the problem of understanding bias caused by NLP due to lack of sufficient training data and inaccurate data labeling.
[0016] In a specific implementation scheme, the step of converting the user voice stream to obtain the voice text is specifically as follows:
[0017] The user voice stream is converted through a speech recognition model to obtain the speech text.
[0018] In a specific implementation scheme, the step of performing multi-intention recognition on the speech text to obtain a multi-intention recognition result is specifically as follows:
[0019] The multi-intention recognition is performed on the speech text through the multi-intention recognition model to obtain the multi-intention recognition result.
[0020] In a specific embodiment, it also includes:
[0021] The user's voice is collected to obtain the user's voice stream.
[0022] In a specific embodiment, it also includes:
[0023] The multi-intention recognition result is output.
[0024] In a specific implementation scheme, the multi-intention recognition results include target intention, action intention, parameter intention, emotional intention and potential intention.
[0025] In a specific implementation scheme, the speech recognition model includes a hidden Markov model.
[0026] In a specific implementation manner, the multivariate model for intent recognition includes a large-model-based multivariate model for intent recognition.
[0027] In a specific embodiment, the large model includes a large language model.
[0028] In a second aspect, a sound-based multi-intention recognition system is provided, comprising:
[0029] An input module, used to collect the user's voice and obtain the user's voice stream;
[0030] The text module is used to convert the user's voice stream into voice text;
[0031] A recognition module, used to perform multi-intention recognition on the speech text to obtain a multi-intention recognition result;
[0032] An output module is used to output the multi-intention recognition result.
[0033] In the above technical scheme, the user sound stream is converted to obtain voice text; the voice text is subjected to multi-intention recognition to obtain multi-intention recognition results; compared with the use of NLP for semantic intent recognition, the semantics are analyzed by multi-intention recognition according to classification models such as target intention, action intention, parameter intention, emotional intention, and potential intention by using large model technology, which can more accurately identify user intentions; the intention recognition large model technology solves the problem of understanding bias caused by NLP due to lack of sufficient training data and inaccurate data labeling. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A flowchart of a method for multi-intention recognition based on sound provided in an embodiment of the present application;
[0035] Figure 2A structural block diagram of a sound-based multi-intention recognition system provided in an embodiment of the present application;
[0036] Figure 3 A specific flow chart of the sound-based multi-intention recognition method provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.
[0038] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.
[0039] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.
[0040] To facilitate understanding of the multi-intention recognition method and system based on sound provided in the embodiment of the present application, its application scenario is first explained. The multi-intention recognition method and system based on sound provided in the embodiment of the present application are used to improve the accuracy of intention recognition in scenarios facing complex language semantics. Traditional intention recognition uses NLP to perform intention recognition. There are some defects in using NLP for sound intention recognition: 1. Diversity and complexity of language: 1.1 Accent and dialect problems: People in different regions have different accents, dialects and language habits, which will lead to differences in pronunciation, intonation, speaking speed and other aspects of speech, increasing the difficulty of recognition. For example, some dialects in southern regions are quite different from Mandarin in pronunciation and wording, which may cause the model to reduce the accuracy of voice intention recognition for users in these regions. For some niche dialects or language variants in specific regions, the training data of the model may be insufficient, making it difficult to accurately understand and recognize. 1.2 Diversity of language expression: People may have many different ways of expressing the same intention, and language expression is often flexible and creative, which brings challenges to the learning and understanding of the model. For example, different expressions such as "I want to eat", "I'm hungry and want to eat something", and "I want to find something to eat" may all express the same intention to eat, but there are great differences in the text. The model needs to be able to accurately understand these different expressions and identify the intentions behind them. 1.3 Difficulty in semantic understanding: There are complex semantic phenomena in language, such as polysemy, multiple words with one meaning, metaphor, and irony, which makes it difficult to accurately understand the true meaning of the sentence. For example, "I feel a little cold today" may simply express the meaning of feeling cold physically, or it may be a euphemism that implies the need for care or other implicit intentions. The NLP model may make mistakes when understanding such ambiguous expressions. 2. Limitations on data quality and quantity: 2.1 Data bias: The data used to train the NLP model may be biased, resulting in inaccurate recognition of the intentions of certain specific groups or scenarios. For example, if the training data mainly comes from users in urban areas, the recognition effect of the voice intentions of users in rural areas may not be good. In addition, some types of intentions may be over-represented in the data set, while other types of intentions are ignored, which will affect the generalization ability of the model. 2.2 Accuracy of data annotation: Accurate annotation is crucial for training NLP models, but the data annotation process may contain errors or inconsistencies. Manually annotating data takes a lot of time and effort, and the subjective understanding and judgment of the annotators may also lead to differences in the annotation results. If the annotated data is inaccurate, the model will learn the wrong pattern, which will affect the accuracy of intent recognition. 2.3 Insufficient data volume: For some specific fields or rare intents, there may be a lack of sufficient training data.NLP models require a large amount of data to learn language patterns and rules. If the amount of data is insufficient, the model cannot fully learn the relevant features, resulting in poor recognition of these rare intents. To this end, the embodiment of the present application provides a method and system for multi-intention recognition based on sound to improve the accuracy of intent recognition in scenarios facing complex language semantics. The following is a detailed description of the embodiment in conjunction with specific drawings.
[0041] refer to Figures 1 to 3 , Figure 1 A flowchart of a method for multi-intention recognition based on sound provided in an embodiment of the present application; Figure 2 A structural block diagram of a sound-based multi-intention recognition system provided in an embodiment of the present application; Figure 3 A specific flow chart of the sound-based multi-intention recognition method provided in an embodiment of the present application.
[0042] exist Figure 1 In the embodiment of the present application, a method for multi-intention recognition based on sound is provided, comprising the following steps:
[0043] Convert the user's voice stream to obtain voice text;
[0044] Perform multi-intention recognition on the speech text to obtain a multi-intention recognition result.
[0045] In the above technical scheme, the user sound stream is converted to obtain voice text; the voice text is subjected to multi-intention recognition to obtain multi-intention recognition results; compared with the use of NLP for semantic intent recognition, the semantics are analyzed by multi-intention recognition according to classification models such as target intention, action intention, parameter intention, emotional intention, and potential intention by using large model technology, which can more accurately identify user intentions; the intention recognition large model technology solves the problem of understanding bias caused by NLP due to lack of sufficient training data and inaccurate data labeling.
[0046] In a specific implementation scheme, the step of converting the user voice stream to obtain the voice text is specifically as follows:
[0047] The user voice stream is converted through a speech recognition model to obtain the speech text.
[0048] In a specific implementation scheme, the step of performing multi-intention recognition on the speech text to obtain a multi-intention recognition result is specifically as follows:
[0049] The multi-intention recognition is performed on the speech text through the multi-intention recognition model to obtain the multi-intention recognition result.
[0050] In a specific embodiment, it also includes:
[0051] The user's voice is collected to obtain the user's voice stream.
[0052] In a specific embodiment, it also includes:
[0053] The multi-intention recognition result is output.
[0054] In a specific implementation scheme, the multi-intention recognition results include target intention, action intention, parameter intention, emotional intention and potential intention.
[0055] Specifically, refer to Figure 3 The sound-based multi-intention recognition method converts the sound stream collected by the user into corresponding text through a speech recognition model, and decomposes the user's intention into target intention, action intention, parameter intention, emotional intention, potential intention, etc. through an intention recognition multi-model, and returns the intention result to the caller.
[0056] For example, the user says: Please set an alarm for me at 6 o'clock tomorrow morning, I need to rush to the airport.
[0057] After model decomposition:
[0058] Target intent: alarm clock, airport;
[0059] Action intention: reservation, going;
[0060] Parameter intention: Calculate tomorrow morning 6 o'clock based on the current time;
[0061] Emotional intention: positive;
[0062] Potential intentions: Check weather conditions, whether a taxi service is needed, etc.
[0063] In the above technical solutions, intent recognition is more accurate in scenarios with complex language semantics.
[0064] In a specific implementation scheme, the speech recognition model includes a hidden Markov model.
[0065] In a specific implementation manner, the multivariate model for intent recognition includes a large-model-based multivariate model for intent recognition.
[0066] In a specific embodiment, the large model includes a large language model.
[0067] Specifically, speech recognition (Automatic Speech Recognition) technology is a technology that converts human speech into text. It involves multiple disciplines such as acoustics, phonetics, linguistics, information theory, pattern recognition theory, and neurobiology. The speech recognition model is the core component to realize this technology.
[0068] 1. The speech recognition system is essentially a pattern recognition system, which mainly includes three basic units: feature extraction, pattern matching, and reference pattern library. Including:
[0069] Audio acquisition and preprocessing: Sound signals are collected through a microphone and digitized (analog-to-digital conversion), denoised, and background sound removed.
[0070] Feature extraction: The audio signal is divided into short time frames (usually 10 to 30 milliseconds) and features are extracted for each frame. Commonly used features include Mel-frequency cepstral coefficients (MFCC), linear prediction coefficients (LPC), perceptual linear prediction (PLP), etc.
[0071] Acoustic model: Use models such as Hidden Markov Model (HMM) and Deep Neural Network (DNN) to map acoustic features to phonemes or other acoustic units. The acoustic model is the most basic and critical part of the speech recognition system, and the quality of its modeling will directly affect the recognition effect and robustness of the speech recognition system.
[0072] Language model: Use N-gram model or neural network language model to provide the prior probability of word sequence. Language model is mainly used to predict the probability of a word or word sequence.
[0073] Decoding: Combine the acoustic model and language model to find the most likely word sequence using methods such as the Viterbi algorithm or beam search.
[0074] Post-processing: insert punctuation marks, convert uppercase and lowercase letters, etc.
[0075] 2. Common models include:
[0076] Dynamic Time Warping (DTW): It is still the mainstream method in continuous speech recognition. This method has a large amount of computation, but is relatively simple in technology and has a high recognition accuracy.
[0077] Vector Quantization (VQ): A method based on a non-parametric model. It requires less model training data, less training and recognition time, and less working storage space. It can be well applied in isolated character (word) speech recognition systems.
[0078] Hidden Markov Models (HMM): A method based on parameter models, mainly used in speech recognition systems with large vocabulary. It requires more model training data, longer training and recognition time, and larger storage space. Generally, the continuous hidden Markov model has a larger amount of calculation than the discrete hidden Markov model, but the recognition rate is relatively higher.
[0079] Deep Neural Networks (DNN): In recent years, with the continuous development of deep learning technology, deep neural networks have achieved remarkable results in the field of speech recognition. DNN models can directly learn from raw audio features to text output, reducing multiple independent steps in traditional methods.
[0080] In addition, there are some speech recognition models that combine deep learning technology, such as recurrent neural network (RNN), long short-term memory network (LSTM), convolutional neural network (CNN), etc. These models perform well in speech recognition tasks and can understand and recognize user voice input more accurately.
[0081] Furthermore, intent recognition is a key component of natural language processing (NLP), and multivariate models play an important role in intent recognition. These models are able to process and understand the complexity and diversity of human language, thereby more accurately identifying the user's intent. Multivariate models for intent recognition include:
[0082] 1. Deep Learning Model:
[0083] Recurrent Neural Network (RNN): Able to process sequence data and capture temporal dependencies in text.
[0084] Long Short-Term Memory (LSTM): As an improved version of RNN, LSTM can solve the long-term dependency problem and better capture the long-distance dependencies in the text.
[0085] Attention mechanism: By assigning different attention weights to different parts of the text, the attention mechanism can focus on key information and improve the accuracy of intent recognition.
[0086] 2. Joint Model:
[0087] Joint BERT model: Based on the BERT architecture, it uses BERT's powerful bidirectional context representation capability to simultaneously handle intent classification and slot filling tasks through fine-tuning. The Joint BERT model uses the first hidden state of the special token [CLS] to predict intent, while the final hidden states of other tokens are used to classify slot filling labels through a softmax layer.
[0088] Other joint models: such as models based on CNN+Tri-CRF and joint models based on attention-based RNN. These models can also achieve good performance in intent recognition and slot filling tasks.
[0089] 3. Traditional machine learning model:
[0090] Although deep learning models have achieved remarkable results in intent recognition, traditional machine learning models such as support vector machines (SVM) still have application value in certain scenarios. These models are trained and predicted by manually extracted features and can recognize user intent to a certain extent.
[0091] 4. Multimodal Model:
[0092] In some complex scenarios, the user's intention may not only be expressed through text, but may also involve multiple modalities such as images and sounds. Therefore, multimodal models are gradually gaining attention in intent recognition. These models can integrate information from multiple modalities to understand the user's intention more comprehensively.
[0093] The multivariate model of intent recognition based on the large model includes:
[0094] 1. Intent Recognition Based on Large Language Model
[0095] Large language models (LLMs) such as Claude have powerful semantic analysis and context understanding capabilities, so they can be used for intent recognition tasks. By training a large language model, it can understand the text input by the user and recognize the intent. This method can process complex natural language input and output formatted intent results.
[0096] 2. Intent Recognition Based on Knowledge Enhancement and Pre-trained Large Model
[0097] This method combines the advantages of knowledge enhancement and pre-trained large models, and improves the accuracy of intent recognition by introducing external knowledge bases and pre-trained models. For example, the business knowledge and professional terms within the enterprise can be incorporated into the model, enabling it to better understand user input in specific fields. At the same time, the powerful representation ability of the pre-trained model can capture the deep semantic information in the text, thereby more accurately identifying the intent.
[0098] 3. Joint Model
[0099] Joint models usually combine intent recognition with other NLP tasks (such as slot filling, text classification, etc.) to improve the overall performance by sharing model parameters and feature information. For example, a joint model based on BERT can be built that can handle intent classification and slot filling tasks at the same time. By fine-tuning the BERT model, it can be adapted to the intent recognition task in a specific domain and achieve good performance.
[0100] 4. Multimodal Intent Recognition Model
[0101] In some complex scenarios, the user's intention may not only be expressed through text, but may also involve multiple modalities such as images and sounds. Therefore, multimodal intent recognition models have emerged. These models can integrate information from multiple modalities to understand the user's intention more comprehensively. For example, speech recognition technology and text analysis technology can be combined to identify the user's intention in voice input.
[0102] 5. Innovation of intent recognition method based on large model
[0103] In recent years, some researchers have proposed innovative intent recognition methods based on large models. For example, by using the generative capabilities of large models to generate candidate sentences and candidate intents similar to the input sentences, the intent recognition process can be assisted. This method can improve the accuracy and recall rate of intent recognition while reducing the reliance on labeled data.
[0104] In the above technical solution, the multivariate model of intent recognition based on a large model has powerful semantic analysis and context understanding capabilities, can process complex natural language inputs, and accurately identify the intent therein.
[0105] exist Figure 2 In the embodiment of the present application, a multi-intention recognition system based on sound is provided, comprising:
[0106] An input module, used to collect the user's voice and obtain the user's voice stream;
[0107] The text module is used to convert the user's voice stream into voice text;
[0108] A recognition module, used to perform multi-intention recognition on the speech text to obtain a multi-intention recognition result;
[0109] An output module is used to output the multi-intention recognition result.
[0110] In the above technical scheme, the user sound stream is converted to obtain voice text; the voice text is subjected to multi-intention recognition to obtain multi-intention recognition results; compared with the use of NLP for semantic intent recognition, the semantics are analyzed by multi-intention recognition according to classification models such as target intention, action intention, parameter intention, emotional intention, and potential intention by using large model technology, which can more accurately identify user intentions; the intention recognition large model technology solves the problem of understanding bias caused by NLP due to lack of sufficient training data and inaccurate data labeling.
[0111] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.
[0112] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.
[0113] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0114] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.
Claims
1. A method for multi-intention recognition based on sound, characterized in that: The following steps are involved: Convert the user's voice stream to obtain voice text; Perform multi-intention recognition on the speech text to obtain a multi-intention recognition result.
2. The method for multi-intention recognition based on sound according to claim 1, characterized in that: The steps of converting the user's voice stream to obtain voice text are as follows: The user voice stream is converted through a speech recognition model to obtain the speech text.
3. The method for multi-intention recognition based on sound according to claim 2, characterized in that: The step of performing multi-intention recognition on the speech text to obtain a multi-intention recognition result is specifically as follows: The multi-intention recognition is performed on the speech text through the multi-intention recognition model to obtain the multi-intention recognition result.
4. The method for multi-intention recognition based on sound according to claim 3, characterized in that: Also includes: The user's voice is collected to obtain the user's voice stream.
5. The method for multi-intention recognition based on sound according to claim 4, characterized in that: Also includes: The multi-intention recognition result is output.
6. The method for multi-intention recognition based on sound according to claim 5, characterized in that: The multi-intention recognition results include target intention, action intention, parameter intention, emotional intention and potential intention.
7. The method for multi-intention recognition based on sound according to claim 6, characterized in that: The speech recognition model includes a hidden Markov model.
8. The method for multi-intention recognition based on sound according to claim 7, characterized in that: The multivariate model for intention recognition includes a multivariate model for intention recognition based on a large model.
9. The method for multi-intention recognition based on sound according to claim 8, characterized in that: The large model includes a large language model.
10. A sound-based multi-intention recognition system, characterized in that: include: An input module, used to collect the user's voice and obtain the user's voice stream; The text module is used to convert the user's voice stream into voice text; A recognition module, used to perform multi-intention recognition on the speech text to obtain a multi-intention recognition result; An output module is used to output the multi-intention recognition result.