Voice control implementation method and system based on large model

Abstract

The invention discloses a voice control implementation method and system based on a large model, belongs to the technical field of artificial intelligence, and aims to solve the technical problems of insufficient recognition precision, low response speed and limited complex context processing capability in voice control. Comprising the following steps: analyzing a user voice instruction into a text instruction through a voice recognition model; performing semantic feature extraction on an input text instruction through a pre-trained text embedding model, and outputting a text embedding vector as a to-be-recognized text embedding vector; searching whether a text embedding vector similar to the to-be-identified text embedding vector exists in a pre-constructed vector database, if not, executing text analysis, and if yes, returning corresponding control information from the vector database; and taking the text instruction and the question statement as input, performing semantic analysis through a pre-trained large language model to obtain control information, and storing a text embedding vector corresponding to the current user voice instruction and the control information into a vector database.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for implementing voice control based on a large model. Background Art

[0002] With the development of artificial intelligence technology, speech recognition systems have shown broad application potential in multiple scenarios such as smart home, car driving assistance, and remote control. However, existing technologies still face challenges in processing complex contexts, improving recognition accuracy, and response speed. In particular, accurately mapping spoken user instructions to specific control commands and efficiently processing repeated or similar requests are the keys to improving user experience.

[0003] Insufficient recognition accuracy, slow response speed and limited ability to handle complex contexts in voice control are technical problems that need to be solved. Summary of the invention

[0004] The technical task of the present invention is to address the above deficiencies and provide a large-model-based voice control implementation method and system to solve the technical problems of insufficient recognition accuracy, slow response speed and limited ability to handle complex contexts in voice control.

[0005] In a first aspect, the present invention provides a method for implementing voice control based on a large model, comprising the following steps:

[0006] Speech recognition: Obtain user voice commands and parse them into text commands through the speech recognition model;

[0007] Text vector conversion: Take the text instruction as input, extract the semantic features of the input text instruction through the pre-trained text embedding model, and output the text embedding vector as the embedding vector of the text to be recognized;

[0008] Vector query matching: for the text embedding vector to be recognized, check whether there is a text embedding vector similar to the text embedding vector to be recognized from the pre-built vector database. If not, perform the next step of text parsing. If so, return the corresponding control information from the vector database based on the text embedding vector similar to the text embedding vector to be recognized. The vector database stores the text embedding vectors and control information corresponding to the historical user voice commands. The control information includes the control intention, target object and specific operation parameters.

[0009] Text parsing: For text instructions, the text instructions and problem statements are used as input, and semantic analysis is performed through a pre-trained large language model to obtain control information including control intent, target objects, and specific operating parameters. The control information is encapsulated into a predetermined format, and the text embedding vector and control information corresponding to the current user's voice instruction are stored in the vector database.

[0010] Preferably, the speech recognition model is a Funasr real-time speech recognition model.

[0011] Preferably, the text embedding model is a Bert model.

[0012] Preferably, for the text embedding vector to be recognized, the distance between the text embedding vector to be recognized and the text embedding vectors stored in the vector database is calculated in sequence as the vector similarity. If there is a vector similarity greater than a threshold, the text embedding vector corresponding to the vector similarity greater than the threshold is taken as a text embedding vector similar to the text embedding vector to be recognized.

[0013] Preferably, during text parsing, for the control information output by the pre-trained large language model, the control information is filled into a pre-configured JSON template to obtain the control information in JSON format, and the text embedding vector corresponding to the current user voice command and the control information in JSON format are used as key-value pairs, and the key-value pairs are cached in the vector database;

[0014] The JSON module includes fields related to the operation type, target device type, and specific operation parameter settings.

[0015] In a second aspect, the present invention provides a large model-based voice control implementation system, including a voice recognition module, a text vector conversion module, a vector query matching module, and a text parsing module;

[0016] The speech recognition module is used to perform the following: obtain user voice commands, and parse the user voice commands into text commands through a speech recognition model;

[0017] The text vector conversion module is used to perform the following: taking the text instruction as input, extracting semantic features of the input text instruction through a pre-trained text embedding model, and outputting a text embedding vector as a text embedding vector to be recognized;

[0018] The vector query matching module is used to perform the following: for a text embedding vector to be recognized, whether there is a text embedding vector similar to the text embedding vector to be recognized from a pre-built vector database, if not, perform the next step of text parsing, if yes, based on the text embedding vector similar to the text embedding vector to be recognized, return the corresponding control information from the vector database, wherein the vector database stores the text embedding vector corresponding to the historical user voice command and the control information, and the control information includes the control intention, the target object and the specific operation parameters;

[0019] The text parsing module is used to perform the following: for text instructions, it takes the text instructions and problem statements as input, performs semantic analysis through a pre-trained large language model, obtains control information including control intentions, target objects, and specific operating parameters, encapsulates the control information into a predetermined format, and stores the text embedding vector and control information corresponding to the current user voice instruction in a vector database.

[0020] Preferably, the speech recognition model is a Funasr real-time speech recognition model.

[0021] Preferably, the text embedding model is a Bert model.

[0022] Preferably, for the text embedding vector to be identified, the vector query matching module is used to sequentially calculate the distance between the text embedding vector to be identified and the text embedding vectors stored in the vector database as vector similarity. If there is a vector similarity greater than a threshold, the text embedding vector corresponding to the vector similarity greater than the threshold is taken as a text embedding vector similar to the text embedding vector to be identified.

[0023] Preferably, for the control information output by the pre-trained large language model, the text parsing module is used to fill the control information into a pre-configured JSON template to obtain the control information in JSON format, and use the text embedding vector corresponding to the current user voice command and the control information in JSON format as a key-value pair, and cache the key-value pair in the vector database;

[0024] The JSON module includes fields related to the operation type, target device type, and specific operation parameter settings.

[0025] The large model-based voice control implementation method and system of the present invention have the following advantages:

[0026] 1. By combining advanced speech recognition technology and intelligent processing of large models, efficient and accurate conversion from speech to control commands is achieved;

[0027] 2. Build a vector database based on the text embedding vectors and control information corresponding to historical user voice commands. For the user voice commands to be recognized, retrieve similar text embedding vectors and control information from the vector database, which effectively shortens the processing time, improves the user experience, reduces the server burden, and enhances stability and efficiency.

[0028] 3. It is suitable for various scenarios that require efficient voice control interaction and has significant technical advantages and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] The present invention is further described below in conjunction with the accompanying drawings.

[0031] Figure 1 This is a flowchart of a method for implementing voice control based on a large model in Example 1. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0033] The embodiments of the present invention provide a large-model-based voice control implementation method and system for solving the technical problems of insufficient recognition accuracy, slow response speed, and limited ability to handle complex contexts in voice control.

[0034] Embodiment 1:

[0035] The present invention discloses a method for realizing voice control based on a large model, which comprises four steps: voice recognition, text vector conversion, vector query matching and text parsing.

[0036] Step S100 speech recognition: obtaining user voice commands and parsing the user voice commands into text commands through a speech recognition model.

[0037] The speech recognition model in this embodiment is the Funasr real-time speech recognition model, which is based on a deep learning algorithm and can accurately and quickly convert speech signals into text. For example, a user issues a voice command "Please increase the gear of device A" and the voice is converted into text after being processed by the Funasr model.

[0038] Step S200 text vector conversion: taking the text instruction as input, extracting semantic features of the input text instruction through a pre-trained text embedding model, and outputting a text embedding vector as the text embedding vector to be recognized.

[0039] In this embodiment, the text embedding model is a Bert model. Through a pre-trained text embedding model such as a Bert model, the text requested by the user is converted into a high-dimensional vector representation, which can effectively capture the semantic features of the text.

[0040] Step S300 vector query matching: for the text embedding vector to be recognized, retrieve from the pre-built vector database whether there is a text embedding vector similar to the text embedding vector to be recognized. If not, execute the next step of text parsing. If so, return the corresponding control information from the vector database based on the text embedding vector similar to the text embedding vector to be recognized, wherein the vector database stores the text embedding vectors and control information corresponding to the historical user voice commands, and the control information includes the control intention, target object and specific operation parameters.

[0041] In this embodiment, when retrieving a matching text embedding vector from a vector database, for the text embedding vector to be recognized, the distance between the text embedding vector to be recognized and the text embedding vectors stored in the vector database is calculated in sequence as the vector similarity. If there is a vector similarity greater than a threshold, the text embedding vector corresponding to the vector similarity greater than the threshold is taken as a text embedding vector similar to the text embedding vector to be recognized.

[0042] Step S400 text parsing: For text instructions, the text instructions and problem statements are used as input, and semantic analysis is performed through a pre-trained large language model to obtain control information including control intentions, target objects, and specific operating parameters. The control information is encapsulated into a predetermined format, and the text embedding vector and control information corresponding to the current user voice instruction are stored in a vector database.

[0043] As a specific implementation of text parsing, for the control information output by the pre-trained large language model, the control information is filled into the pre-configured JSON template to obtain the control information in JSON format, and the text embedding vector corresponding to the current user voice command and the control information in JSON format are used as key-value pairs, and the key-value pairs are cached in the vector database. Among them, the JSON module includes fields related to the operation type, target device type, and specific operation parameter settings.

[0044] This embodiment designs a flexible JSON template system, which dynamically fills the JSON template with the control information parsed according to the large language model. The JSON template includes fields such as operation type, target device type and specific operation parameters to ensure that the generated control message is structured and standardized.

[0045] The control message template is defined as follows:

[0046]

[0047] After the information output by the large language model is processed logically, it is automatically filled into the corresponding JSON template according to pre-defined rules to generate the final control message. The final message is as follows:

[0048]

[0049] At the same time, this embodiment introduces a text embedding vector caching strategy. Based on this strategy, a pre-trained text embedding model (such as BERT, etc.) is used to convert the text requested by the user into a high-dimensional vector representation. The vector can effectively capture the semantic features of the text and establish a vector database based on key-value pairs (such as FAISS) to store text embedding vectors and their corresponding final JSON control messages. When a new voice request is received, the vector database is first queried. If an existing vector with a similarity higher than a threshold is found, its associated control message is directly returned to avoid repeated processing. In the case of a cache miss, the above-mentioned speech-to-text and information extraction process is executed, and the newly generated text embedding vector and its corresponding control message are added to the cache database for quick call by subsequent requests.

[0050] The following specific examples are given based on the method of this embodiment:

[0051] When the user inputs the voice input: "Please increase the gear of (device) A", the text is first vectorized through the text embedding model (such as Bert) to obtain its semantic related vector VECTOR_A. Use VECTOR_A to query the vector database FAISS for the vector closest to it. If the distance between the two exceeds a certain threshold (the distance is close enough to be considered the same command), it is determined that the cache does not hit. Extract information through the large model, and then fill in the json template to obtain its corresponding command message CMD_A. Store key:VECTOR_A, value:CMD_A in the FAISS database.

[0052] When the user issues a similar voice command again: "Please increase gear A a little bit", the text is vectorized through the same text embedding model to obtain its semantically related vector VECTOR_B. Use VECTOR_B as the key to retrieve the closest vector from the vector database FAISS. Since the semantics are similar to the previously processed commands, VECTOR_A is cached. The command message CMD_A corresponding to VECTOR_A is taken out as the final command message.

[0053] In specific implementation, the latest version of Funasr model is selected to ensure the accuracy of speech recognition, and large models such as GPT-3 are considered for their excellent performance in natural language understanding and generation. The cache strategy needs to adjust the similarity threshold and the maintenance strategy of the vector database according to the actual application scenario to balance query efficiency and storage cost.

[0054] The method of this embodiment improves recognition accuracy and understanding depth: it utilizes the high-precision speech-to-text capability of the Funasr model, combined with the large model's deep understanding of natural language, to ensure that even when faced with voice commands with ambiguous pronunciations, dialects, or complex grammatical structures, the system can accurately capture user intent.

[0055] The method of this embodiment speeds up the response speed: the introduced cache acceleration mechanism, especially the associated storage strategy of text embedding vectors and control messages, greatly reduces the time required to process the same or similar requests, thereby achieving instant feedback and enhancing user experience.

[0056] The method of this embodiment optimizes resource utilization and system efficiency: through intelligent caching, repeated calls to large models are avoided, the server load is effectively reduced, and the overall operating efficiency and stability of the system are improved.

[0057] The method of this embodiment enhances the flexibility and scalability of the system: the designed JSON template system facilitates rapid customization of the control instruction structure according to different application scenarios, so that the system can easily adapt to various smart devices and changing user needs.

[0058] Embodiment 2:

[0059] The invention discloses a large-model-based voice control implementation system, which comprises a voice recognition module, a text vector conversion module, a vector query matching module and a text parsing module.

[0060] The speech recognition module is used to perform the following: obtain user voice commands, and parse the user voice commands into text commands through a speech recognition model.

[0061] The speech recognition model in this embodiment is the Funasr real-time speech recognition model, which is based on a deep learning algorithm and can accurately and quickly convert speech signals into text. For example, a user issues a voice command "Please increase the gear of device A" and the voice is converted into text after being processed by the Funasr model.

[0062] The text vector conversion module is used to perform the following: taking a text instruction as input, extracting semantic features of the input text instruction through a pre-trained text embedding model, and outputting a text embedding vector as the text embedding vector to be recognized.

[0063] In this embodiment, the text embedding model is a Bert model. Through a pre-trained text embedding model such as a Bert model, the text requested by the user is converted into a high-dimensional vector representation, which can effectively capture the semantic features of the text.

[0064] The vector query matching module is used to perform the following: for a text embedding vector to be recognized, retrieve from a pre-built vector database whether there is a text embedding vector similar to the text embedding vector to be recognized; if not, perform the next step of text parsing; if so, return corresponding control information from the vector database based on the text embedding vector similar to the text embedding vector to be recognized, wherein the vector database stores text embedding vectors and control information corresponding to historical user voice commands, and the control information includes control intent, target object, and specific operation parameters.

[0065] In this embodiment, when retrieving a matching text embedding vector from a vector database, for the text embedding vector to be identified, the vector query matching module is used to sequentially calculate the distance between the text embedding vector to be identified and the text embedding vectors stored in the vector database as vector similarity. If there is a vector similarity greater than a threshold, the text embedding vector corresponding to the vector similarity greater than the threshold is used as a text embedding vector similar to the text embedding vector to be identified.

[0066] The text parsing module is used to perform the following: for text instructions, it takes the text instructions and problem statements as input, performs semantic analysis through a pre-trained large language model, obtains control information including control intentions, target objects, and specific operating parameters, encapsulates the control information into a predetermined format, and stores the text embedding vector and control information corresponding to the current user voice instruction in a vector database.

[0067] As a specific implementation of text parsing, for the control information output by the pre-trained large language model, the text parsing module is used to fill the control information into the pre-configured JSON template, obtain the control information in JSON format, and use the text embedding vector corresponding to the current user voice command and the control information in JSON format as key-value pairs, and cache the key-value pairs in the vector database. Among them, the JSON module includes fields related to the operation type, target device type, and specific operation parameter settings.

[0068] This embodiment designs a flexible JSON template system, which dynamically fills the JSON template with the control information parsed according to the large language model. The JSON template includes fields such as operation type, target device type and specific operation parameters to ensure that the generated control message is structured and standardized.

[0069] The control message template is defined as follows:

[0070]

[0071] After the information output by the large language model is processed logically, it is automatically filled into the corresponding JSON template according to pre-defined rules to generate the final control message. The final message is as follows:

[0072]

[0073] At the same time, this embodiment introduces a text embedding vector caching strategy. Based on this strategy, a pre-trained text embedding model (such as BERT, etc.) is used to convert the text requested by the user into a high-dimensional vector representation. The vector can effectively capture the semantic features of the text and establish a vector database based on key-value pairs (such as FAISS) to store text embedding vectors and their corresponding final JSON control messages. When a new voice request is received, the vector database is first queried. If an existing vector with a similarity higher than a threshold is found, its associated control message is directly returned to avoid repeated processing. In the case of a cache miss, the above-mentioned speech-to-text and information extraction process is executed, and the newly generated text embedding vector and its corresponding control message are added to the cache database for quick call by subsequent requests.

[0074] The system of this embodiment can execute the method disclosed in Example 1, combine the advanced Funasr speech recognition technology with the large model, realize the analysis of user voice commands and thus realize voice control.

[0075] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for implementing voice control based on a large model, characterized in that: The steps include: Speech recognition: Obtain user voice commands and parse them into text commands through the speech recognition model; Text vector conversion: Take the text instruction as input, extract the semantic features of the input text instruction through the pre-trained text embedding model, and output the text embedding vector as the embedding vector of the text to be recognized; Vector query matching: for the text embedding vector to be recognized, check whether there is a text embedding vector similar to the text embedding vector to be recognized from the pre-built vector database. If not, perform the next step of text parsing. If so, return the corresponding control information from the vector database based on the text embedding vector similar to the text embedding vector to be recognized. The vector database stores the text embedding vectors and control information corresponding to the historical user voice commands. The control information includes the control intention, target object and specific operation parameters. Text parsing: For text instructions, the text instructions and problem statements are used as input, and semantic analysis is performed through a pre-trained large language model to obtain control information including control intent, target objects, and specific operating parameters. The control information is encapsulated into a predetermined format, and the text embedding vector and control information corresponding to the current user's voice instruction are stored in the vector database.

2. The method for implementing voice control based on a large model according to claim 1, characterized in that: The speech recognition model is a Funasr real-time speech recognition model.

3. The method for implementing voice control based on a large model according to claim 1, characterized in that: The text embedding model is a Bert model.

4. The method for implementing voice control based on a large model according to claim 1, characterized in that: For the text embedding vector to be recognized, the distance between the text embedding vector to be recognized and the text embedding vector stored in the vector database is calculated in turn as the vector similarity. If there is a vector similarity greater than a threshold, the text embedding vector corresponding to the vector similarity greater than the threshold is taken as the text embedding vector similar to the text embedding vector to be recognized.

5. The method for implementing voice control based on a large model according to claim 1, characterized in that: During text parsing, for the control information output by the pre-trained large language model, the control information is filled into the pre-configured JSON template to obtain the control information in JSON format, and the text embedding vector corresponding to the current user voice command and the control information in JSON format are used as key-value pairs, and the key-value pairs are cached in the vector database; The JSON module includes fields related to the operation type, target device type, and specific operation parameter settings.

6. A voice control implementation system based on a large model, characterized in that: It includes speech recognition module, text vector conversion module, vector query matching module and text parsing module; The speech recognition module is used to perform the following: obtain user voice commands, and parse the user voice commands into text commands through a speech recognition model; The text vector conversion module is used to perform the following: taking the text instruction as input, extracting semantic features of the input text instruction through a pre-trained text embedding model, and outputting a text embedding vector as a text embedding vector to be recognized; The vector query matching module is used to perform the following: for a text embedding vector to be recognized, whether there is a text embedding vector similar to the text embedding vector to be recognized from a pre-built vector database, if not, perform the next step of text parsing, if yes, based on the text embedding vector similar to the text embedding vector to be recognized, return the corresponding control information from the vector database, wherein the vector database stores the text embedding vector corresponding to the historical user voice command and the control information, and the control information includes the control intention, the target object and the specific operation parameters; The text parsing module is used to perform the following: for text instructions, it takes the text instructions and problem statements as input, performs semantic analysis through a pre-trained large language model, obtains control information including control intentions, target objects, and specific operating parameters, encapsulates the control information into a predetermined format, and stores the text embedding vector and control information corresponding to the current user voice instruction in a vector database.

7. The large model-based voice control implementation system according to claim 6, characterized in that: The speech recognition model is a Funasr real-time speech recognition model.

8. The large model-based voice control implementation system according to claim 6, characterized in that: The text embedding model is a Bert model.

9. The large model-based voice control implementation system according to claim 6, characterized in that: For the text embedding vector to be recognized, the vector query matching module is used to calculate the distance between the text embedding vector to be recognized and the text embedding vectors stored in the vector database in sequence as vector similarity. If there is a vector similarity greater than a threshold, the text embedding vector corresponding to the vector similarity greater than the threshold is taken as a text embedding vector similar to the text embedding vector to be recognized.

10. The large model-based voice control implementation system according to claim 6, characterized in that: For the control information output by the pre-trained large language model, the text parsing module is used to fill the control information into the pre-configured JSON template to obtain the control information in JSON format, and use the text embedding vector corresponding to the current user voice command and the control information in JSON format as a key-value pair, and cache the key-value pair in the vector database; The JSON module includes fields related to the operation type, target device type, and specific operation parameter settings.

Images