An intelligent interactive system and method integrating language recognition and large language model

Abstract

The present invention relates to an intelligent interactive system and method for integrating language recognition and a large language model. In the intelligent interactive system, a stored vector database can be provided by setting a storage device; a command function module can be provided to obtain corresponding voiceprint features, understanding commands and action commands according to input voice, and write into a historical command knowledge base, thereby updating the content of the vector database. Since the command function module first corrects the text information through the error correction unit of the large language model when obtaining the action command, the accuracy rate can be improved; then semantic analysis is performed to obtain the understanding command to ensure that multiple intentions can be understood; then, the action command for controlling the object to be controlled is generated according to the matching results of the voiceprint features, the understanding command, the place name address database, the historical command knowledge base and the voice command database, and the matching rate can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of voice commands and large language models, and in particular to an intelligent interaction system and method integrating language recognition and a large language model. Background Art

[0002] The process of traditional voice interaction analysis methods includes: converting voice data into text content, analyzing the text content according to the preset grammatical structure to form a fixed instruction structure; mapping the instruction structure with the control code to generate an executable control code; sending the control code to the running program for execution. Among them, the conversion of voice data into text content is affected by whether each person's expression is accurate and clear, and the result is often different from the content said; the analysis of the grammatical structure of the text content is affected by whether the text content conforms to the fixed structure, and whether the matching algorithm understands its intention, and the result often has a certain degree of inaccuracy.

[0003] In this regard, patent publication number CN105070288B discloses a method and device for in-vehicle voice command recognition, which uses a deep learning model to judge the user's emotions, then combines the emotions with the context scene to infer the user's intentions, and selects possible commands to execute based on the user's intentions. This method of combining emotions with context has a certain degree of uncertainty and a relatively low accuracy rate.

[0004] Patent publication number CN108074565A discloses a system and method for voice command jump and detailed command execution. It converts voice data into text, then performs word segmentation and grammatical analysis on the text, and finally determines the matching instructions based on the part of speech and meaning of each word. In the short voice-to-text scenario, the error rate of recognized text is relatively high. This method lacks correction of voice text and judges based on the part of speech of the word segmentation, relying on the accuracy of the word segmentation. However, some fields have proper nouns, which may lead to incorrect segmentation and reduce the matching rate.

[0005] It can be seen that the voice interaction analysis solutions in the prior art have the problem that command recognition tends to be specific to a certain field and can only recognize a limited number of voice commands, and have technical defects such as low accuracy and unsatisfactory matching rate. Summary of the invention

[0006] The technical problem to be solved by the present invention is how to overcome the technical defects of the existing voice interaction analysis solutions, which have the problem of command recognition being biased towards specific fields, and having low accuracy and unsatisfactory matching rates. In order to overcome the above defects of the existing technology, the present invention provides an intelligent interaction system and method integrating language recognition and a large language model, including an intelligent interaction system integrating language recognition and a large language model and an intelligent interaction method integrating language recognition and a large language model.

[0007] The present invention provides an intelligent interactive system integrating language recognition and a large language model, the intelligent interactive system comprising:

[0008] A storage device, used for constructing and / or storing a vector database, wherein the vector database includes a place name address database, a historical instruction knowledge base, and a voice instruction database;

[0009] The instruction function module is configured to extract voiceprint features and text information from the input voice, and perform error correction on the text information through the error correction unit of the large language model, and then perform semantic analysis to obtain an understanding instruction, and then generate an action instruction for controlling a desired object according to the matching results of the voiceprint features, the understanding instruction, the place name address database, the historical instruction knowledge base, and the voice instruction database, and finally write the voiceprint features, the understanding instruction, and the action instruction into the historical instruction knowledge base to be stored in the storage device;

[0010] The instruction function module is electrically connected to the storage device.

[0011] The intelligent interactive system of language recognition and large language model fusion disclosed by the present invention can provide a stored vector database by setting a storage device; by setting an instruction function module, the corresponding voiceprint features, understanding instructions and action instructions can be obtained according to the input voice, and written into the historical instruction knowledge base, so as to update the content of the vector database. Because the instruction function module first corrects the text information through the error correction part of the large language model when obtaining the action instruction, the accuracy can be improved; then the semantic analysis is performed to obtain the understanding instruction to ensure that multiple intentions can be understood; then the action instruction for controlling the object to be controlled is generated according to the matching results of the voiceprint features, the understanding instruction and the place name address database, the historical instruction knowledge base and the voice instruction database, and the matching rate can be improved. Therefore, not only can multiple intentions be understood to form continuous instructions, but also the intentions across scenes (such as office scenes, sports scenes, and travel scenes) can be understood, which is superior to the traditional grammatical structure analysis scheme, and at the same time makes up for the defect that the traditional method can only execute one command and has no system interaction prompts.

[0012] In a possible implementation, the instruction function module includes:

[0013] A voice dialogue module, configured to obtain audio data and the voiceprint feature according to the input voice, and convert the audio data into the text information using a voice recognition algorithm;

[0014] A word correction module is configured to correct the text information through the error correction unit of the large language model to obtain an error correction result;

[0015] An instruction extraction module is configured to perform semantic analysis on the error correction result through a semantic analysis unit of a large language model, extract instruction names, input parameters and scenario keywords to obtain the understanding instruction;

[0016] The instruction expansion module is configured to perform an extended interpretation on the error correction result through an extended interpretation unit of the large language model to obtain an extended voice instruction;

[0017] The instruction optimization module is configured to form a multiple-choice question through the language organization part of the large language model for the user to answer;

[0018] An instruction matching module is configured to generate an action instruction for controlling a desired object according to a matching result of the voiceprint feature, the understood instruction, the place name address database, the historical instruction knowledge base, and the voice instruction database;

[0019] an instruction execution module, configured to control the object to be controlled according to the action instruction, and write the voiceprint feature, the understanding instruction and the action instruction into the historical instruction knowledge base to store in the storage device;

[0020] in,

[0021] The word correction module is electrically connected to the voice dialogue module, the instruction extraction module is electrically connected to the word correction module, the instruction matching module is electrically connected to the instruction extraction module, the instruction expansion module, the instruction optimization module and the instruction execution module are all electrically connected to the instruction matching module, and the instruction execution module is electrically connected to the storage device;

[0022] Compared with the existing technology, the above technical solution can ensure the acquisition, error correction and analysis of text information. For texts with unclear meanings, semantic expansion can be performed through the instruction expansion module or the user can choose through the instruction optimization module, thereby ensuring the accuracy of speech analysis and improving efficiency.

[0023] In a possible implementation, the voice dialogue module includes:

[0024] An audio device, configured to allow a user to input voice, obtain audio data, and perform voiceprint recognition on the user's input voice to obtain the voiceprint feature;

[0025] A speech recognition device, configured to convert the audio data into the text information through a speech recognition algorithm;

[0026] in,

[0027] The speech recognition device is electrically connected to the audio device, and the word correction module is electrically connected to the speech recognition device.

[0028] This solution can ensure the execution of text information acquisition and voiceprint information collection, and further ensure the efficient operation of text information acquisition, error correction and analysis.

[0029] In a possible implementation, the voice dialogue module further includes:

[0030] A speech synthesis device, configured to convert text content into artificial speech according to the conversion instruction;

[0031] A playback device configured to play the artificial voice;

[0032] in,

[0033] The instruction matching module is electrically connected to the speech synthesis device, and the playback device is electrically connected to the speech synthesis device and the audio device;

[0034] This solution not only diversifies the functions of the voice dialogue module by adding a speech synthesis device and a playback device, but also ensures the improvement of the matching efficiency of the command matching module.

[0035] In a possible implementation, the instruction matching module is configured to obtain the action instruction by executing the following steps:

[0036] A1: searching the historical instruction knowledge base to determine whether the understood instruction and the voiceprint feature match the content in the historical instruction knowledge base;

[0037] If yes, extracting the action instruction from the understanding instruction and transmitting it to the instruction execution module;

[0038] If not, proceed to the next step;

[0039] A2: Determine the matching status between the understanding instruction and the content of the voice instruction database;

[0040] If there is no matching item, the instruction expansion module is called to perform an extended interpretation on the error correction result, obtain an extended voice instruction, and then execute the next step;

[0041] If there is a matching item, the action instruction extracted from the understanding instruction is used as the first action instruction, and then step A5 is executed;

[0042] If there are multiple matching items, the instruction optimization module is called to form a multiple-choice question with multiple action options for the user to answer, and then the action selected by the user is used as the first action instruction, and then step A5 is executed;

[0043] A3: calling the instruction extraction module to perform semantic analysis on the extended voice instruction, extracting the instruction name, input parameters and scene keywords to obtain an understanding result, and using the understanding result as the current understanding instruction, and then executing the next step;

[0044] A4: Searching the historical instruction knowledge base to determine whether the currently understood instruction and the voiceprint feature match the content in the historical instruction knowledge base;

[0045] If yes, extracting the action instruction from the current understanding instruction and transmitting it to the instruction execution module;

[0046] If not, outputting a conversion instruction for converting the text content prompting the user to re-input the speech into artificial speech to the speech synthesis device, and then returning to execute step A1 until the next understanding instruction is obtained;

[0047] A5: Determine whether the instruction type of the first action instruction is a positioning instruction;

[0048] If yes, proceed to the next step;

[0049] If not, the first action instruction is used as an action instruction and transmitted to the instruction execution module;

[0050] A6: extracting an input parameter from the first action instruction as a place name address, searching the place name address database using a vector index algorithm to obtain corresponding geographic coordinates, and replacing the input parameter in the first action instruction with the geographic coordinates;

[0051] A7: taking the first action instruction replaced by the step A6 as the action instruction and transmitting it to the instruction execution module;

[0052] The operation of the command matching module is set to the above form. First, a search is performed in the historical command knowledge base to find the command that matches the understanding command and voiceprint features. If it can be found, the action command can be directly extracted. If it cannot be found, the match between the understanding command and the content of the voice command database is determined, and then it is determined whether it is a positioning command. This can further understand multiple intentions and form continuous instructions. It can also further understand intentions across scenarios (such as office scenarios, sports scenarios, and travel scenarios), further making up for the defect that the traditional method can only execute one command without system interaction prompts.

[0053] In a possible implementation manner, the instruction matching module is configured to: determine whether the understood instruction and the voiceprint feature match the content in the historical instruction knowledge base, including the following steps:

[0054] A11: Calculate the similarity between the instruction name in the understood instruction and the instruction name in the historical instruction knowledge base through a semantic matching model algorithm to obtain a first score;

[0055] A12: Calculate the similarity between the voiceprint feature and the voiceprint feature in the historical instruction knowledge base through a vector similarity algorithm to obtain a second score;

[0056] A13: Calculate the supply-side matching of the scene keywords in the understanding instruction and the scene in the historical instruction knowledge base through a semantic matching model algorithm to obtain a third score;

[0057] A14: weighting the first score, the second score, and the third score to obtain an instruction matching score;

[0058] A15: Determine whether the instruction matching score is not less than a threshold;

[0059] If so, the match is determined to be yes;

[0060] If not, the match is determined to be no;

[0061] This solution can realize retrieval in the historical command knowledge base to find commands that are similar in semantics to the understood commands and have similar voiceprint features, thereby ensuring the improvement of the matching efficiency and accuracy of the command matching module.

[0062] Another technical solution of the present invention is to provide an intelligent interaction method for integrating language recognition and a large language model, comprising the following steps:

[0063] S1: Collecting the user's input voice through the voice dialogue module to obtain audio data and voiceprint features, and converting the audio data into text information using a voice recognition algorithm;

[0064] S2: Correcting errors in the text information by a word correction module to obtain an error correction result;

[0065] S3: semantically analyzing the error correction result through an instruction extraction module, extracting instruction names, input parameters and scenario keywords to obtain understanding instructions;

[0066] S4: Generate, by a command matching module, an action command for controlling the object to be controlled according to the matching results of the voiceprint feature, the understood command, the place name address database, the historical command knowledge base and the voice command database;

[0067] S5: controlling the object to be controlled according to the action instruction through the instruction execution module, and writing the voiceprint feature, the understanding instruction and the action instruction into the historical instruction knowledge base to store in the storage device.

[0068] The method disclosed in the present invention, after obtaining text information and voiceprint features, first corrects the text information through the word correction module to improve the accuracy; then performs semantic analysis through the instruction extraction module to obtain the understanding instruction to ensure that multiple intentions can be understood; then generates action instructions for controlling the object to be controlled based on the matching results of the voiceprint features, the understanding instructions and the place name address database, the historical instruction knowledge base and the voice instruction database, and the matching rate can be improved. Therefore, not only can multiple intentions be understood to form continuous instructions, but also intentions across scenes (such as office scenes, sports scenes, and travel scenes) can be understood, which is superior to the traditional grammatical structure analysis solution, and at the same time makes up for the defect that the traditional method can only execute one command without system interactive prompts.

[0069] In a possible implementation, step S1 includes the following steps:

[0070] S11: When the user inputs voice, audio data is obtained through the audio device of the voice dialogue module, and voiceprint recognition is performed on the user's input voice to obtain the voiceprint feature;

[0071] S12: converting the audio data into the text information through the voice recognition device of the voice dialogue module;

[0072] This solution can ensure the orderly execution of text information acquisition and voiceprint information collection, making text information acquisition, error correction and analysis more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of the structure of an intelligent interactive system integrating language recognition and a large language model disclosed in an embodiment of the present application;

[0074] Figure 2 This is a schematic diagram of the instruction function module structure disclosed in the embodiment of the present application;

[0075] Figure 3 This is a schematic diagram of the structure of the voice dialogue module disclosed in the embodiment of the present application;

[0076] Figure 4 This is a flowchart of the operation of the instruction matching module disclosed in the embodiment of the present application;

[0077] Figure 5 A flowchart of the instruction matching module disclosed in the embodiment of the present application determining whether the understood instruction and the voiceprint feature match the content in the historical instruction knowledge base;

[0078] Figure 6 This is a flow chart of the method disclosed in the embodiment of the present application. DETAILED DESCRIPTION

[0079] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present application, and are not intended to limit the protection scope of the embodiments of the present application. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0080] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "electrical connection" and "electrical connection relationship" should be understood in a broad sense, that is, referring to a connection method with an electrical relationship, for example, it can be a circuit connection through a conductive wire, or it can be an electrical connection through a radio signal channel (channel), or a combination of the two. In addition, "electrical connection" and "electrical connection relationship" can be based on mechanical connection (such as a conductive wire set in a connecting key); it can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0081] The present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0082] See also Figure 1 to Figure 5 As shown, the embodiment of the present application discloses an intelligent interactive system integrating language recognition and a large language model, see Figure 1 As shown, the intelligent interactive system includes a storage device and an instruction function module, wherein the instruction function module is electrically connected to the storage device.

[0083] In the intelligent interactive system, the storage device is used to construct and / or store a vector database. In this embodiment, the vector database includes a place name and address database, a historical instruction knowledge base, and a voice instruction database. The instruction function module is configured to extract voiceprint features and text information from the input voice, and correct the text information through the error correction unit of the large language model, and then perform semantic analysis to obtain understanding instructions, and then generate action instructions for controlling the object to be controlled based on the matching results of the voiceprint features, understanding instructions, place name and address database, historical instruction knowledge base, and voice instruction database, and finally write the voiceprint features, understanding instructions, and action instructions into the historical instruction knowledge base to be stored in the storage device.

[0084] The content of the vector database is diverse. To facilitate understanding by those skilled in the art, an example is given below to illustrate the content of the vector database.

[0085] Example 1: A common vector database consists of a place name and address database, a historical command knowledge base, and a voice command database. The place name and address data includes the following:

[0086] (1) Standard name: the official standard name of a place name, which is the core information of the place name;

[0087] (2) Chinese address: the specific Chinese address of the place name;

[0088] (3) Geographic coordinates: including longitude and latitude information corresponding to place names;

[0089] (4) Extended information: the administrative level to which the place name belongs, such as province, municipality, county, etc.; and the cultural connotation of the place name, such as historical evolution, cultural background, etc.

[0090] The historical instruction knowledge base includes the following:

[0091] (1) Voiceprint features: features generated using voiceprint feature algorithms;

[0092] (2) Original speech text: the instructions after the collected audio data has been processed through speech recognition and error correction (e.g., word correction);

[0093] (3) Original voice command: refers to the voice command obtained by processing the original voice text through the command extraction module.

[0094] The system voice command is a correct voice command with high confidence or confirmed by the user, corresponding to a command item in the voice command database; the voice command database includes the following contents:

[0095] (1) Instruction name: the Chinese name corresponding to the execution instruction function;

[0096] (2) Input parameters: Input parameters required to execute the instruction function, used to control the behavior of the program;

[0097] (3) Functional description: extended explanation of the command name, detailed description of the function, detailed display of the operation steps, and specific definition of the input parameters.

[0098] See also Figure 2 As shown, in this embodiment, the instruction function module includes a voice dialogue module, a word correction module, an instruction extraction module, an instruction expansion module, an instruction optimization module, an instruction matching module and an instruction execution module, wherein the word correction module is electrically connected to the voice dialogue module, the instruction extraction module is electrically connected to the word correction module, the instruction matching module is electrically connected to the instruction extraction module, the instruction expansion module, the instruction optimization module and the instruction execution module are all electrically connected to the instruction matching module, and the instruction execution module is electrically connected to the storage device.

[0099] In the instruction function module, the voice dialogue module is configured to obtain audio data and voiceprint features based on the input voice, and convert the audio data into text information using a voice recognition algorithm.

[0100] See also Figure 3As shown, in this embodiment, the voice dialogue module includes an audio device, a voice recognition device, a voice synthesis device and a playback device, wherein the voice recognition device is electrically connected to the audio device, and the word correction module is electrically connected to the voice recognition device; the instruction matching module is electrically connected to the voice synthesis device, and the playback device is electrically connected to the voice synthesis device and the audio device.

[0101] In the voice dialogue module, the audio device is configured to allow the user to input voice, obtain audio data, and perform voiceprint recognition on the user's input voice to obtain voiceprint features. The voice recognition device is configured to convert the audio data into text information through a voice recognition algorithm. The voice synthesis device is configured to convert the text content into artificial voice according to the conversion instruction. The playback device is configured to play the artificial voice.

[0102] The large language model used in this embodiment includes an error correction unit, a semantic analysis unit, an extensible interpretation unit, and a language organization unit with optimized parameters. In the instruction function module, the word correction module is configured to correct text information through the error correction unit of the large language model to obtain error correction results. The error correction unit is a text correction intelligent entity or intelligent device built based on the prompt words of the large language model. It uses the context understanding ability of the large language model to correct words for input sentences with obvious errors in semantic understanding. The error correction unit is embedded in the word correction module as a member of the internal structure of the word correction module. In addition, the word correction module also includes a calling device corresponding to the error correction unit, which is used to call the error correction unit to run after receiving the text information.

[0103] In the instruction function module, the instruction extraction module is configured to perform semantic analysis on the error correction result through the semantic analysis unit, extract the instruction name, input parameters and scene keywords to obtain the understanding instruction. The semantic analysis unit is an intelligent semantic analysis device built based on the large language model prompt words, which is used to perform semantic analysis on the input voice text, extract the instruction name, input parameters and scene keywords, and generate the understanding instruction. As a member of the internal structure of the instruction extraction module, the instruction extraction module also includes a calling device corresponding to the semantic analysis unit, which is used to call the semantic analysis unit to run after receiving the error correction result.

[0104] In the instruction function module, the instruction expansion module is configured to perform an extended interpretation of the error correction result through the extended interpretation unit to obtain an extended voice instruction. The extended interpretation unit is a voice instruction intelligent device built based on the large language model prompt words, which is responsible for performing an extended interpretation of the input voice instruction and outputting the extended voice instruction. The extended interpretation unit is a member of the internal structure of the instruction expansion module. The instruction expansion module also includes an intelligent body corresponding to the extended interpretation unit, which is used to call the extended interpretation unit to perform an extended interpretation on the object after receiving an instruction that requires semantic expansion of an object, and obtain the corresponding extended voice instruction.

[0105] In the instruction function module, the instruction optimization module is configured to form a multiple-choice question through the language organization unit for the user to answer. The language organization unit is a large language model intelligent entity, which is used to organize the multiple voice instructions input by language to form a multiple-choice question and transmit it to the instruction matching module. The instruction matching module outputs it to the speech synthesis device of the dialogue module, prompts the user to make a single choice in the form of voice, and outputs the preferred instruction. As a member of the internal structure of the instruction optimization module, the instruction optimization module also includes a calling device corresponding to the language organization unit, which is used to call the language organization unit to run after receiving an instruction that requires the user to make a choice.

[0106] In the command function module, the command matching module is configured to generate action commands for controlling the object to be controlled based on the matching results of voiceprint features, understanding commands and place name address database, historical command knowledge base and voice command database.

[0107] Specifically, using the database of Example 1, see Figure 4 As shown, the instruction matching module is configured to obtain the action instruction by executing the following steps:

[0108] A1: Search the historical command knowledge base to determine whether the understood command and voiceprint features match the content in the historical command knowledge base;

[0109] If yes, extracting the action instruction from the understanding instruction and transmitting it to the instruction execution module;

[0110] If not, proceed to the next step.

[0111] For the process of determining whether the understood command and voiceprint features match the content in the historical command knowledge base, see Figure 5 As shown, the process includes the following steps:

[0112] A11: The semantic matching model algorithm is used to calculate the similarity between the instruction name in the understanding instruction and the instruction name in the historical instruction knowledge base to obtain the first score.

[0113] A12: The voiceprint feature is calculated by a vector similarity algorithm and matched with the voiceprint feature in the historical command knowledge base to obtain a second score.

[0114] A13: The third score is obtained by calculating the semantic matching model algorithm to understand the scene keywords in the instructions and matching the scene supply side of the historical instruction knowledge base.

[0115] A14: The first score, the second score, and the third score are weighted to obtain an instruction matching score; the weight configuration may be set according to a specific situation.

[0116] A15: Determine whether the instruction matching score is not less than a threshold;

[0117] If so, the match is determined to be yes;

[0118] If not, the match is determined to be no.

[0119] A2: Determine the match between the understood command and the content of the voice command database;

[0120] If there is no match, the command extension module is called to perform an extended interpretation of the error correction result, obtain the extended voice command, and then execute the next step;

[0121] If there is a matching item, the action instruction extracted from the understanding instruction is used as the first action instruction, and then step A5 is executed;

[0122] If there are multiple matching items, the instruction optimization module is called to form a multiple-choice question with multiple action options for the user to answer, and then the user's action is used as the first action instruction, and then step A5 is executed.

[0123] A3: Call the command extraction module to perform semantic analysis on the extended voice command, extract the command name, input parameters and scene keywords to obtain the understanding result, and use the understanding result as the current understanding command, and then execute the next step.

[0124] A4: Search the historical command knowledge base to determine whether the currently understood command and voiceprint features match the content in the historical command knowledge base;

[0125] If yes, the action instruction is extracted from the current understanding instruction and passed to the instruction execution module;

[0126] If not, a conversion instruction for converting the text content prompting the user to re-input the voice into artificial voice is output to the voice synthesis device, and then the step A1 is executed again until the next understanding instruction is obtained.

[0127] A5: Determine whether the instruction type of the first action instruction is a positioning instruction;

[0128] If yes, proceed to the next step;

[0129] If not, the first action instruction is used as the action instruction and transmitted to the instruction execution module.

[0130] A6: Extract the input parameter from the first action instruction as the place name address, use the vector index algorithm to search the place name address database to obtain the corresponding geographic coordinates, and replace the input parameter in the first action instruction with the geographic coordinates.

[0131] A7: The first action instruction replaced by step A6 is used as the action instruction and transmitted to the instruction execution module.

[0132] In the command function module, the command execution module is configured to control the object to be controlled according to the action command, and write the voiceprint features, understanding commands and action commands into the historical command knowledge base to store them in the storage device. The specific process can be: first check whether the command name of the action command is consistent with the corresponding preset command name to ensure that the input parameters meet the specifications. If they are inconsistent, issue a command to re-input the voice. If they are consistent, send the command corresponding to the action command to the object to be controlled to execute.

[0133] See also Figure 6 As shown below, a large-screen visual interactive system is taken as the object to be controlled. At this time, the contents of the place name and address database, the historical command knowledge base, and the voice command database of the vector database are the contents corresponding to the large-screen visual interactive system, which are recorded as place name and address database A, historical command knowledge base B, and voice command database C, respectively. Figure 6 As shown in the shaded box.

[0134] Using the intelligent interactive system integrating language recognition and large language model disclosed in this embodiment, the large-screen visual interactive system is controlled according to the following steps:

[0135] S1: The user's input voice is collected through the voice dialogue module to obtain audio data and voiceprint features, and the audio data is converted into text information using the voice recognition algorithm.

[0136] Specifically, step S1 includes the following steps:

[0137] S11: When the user inputs a control voice for the large-screen visual interactive system, audio data is obtained through the audio device of the voice dialogue module, and voiceprint recognition is performed on the user's input voice to obtain voiceprint features.

[0138] S12: The audio data is converted into text information through the voice recognition device of the voice dialogue module.

[0139] S2: Correct the text information through the word correction module to obtain the error correction result.

[0140] S3: The error correction results are semantically parsed through the instruction extraction module to extract the instruction name, input parameters and scenario keywords to obtain understood instructions.

[0141] S4: Generate action instructions for controlling the large-screen visual interactive system through the instruction matching module according to the voiceprint features, the understanding instructions, the matching results of the place name address database A, the historical instruction knowledge base B and the voice instruction database C. The specific process can be seen in steps A1 to A7, which will not be described here.

[0142] S5: The object to be controlled is controlled according to the action command through the command execution module, and the voiceprint features, understanding commands and action commands are written into the historical command knowledge base B to be stored in the storage device.

[0143] The intelligent interactive system that integrates language recognition and a large language model disclosed in this embodiment can provide a stored vector database by setting a storage device; by setting an instruction function module, it can obtain corresponding voiceprint features, understand instructions and action instructions according to the input voice, and write into the historical instruction knowledge base, thereby updating the content of the vector database. The instruction function module includes a voice dialogue module, a word correction module, an instruction extraction module, an instruction expansion module, an instruction optimization module, an instruction matching module and an instruction execution module. When the instruction function module obtains the action instruction, it first corrects the text information through the error correction part of the word correction module, so as to improve the accuracy; then the instruction extraction module performs semantic analysis to obtain the understanding instruction to ensure that multiple intentions can be understood; then the instruction matching module generates the action instruction for controlling the object to be controlled according to the matching results of the voiceprint features, the understanding instruction and the place name address database, the historical instruction knowledge base and the voice instruction database, so that the matching rate can be improved, so that not only can multiple intentions be understood to form continuous instructions, but also intentions across scenarios (such as office scenarios, sports scenarios, and travel scenarios) can be understood, so it is better than the traditional grammatical structure analysis scheme, and at the same time makes up for the defect that the traditional method can only execute one command and has no system interaction prompts.

[0144] In the description of the embodiments of the present application, it should be noted that in the description of the present application, terms such as "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is only for the convenience of description, and does not indicate or imply that the device or component must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present application.

[0145] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0146] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An intelligent interactive system integrating language recognition and a large language model, characterized in that: include: A storage device, used for constructing and / or storing a vector database, wherein the vector database includes a place name address database, a historical instruction knowledge base, and a voice instruction database; The instruction function module is configured to extract voiceprint features and text information from the input voice, and perform error correction on the text information through the error correction unit of the large language model, and then perform semantic analysis to obtain an understanding instruction, and then generate an action instruction for controlling a desired object according to the matching results of the voiceprint features, the understanding instruction, the place name address database, the historical instruction knowledge base, and the voice instruction database, and finally write the voiceprint features, the understanding instruction, and the action instruction into the historical instruction knowledge base to be stored in the storage device; in, The process of generating an action instruction for controlling a desired object according to the matching results of the voiceprint feature, the understanding instruction, the place name and address database, the historical instruction knowledge base, and the voice instruction database comprises the following steps: R1: searching the historical instruction knowledge base to determine whether the understood instruction and the voiceprint feature match the content in the historical instruction knowledge base; If yes, extracting the action instruction from the understanding instruction; If not, proceed to the next step; R2: Determine the matching status between the understanding instruction and the content of the voice instruction database; If there is no match, an extended interpretation is performed on the error correction result to obtain an extended voice instruction, and then the next step is executed; If there is a matching item, the action instruction extracted from the understanding instruction is used as the first action instruction, and then step R5 is executed; If there are multiple matching items, a multiple-choice question with multiple action options is formed for the user to answer, and then the action selected by the user is used as the first action instruction, and then step R5 is executed; R3: semantically parse the extended voice command, extract the command name, input parameters and scene keywords to obtain an understanding result, and use the understanding result as the current understanding command, and then execute the next step; R4: Searching the historical instruction knowledge base to determine whether the currently understood instruction and the voiceprint feature match the content in the historical instruction knowledge base; If yes, extract the action instruction from the current understanding instruction and transmit it to the instruction execution module; If not, output a conversion instruction for converting the text content prompting the user to re-input the voice into artificial voice, and then return to execute step R1 until the next understanding instruction is obtained; R5: Determine whether the instruction type of the first action instruction is a positioning instruction; If yes, proceed to the next step; If not, taking the first action instruction as the action instruction; R6: extracting an input parameter from the first action instruction as a place name address, searching the place name address database using a vector index algorithm to obtain corresponding geographic coordinates, and replacing the input parameter in the first action instruction with the geographic coordinates; R7: taking the first action instruction replaced by step R6 as the action instruction; The instruction function module is electrically connected to the storage device.

2. The intelligent interactive system integrating language recognition and large language model according to claim 1, characterized in that: The instruction function module includes: A voice dialogue module, configured to obtain audio data and the voiceprint feature according to the input voice, and convert the audio data into the text information using a voice recognition algorithm; A word correction module is configured to correct the text information through the error correction unit of the large language model to obtain an error correction result; An instruction extraction module is configured to perform semantic analysis on the error correction result through a semantic analysis unit of a large language model, extract instruction names, input parameters and scenario keywords to obtain the understanding instruction; The instruction expansion module is configured to perform an extended interpretation on the error correction result through an extended interpretation unit of the large language model to obtain an extended voice instruction; The instruction optimization module is configured to form a multiple-choice question through the language organization part of the large language model for the user to answer; An instruction matching module is configured to generate an action instruction for controlling a desired object according to a matching result of the voiceprint feature, the understood instruction, the place name address database, the historical instruction knowledge base, and the voice instruction database; an instruction execution module, configured to control the object to be controlled according to the action instruction, and write the voiceprint feature, the understanding instruction and the action instruction into the historical instruction knowledge base to store in the storage device; in, The word correction module is electrically connected to the voice dialogue module, the instruction extraction module is electrically connected to the word correction module, the instruction matching module is electrically connected to the instruction extraction module, the instruction expansion module, the instruction optimization module and the instruction execution module are all electrically connected to the instruction matching module, and the instruction execution module is electrically connected to the storage device.

3. The intelligent interactive system integrating language recognition and large language model according to claim 2, characterized in that: The voice dialogue module comprises: An audio device, configured to allow a user to input voice, obtain audio data, and perform voiceprint recognition on the user's input voice to obtain the voiceprint feature; A speech recognition device, configured to convert the audio data into the text information through a speech recognition algorithm; in, The speech recognition device is electrically connected to the audio equipment, and the word correction module is electrically connected to the speech recognition device.

4. The intelligent interactive system integrating language recognition and large language model according to claim 3, characterized in that: The voice dialogue module also includes: A speech synthesis device, configured to convert text content into artificial speech according to the conversion instruction; A playback device configured to play the artificial voice; in, The instruction matching module is electrically connected to the speech synthesis device, and the playback device is electrically connected to the speech synthesis device and the audio device.

5. The intelligent interactive system integrating language recognition and large language model according to claim 4, characterized in that: The instruction matching module is configured to: determine whether the understood instruction and the voiceprint feature match the content in the historical instruction knowledge base, including the following steps: A11: Calculate the similarity between the instruction name in the understood instruction and the instruction name in the historical instruction knowledge base through a semantic matching model algorithm to obtain a first score; A12: Calculate the similarity between the voiceprint feature and the voiceprint feature in the historical instruction knowledge base through a vector similarity algorithm to obtain a second score; A13: Calculate the supply-side matching of the scene keywords in the understanding instruction and the scene in the historical instruction knowledge base through a semantic matching model algorithm to obtain a third score; A14: weighting the first score, the second score, and the third score to obtain an instruction matching score; A15: Determine whether the instruction matching score is not less than a threshold; If so, the match is determined to be yes; If not, the match is determined to be no.

6. An intelligent interactive method for integrating language recognition and a large language model, characterized in that: The intelligent interactive system based on the language recognition and large language model fusion according to any one of claims 1 to 5 comprises the following steps: S1: Collecting the user's input voice through the voice dialogue module to obtain audio data and voiceprint features, and converting the audio data into text information using a voice recognition algorithm; S2: Correcting errors in the text information by a word correction module to obtain an error correction result; S3: semantically analyzing the error correction result through an instruction extraction module, extracting instruction names, input parameters and scenario keywords to obtain understanding instructions; S4: Generate, by a command matching module, an action command for controlling the object to be controlled according to the matching results of the voiceprint feature, the understood command, the place name address database, the historical command knowledge base and the voice command database; S5: controlling the object to be controlled according to the action instruction by the instruction execution module, and writing the voiceprint feature, the understanding instruction and the action instruction into the historical instruction knowledge base to store in the storage device; The step S4 comprises the following steps: S41: searching the historical instruction knowledge base to determine whether the understood instruction and the voiceprint feature match the content in the historical instruction knowledge base; If yes, extracting the action instruction from the understanding instruction; If not, proceed to the next step; S42: Determine the matching status between the understanding instruction and the content of the voice instruction database; If there is no match, an extended interpretation is performed on the error correction result to obtain an extended voice instruction, and then the next step is executed; If there is a matching item, the action instruction extracted from the understanding instruction is used as the first action instruction, and then step S45 is executed; If there are multiple matching items, a multiple-choice question with multiple action options is formed for the user to answer, and then the action selected by the user is used as the first action instruction, and then step S45 is executed; S43: semantically analyze the extended voice command, extract the command name, input parameters and scene keywords to obtain an understanding result, and use the understanding result as the current understanding command, and then execute the next step; S44: searching the historical instruction knowledge base to determine whether the currently understood instruction and the voiceprint feature match the content in the historical instruction knowledge base; If yes, extracting the action instruction from the current understanding instruction and transmitting it to the instruction execution module; If not, output a conversion instruction for converting the text content prompting the user to re-input the voice into artificial voice, and then return to execute the step S41 until the next understanding instruction is obtained; S45: Determine whether the instruction type of the first action instruction is a positioning instruction; If yes, proceed to the next step; If not, taking the first action instruction as the action instruction; S46: extracting an input parameter from the first action instruction as a place name address, searching the place name address database using a vector index algorithm to obtain corresponding geographic coordinates, and replacing the input parameter in the first action instruction with the geographic coordinates; S47: Use the first action instruction replaced by step S46 as the action instruction.

7. The intelligent interaction method for integrating language recognition and a large language model according to claim 6, characterized in that: The step S1 comprises the following steps: S11: When the user inputs voice, audio data is obtained through the audio device of the voice dialogue module, and voiceprint recognition is performed on the user's input voice to obtain the voiceprint feature; S12: Converting the audio data into the text information through the voice recognition device of the voice dialogue module.

Images