Intelligent voice interaction method and device based on large language model

Abstract

The application provides an intelligent voice interaction method and device based on a large language model, and relates to the technical field of computers.The method comprises the following steps: converting target voice input by a user and received into target question text based on a voice recognition module; inputting the target question text and a pre-defined intent prompt word template into a first large language model to obtain an intent rating of the target question text; matching a second large language model and a third large language model; inputting the target question text and a keyword prompt word template into the third large language model to obtain an abstract and keywords; and inputting the target question text and historical context related to the target question text into the second large language model to obtain a high-quality text stream.Through multi-level model division and cooperation, a dynamic balance is achieved among resource consumption, generation quality and response speed.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, more particularly, to an intelligent voice interaction method and device based on large language model. BACKGROUND

[0002] At present, intelligent voice interaction systems based on large language models (LLM) have been widely applied in many fields, but their core technical architecture still faces the following limitations:

[0003] Single model static selection mechanism: Most systems use fixed large models or lightweight models to process all requests. For example, relying only on large models such as GPT-3 to achieve high-precision generation results in high response delay (often more than 1.5 seconds); or using lightweight models (such as DistilBERT) to sacrifice accuracy for speed, which drastically reduces accuracy in complex intent scenarios (measured accuracy is less than 60%). SUMMARY

[0004] Therefore, the present disclosure provides the following solutions to solve the above problems.

[0005] In a first aspect, the present disclosure provides an intelligent voice interaction method based on a large language model, comprising:

[0006] Converting a target voice input by a user received by a voice recognition module into a target question text;

[0007] Inputting the target question text and a pre-defined intent prompt word template into a first large language model to obtain an intent rating of the target question text, the intent rating indicating the level of accuracy requirement; wherein a plurality of levels of large language models are pre-determined, each level of large language model having a different size, and the larger the large language model, the higher the accuracy of the content output by the large language model, and the first large language model being the most lightweight one among the plurality of levels of large language models;

[0008] Based on the intent rating and the target question text, matching a second large language model and a third large language model among the plurality of levels of large language models, the second large language model being a main model and the third large language model being an auxiliary model;

[0009] Inputting the target question text and a keyword prompt word template into the third large language model to obtain an abstract and keywords;

[0010] Generating a first output result based on the abstract and the keywords and outputting the first output result by voice;

[0011] Inputting the target question text and a historical context related to the target question text into the second large language model to obtain a high-quality text stream;

[0012] The high-quality text stream, the summary, and the keywords are fused to obtain a second output result. After the first output result is played, the second output result is continued to be output by voice.

[0013] Secondly, the present invention provides an intelligent voice interaction device based on a large language model, comprising:

[0014] The conversion module is used to convert the target speech input by the user into target question text based on the speech recognition module.

[0015] The input module is used to input the target question text and the predefined intent prompt word template into the first large language model to obtain the intent rating of the target question text. The intent rating is used to indicate the level of accuracy required. There are multiple levels of large language models in advance, and the size of each level of large language model is different. The larger the large language model, the higher the accuracy of the output content. The first large language model is the lightest one among the multiple levels of large language models.

[0016] The matching module is used to match the second and third major language models in multiple levels of major language models based on the intent rating and the target question text, with the second major language model serving as the main model and the third major language model serving as the auxiliary model.

[0017] The input module is also used to input the target question text and keyword prompt template into the third language model to obtain a summary and keywords;

[0018] The output module is used to generate a first output result based on the summary and keywords and then output it via voice.

[0019] The input module is also used to input the target question text and the historical context related to the target question text into the second large language model to obtain a high-quality text stream;

[0020] The output module is further configured to fuse the high-quality text stream, the summary, and the keywords to obtain a second output result, and continue to output the second output result by voice after the first output result has finished playing.

[0021] In an optional implementation, the output module is specifically used for:

[0022] The abstract and keywords are input into the first large language model for fusion to obtain the first output result.

[0023] The intelligent voice interaction method and device based on a large language model provided in this application have the following beneficial effects:

[0024] The lightest large model (first model) is used in combination with a pre-defined intent prompt word template to perform preliminary intent rating, which can quickly filter the problem complexity with extremely low computational resource consumption. The main model (second model) and the auxiliary model (third model) are dynamically selected based on the intent rating results to achieve the optimal balance between accuracy and efficiency. The auxiliary model quickly extracts core information and generates the first paragraph of response through the keyword prompt word template, so that the user can obtain preliminary feedback (such as confirming the problem intent) in a very short time. This phased output strategy breaks the limitation that the complete result must be generated in the traditional scheme, significantly improving the response speed perceived by the user, and at the same time, gaining time for the deep generation of the subsequent main model. The main model generates high-quality text streams based on historical context, and through the integration of long-term dialogue memory and domain knowledge, ensures the completeness and logical coherence of the solution to complex problems. Compared with the preliminary results of the auxiliary model, the output of the main model has significant advantages in professionalism and detail depth, and the combination of the two realizes seamless transition from fast response to deep solution. The fast summary of the auxiliary model and the deep generation result of the main model are output in sequence, which not only meets the psychological expectation of the user for immediate feedback, but also guarantees the accuracy of the final answer. Through the streaming voice synthesis technology, the system can generate text streams while gradually playing the voice, further reducing the end-to-end delay, and avoiding the accumulation of waiting caused by the serial process of "generation-synthesis-play" in the traditional scheme. Through the division of labor and cooperation of multiple levels of models, a dynamic balance is achieved among resource consumption (lightweight model processing simple tasks), generation quality (large model processing complex tasks), and response speed (phased output). Compared with the "one-size-fits-all" model usage strategy in the traditional scheme, the present scheme maximizes the utilization of hardware resources under the premise of ensuring service quality, and is particularly suitable for high-concurrency and multi-terminal practical application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A flowchart of an intelligent voice interaction method based on a large language model is provided for the embodiments of the present application.

[0026] Figure 2 A structural schematic diagram of an intelligent voice interaction device based on a large language model is provided for the embodiments of the present application.

[0027] Figure 3 A structural schematic diagram of an electronic device for implementing embodiments of the present application is shown. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. Furthermore, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concept of the present disclosure.

[0029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the terms "a," "an" and "the" are intended to include both singular and plural forms, unless the context clearly indicates otherwise. Furthermore, to the extent that the terms "comprises," "comprising," "includes," "including" and / or "contains," "containing" are used in the detailed description and / or claims, such terms can be used herein to mean including but not limited to, and / or one or more, for example, of the listed components, features, steps and / or elements.

[0030] All terms used herein including technical and scientific terms have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined herein. It should be noted that the terms used herein are defined as having meanings that are consistent with the context of the specification, and should not be interpreted in an idealized or overly formal way.

[0031] Figure 1 A large language model-based intelligent voice interaction method flowchart is provided for the embodiments of the present application. As shown in the figure, the method comprises: Figure 1

[0032] S110, converting the target speech input by the user received into target question text based on a speech recognition module.

[0033] The target speech of the user is converted into target question text by the speech recognition module, thereby achieving the goal of human-computer interaction.

[0034] The speech recognition module can be implemented by software or hardware components, and it can convert human voice signals into text form that can be processed by a computer. It is based on machine learning and deep learning algorithms. For example, it can be implemented through natural language processing (NLP).

[0035] As an example, the NLP receives the user's voice input. Then the voice signal is preprocessed, feature extracted and pattern matched. Finally, the text information corresponding to the voice content is output.

[0036] The target speech here can refer to the voice instructions or questions issued by the user, that is, the conversation content that needs to be recognized and processed. The target question text can refer to the text form of the question generated by the speech recognition module, which can be used as the input of subsequent tasks (such as answering questions, executing commands, etc.).

[0037] In some embodiments, when performing speech recognition, background noise or other interference factors may be involved. Noise reduction processing is needed to improve the recognition accuracy.​

[0038] S120, input the target question text and the pre-defined intent prompt word template into a first large language model to obtain an intent rating of the target question text, the intent rating being used to indicate the degree of accuracy requirement; wherein a plurality of hierarchical large language models are pre-determined, each hierarchical large language model being different in size, and the larger the large language model, the higher the accuracy of the content output by the large language model, and the first large language model being the most lightweight one among the plurality of hierarchical large language models.

[0039] The intent rating can refer to an evaluation result obtained by analyzing the target question text through a language model, which is used to measure the degree of accuracy requirement of the question for the answer.

[0040] A large language model is a natural language processing tool based on artificial intelligence technology, which can understand and generate human language. Its "size" is usually determined by the number of parameters, the more parameters, the more complex the model, and the more accurate the output content.

[0041] The first large language model can refer to the model with the smallest size and the lowest computing resource requirement among the plurality of hierarchical language models.

[0042] The degree of accuracy here can refer to the matching degree between the model output content and the actual requirement, and high accuracy means that the output result is more accurate and meets the expectation.

[0043] In some embodiments, the "first input content" is the target question text, and the intent prompt word template includes the following content:

[0044] Role: intent analysis and model selection expert;

[0045] Background: the user needs to perform intent rating on the "first input content" to determine the degree of accuracy requirement; this involves accurate identification of text intent and reasonable selection of large language models of different levels;

[0046] Personality: proficient in natural language processing and intent analysis, with deep understanding and rich practical experience of the characteristics and application of multi-level large language models, and capable of determining model requirements according to text characteristics;

[0047] Skills: possess professional skills in deep learning, natural language processing, text intent recognition and model optimization, etc., and can efficiently combine target question text with model characteristics to achieve accurate intent rating.

[0048] Objective: to quickly and accurately perform intent rating on the target question text, and help the user to select a suitable large language model according to the intent rating;

[0049] Constraint: The intent rating should be based on the logicality, semantic clarity, and speech rate of the text content, while considering the performance and resource consumption balance of different levels of large language models.

[0050] Output format: Intent rating result.

[0051] In other embodiments, the first input content: user review text, the intent prompt word template includes the following content:

[0052] Role: intent analysis and model selection in sentiment analysis aspect;

[0053] Background: The user needs to perform sentiment analysis on the "first input content" to identify its emotional tendency; this involves accurate capture of text sentiment and suitability assessment of different sentiment models.

[0054] Character portrait: proficient in sentiment computing and natural language processing, with a deep understanding of the advantages and disadvantages of various sentiment models, and can select the optimal analysis method according to the text emotional characteristics.

[0055] Skills: with professional skills such as sentiment recognition, text mining, and model tuning, can efficiently combine target problem text with model characteristics to achieve accurate intent rating.

[0056] Objective: quickly and accurately rate the intent of the target problem text to help users choose the appropriate large language model based on the intent rating.

[0057] Constraint: sentiment analysis should be based on the emotional intensity, keyword distribution, and contextual coherence of the text, while considering the computational efficiency and resource consumption of the model.

[0058] Output format: intent rating result.

[0059] In other embodiments, the first input content: user review text, the intent prompt word template includes the following content:

[0060] Role: intent analysis and model selection in text summarization aspect;

[0061] Background: The user needs to generate a summary of "the first input content" to extract core information; this involves accurate analysis of text structure and suitability judgment of different summary models.

[0062] Character portrait: proficient in text compression and natural language generation, with rich experience in the hierarchical characteristics of summary models, and can adjust the summary strategy according to the complexity of the article.

[0063] Skills: with professional skills such as information extraction, semantic condensation, and model optimization, can efficiently combine target problem text with model characteristics to achieve accurate intent rating.

[0064] Objective: Quickly and accurately grade the target problem text according to its intent, helping users choose the appropriate large language model based on the intent rating.

[0065] Constraints: The summary generation should be based on the consistency of the text's theme, the preservation of key details, and the control of length, while considering the balance between the model's response speed and accuracy.

[0066] Output format: Summary text result.

[0067] In other embodiments, the first input content: user review text, and the intent prompt word template includes the following content:

[0068] Role: Intent analysis and model selection in named entity recognition;

[0069] Background: The user needs to perform entity recognition on the "first input content" to classify key entities; this involves accurate positioning of text entities and performance matching of different entity models.

[0070] Character portrait: proficient in information extraction and entity analysis, with a deep understanding of the application scenarios of various entity recognition models, and the ability to select efficient models according to the characteristics of news.

[0071] Skills: Possess professional skills such as entity detection, classification modeling, and model evaluation, and can efficiently combine target problem text with model characteristics to achieve accurate intent rating.

[0072] Objective: Quickly and accurately grade the target problem text according to its intent, helping users choose the appropriate large language model based on the intent rating.

[0073] Constraints: Entity recognition should be based on the entity density, semantic context, and ambiguity handling of the text, while considering the optimization of model resource consumption and recognition accuracy.

[0074] Output format: Entity list result (such as person name, place name, organization name).

[0075] S130, based on the intent rating and the target problem text, match the second large language model and the third large language model in multiple levels of large language models, the second large language model as the main model, and the third large language model as the auxiliary model.

[0076] In some embodiments, the model is determined based on the following formula: The main score for the current task , and based on Determine the second large language model among multiple large language models:

[0077] ;

[0078] wherein, is a model accuracy in the same task;

[0079] is an intent rating;

[0080] is a numerical stability coefficient;

[0081] is a time sensitivity coefficient; the determined according to a speech rate of the target question text, wherein the target question text corresponds to a speech duration label, the speech rate is determined based on a duration label and a word count of the target question text, and the higher the speech rate, the greater the time sensitivity coefficient;

[0082] is a model generation delay;

[0083] is a model computational cost;

[0084] is a system available computing resource budget;

[0085] determined based on the following formula: in the auxiliary score for the current task , and based on determining the third large language model in a multi-level large language model:

[0086] ;

[0087] is a model response time;

[0088] is a one with the largest response time in the multi-level large language model;

[0089] is a model auxiliary function completeness;

[0090] is an output alignment degree with the main model ; can be determined by cosine similarity;

[0091] is a function and alignment trade-off coefficient.

[0092] In the specific implementation process, the main score function This is used to comprehensively evaluate the performance of candidate master models on the core requirements of the task. In other embodiments, the intent rating can also be determined by both the complexity and criticality of the user input statement; for example, an urgent query command may be assigned a higher intent rating. This time sensitivity coefficient... It can also be dynamically configured according to the real-time requirements of the task: for example, when handling real-time dialogue tasks, The value is taken to be close to 1 to strengthen the delay term. The weights; while in offline analysis scenarios, It can be set to a value close to 0. Calculate the cost item. A balance was achieved between resource utilization and model performance, when Exceeding the budget When this happens, the model will be automatically excluded from the candidate queue.

[0093] For auxiliary scoring The implementation of this method can establish a multi-level model response time baseline library. The functionality can be dynamically updated by periodically scanning the performance of each auxiliary model on the benchmark set. A five-dimensional scale can be used for assessment, covering multilingual support, sentiment analysis, entity recognition, knowledge retrieval, and logical reasoning abilities. Alignment. The calculation can include semantic consistency checks (through cosine similarity) and structural compliance checks (achieved through preset rule templates). It can also be dynamically adjusted. For example, when A value of 0.7 indicates a greater emphasis on auxiliary function coverage; if the main model is detected... If the output confidence level is lower than the threshold, it will automatically switch. To 0.3 to enhance output alignment supervision.

[0094] In some embodiments, the primary and secondary scores of the model for the current task are determined based on specific formulas, thereby identifying the second and third largest language models among multiple large language models. However, in practice, although the current approach proposes to determine the intent rating based on the complexity and criticality of the user input statement, refining this rating criterion and ensuring its accuracy remains a challenge. For example, the complexity and criticality of query instructions may differ across different domains, thus requiring more refined evaluation criteria.

[0095] Domain-specific intent rating criteria can be introduced, and different evaluation indicators and weights can be developed for different domains.

[0096] The current solution has dynamically configured the time-sensitive coefficient according to the task real-time requirement, but in some cases, such as handling urgent but non-real-time dialogue tasks, how to balance the latency and model performance is still a difficult problem.

[0097] An emergency task recognition mechanism can be introduced to automatically adjust the time-sensitive coefficient for urgent tasks to prioritize fast response.

[0098] A task priority queue can also be established to dynamically adjust model configuration and resource allocation based on the urgency and importance of the task.

[0099] In the calculation cost item, the current solution mainly considers whether the model's calculation cost exceeds the system's available calculation resource budget. However, in practical applications, how to more finely manage the calculation cost and resource budget to improve resource utilization and model performance is still a problem to be solved.

[0100] A cost-benefit analysis model can also be introduced to comprehensively evaluate the model's calculation cost and the benefits it brings to optimize resource allocation.

[0101] A dynamic resource scheduling mechanism can also be established to dynamically adjust model configuration and resource budget based on system load and task demand.

[0102] In the auxiliary score, the current solution has considered the model's response time and auxiliary function completeness. However, in some cases, how to balance the two to ensure the model's practicality and efficiency is still a challenge.

[0103] A response time optimization algorithm can also be introduced to finely adjust the model's response time to improve its response speed.

[0104] A function completeness evaluation system can also be established to periodically evaluate and optimize the model's auxiliary functions to ensure they meet the needs of practical applications.

[0105] S140, input the target problem text and keyword prompt word template into the third large language model to obtain the summary and keywords.

[0106] The "first input content" is the target problem text, and the keyword prompt word template includes generating a summary and keywords based on the "first input content".

[0107] During summary generation, the third large language model first performs semantic understanding on the target problem text, then extracts key information and forms a concise and clear summary in combination with the keyword prompt word template. At the same time, the model will intelligently generate keywords related to the target problem text content and context to help users better understand the core points of the problem text.

[0108] S150, generating a first output result based on the summary and the keywords for voice output.

[0109] In some embodiments, the summary and the keywords can be input into a first large language model for fusion to obtain the first output result.

[0110] The first large language model can generate coherent and natural voice content through deep understanding and fusion of the summary and the keywords. The model will use advanced natural language processing techniques to ensure that the output voice content not only accurately conveys the core information of the summary and the keywords, but also conforms to human voice expression habits, so that users can easily understand and obtain the required information. In addition, the first large language model also has emotional modeling capabilities, which can adjust the tone, rhythm, etc. of the voice according to different contexts and user needs, further improving the user's voice interaction experience.

[0111] S160, inputting the target question text and the historical context related to the target question text into a second large language model to obtain a high-quality text stream.

[0112] The second large language model has strong context understanding and generation capabilities, which can deeply analyze the target question text and its historical context, capture key information and logical relationships therein. Through this step, the model can generate a high-quality text stream that not only contains the core points of the target question text, but also incorporates relevant information from the historical context. This text stream not only helps users better understand the problem background, but also provides a more rich and accurate information base for subsequent intelligent voice interaction.

[0113] S170, fusing the high-quality text stream, the summary and the keywords to obtain a second output result, and continuing to voice output the second output result after the first output result is played.

[0114] Among them, the first output result and the second output result can be text content, which can be converted into corresponding semantics based on the first output result and the second output result respectively through a text-to-speech module, and then played through a loudspeaker.

[0115] In order to enhance the user's interaction experience, the first output result and the second output result can also adjust the tone, volume and rhythm of the voice according to the emotional color of the content when playing. For example, when playing content containing positive emotions, the voice can be more cheerful and bright; while playing content containing negative emotions, the voice can be more low and slow. This emotional voice output method not only helps users better understand the content, but also improves user's emotional resonance and interaction satisfaction.

[0116] In some embodiments, the high-quality text stream, the summary, and the keywords can be input into the first large language model according to a fusion prompt word template to obtain a second output result.

[0117] The fusion prompt word template can be flexibly designed according to actual needs to ensure that the second output result contains both detailed information of the high-quality text stream and core points of the summary and the keywords. For example, the template can be designed as: "Next, let me introduce the relevant content in detail, which includes the core information of …, that is, the summary and the keywords have …". Through such a design, the text stream generated by the second large language model in combination with the fusion capability of the first large language model can provide users with a comprehensive and refined voice interaction experience. After playing the first output result, the system seamlessly transitions to the second output result, ensuring the coherence and integrity of the information.

[0118] In some embodiments, the fusion prompt word template can include:

[0119] Based on the "second input content", "third input content" and "fourth input content", the syntax is arranged to generate a second output result, wherein in the syntax arrangement process, the connection words can be increased or decreased, but the "second input content", "third input content" and "fourth input content" cannot be modified;

[0120] Among them, the "second input content", "third input content" and "fourth input content" are respectively the high-quality text stream, the summary and the keywords.

[0121] The lightest large model (first model) is combined with a pre-defined intent prompt word template to preliminarily rate the intent, which can quickly filter the problem complexity with extremely low computational resource consumption. The main model (second model) and the auxiliary model (third model) are dynamically selected based on the intent rating result, realizing the optimal balance between accuracy and efficiency. The auxiliary model quickly extracts core information through a keyword prompt word template and generates a first paragraph of response, so that the user can obtain preliminary feedback (such as confirming the problem intent) in a very short time. This phased output strategy breaks the limitation that the complete result must be generated in the traditional scheme, significantly improving the response speed perceived by the user, and at the same time, gaining time for the deep generation of the subsequent main model. The main model generates a high-quality text stream based on historical context, and integrates long-term dialogue memory and domain knowledge to ensure the completeness and logical consistency of the solution to complex problems. Compared with the preliminary result of the auxiliary model, the output of the main model has significant advantages in professionalism and detail depth, and the combination of the two realizes seamless transition from fast response to deep solution. The fast summary of the auxiliary model and the deep generation result of the main model are output in sequence, which not only meets the psychological expectation of the user for immediate feedback, but also guarantees the accuracy of the final answer. Through the streaming voice synthesis technology, the system can generate text stream while gradually playing voice, further reducing the end-to-end delay, and avoiding the accumulation of waiting caused by the serial process of "generation-synthesis-play" in the traditional scheme. Through the division of labor of multi-level models, a dynamic balance is realized among resource consumption (lightweight model processing simple tasks), generation quality (large model processing complex tasks), and response speed (phased output). Compared with the "one-size-fits-all" model usage strategy in the traditional scheme, this scheme maximizes the utilization of hardware resources under the premise of ensuring service quality, and is especially suitable for high-concurrency and multi-terminal practical application scenarios.

[0122] Figure 2 A structure schematic diagram of an intelligent voice interaction device based on a large language model is provided for the embodiments of the present application. As shown in Figure 2 the device can include:

[0123] The conversion module 201 is configured to convert the target voice input by the user into target question text based on the voice recognition module;

[0124] The input module 202 is configured to input the target question text and the pre-defined intent prompt word template into the first large language model to obtain the intent rating of the target question text, and the intent rating is used to indicate the degree of accuracy requirement; wherein a plurality of levels of large language models are pre-determined, and each level of large language model has a different size, and the larger the large language model, the higher the accuracy of the content output by the large language model, and the first large language model is the lightest one among the plurality of levels of large language models;

[0125] The matching module 203 is configured to match the second large language model and the third large language model based on the intention rating and the target question text in a plurality of hierarchical large language models, the second large language model as a main model, and the third large language model as an auxiliary model.

[0126] The input module 202 is further configured to input the target question text and the keyword prompt word template into the third large language model to obtain the summary and the keyword.

[0127] The output module 204 is configured to generate the first output result based on the summary and the keyword and perform voice output.

[0128] The input module 202 is further configured to input the target question text and the historical context related to the target question text into the second large language model to obtain the high-quality text stream.

[0129] The output module 204 is further configured to fuse the high-quality text stream, the summary and the keyword to obtain the second output result, and continue to perform voice output of the second output result after the first output result is played.

[0130] In some embodiments, the output module 204 is specifically configured to:

[0131] fuse the high-quality text stream, the summary and the keyword according to the fusion prompt word template to input into the first large language model to obtain the second output result.

[0132] In some embodiments, the output module 204 is specifically configured to:

[0133] fuse the summary and the keyword to input into the first large language model to obtain the first output result.

[0134] Figure 3 A structural schematic diagram of an electronic device for implementing the embodiments of the present application is shown. As shown in the figure, Figure 3 The electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or programs loaded from a storage portion 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0135] In some embodiments, the electronic device 300 can be a smart speaker, a voice intelligent assistant, or the like, based on which the following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, a microphone, or the like; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), or the like, and a speaker, or the like; a storage section 308 including a hard disk, or the like; and a communication section 309 including a network interface card such as a LAN card, a modem, or the like. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as necessary. A removable media 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 310 as necessary, so that a computer program read therefrom is installed in the storage section 308 as necessary.

[0136] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising instructions carried on a computer-readable medium, in such embodiments the instructions can be downloaded from a network via the communication section 309 and installed, and / or installed from the removable media 311. When the instructions are executed by the central processing unit (CPU) 301, the individual method steps described in the present disclosure are performed.

[0137] While example embodiments have been described, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the present inventive concept. Therefore, it is to be understood that the foregoing example embodiments are illustrative of the present inventive concept and are not to be construed as limiting.

Claims

1. A method for intelligent voice interaction based on a large language model, characterized in that, Comprise: Converting the target voice of the received user input into target question text based on a voice recognition module; Inputting the target question text and a pre-defined intent prompt word template into a first large language model to obtain an intent rating of the target question text, the intent rating indicating the level of precision required; wherein a plurality of hierarchical large language models are pre-determined, each hierarchical large language model being different in size, and the larger the large language model, the higher the precision of the content output; the first large language model being the most lightweight one among the plurality of hierarchical large language models; Matching the intent rating and the target question text to a second large language model and a third large language model among the plurality of hierarchical large language models, the second large language model being the primary model and the third large language model being the auxiliary model; Inputting the target question text and a keyword prompt word template into the third large language model to obtain an abstract and keywords; Generating a first output result based on the abstract and keywords for voice output; Inputting the target question text and the historical context related to the target question text into the second large language model to obtain a high-quality text stream; Fusing the high-quality text stream, the abstract, and the keywords to obtain a second output result, and continuing to voice output the second output result after the first output result is played; Wherein, "the first input content" is the target question text, and the intent prompt word template includes the following content: Role: intent analysis and model selection expert; Background: intent rating of "the first input content" to determine its demand for precision; Character portrait: proficient in natural language processing and intent analysis, with deep understanding and rich practical experience of the characteristics and application of multi-level large language models, capable of determining model requirements according to text characteristics; Skills: with professional skills in deep learning, natural language processing, text intent recognition, and model optimization, capable of efficiently combining target question text with model characteristics to achieve accurate intent rating; Objective: to quickly and accurately rate the intent of the target question text and help users choose the appropriate large language model according to the intent rating; Constraints: intent rating should be based on the logicality, semantic clarity, and speech rate of the text content, while considering the performance and resource consumption balance of different hierarchical large language models; Output format: intent rating result; The keyword prompt word template includes generating an abstract and keywords based on "the first input content"; The model is determined based on the following equation In the main score for the current task And based on Among the plurality of large language models, the second large language model is determined: ； wherein as a model accuracy in a same task; Intended rating; is the numerical stability coefficient; is the time sensitive coefficient; To model Generate latency; for the model computational cost; a budget of available computing resources for the system; determining a model based on the following equation in the auxiliary score for the current task and based on determining the third large language model in a multi-level large language model ； wherein for the model response time; is the largest response time in the multi-level large language model; For models of assistive functionality completeness; to align with the output of the master model ; is a function of the trade-off coefficient.

2. The method of claim 1, wherein, determining a speech rate according to the target question text, wherein the target question text corresponds to a speech duration label, and the speech rate is determined based on a duration label and a word count of the target question text.

3. The method of claim 1, wherein, The By cosine similarity determination.

4. The method of claim 1, wherein, Fusing the high-quality text stream, the abstract, and the keywords to obtain a second output result, including: Inputting the high-quality text stream, the abstract, and the keywords into the first large language model according to the fusion prompt word template to obtain a second output result; The fusion prompt word template includes: The second input content, the third input content, and the fourth input content are used to generate a second output result by sequence arrangement, wherein in the sequence arrangement process, the connectivity of the words can be increased or decreased, but the second input content, the third input content, and the fourth input content cannot be modified. The second input content, the third input content, and the fourth input content are respectively the high-quality text stream, the summary, and the keyword.

5. The method of claim 1, wherein, The first output result is generated based on the summary and the keyword, including: The summary and the keyword are input into the first large language model for fusion to obtain the first output result.

6. The intelligent voice interaction device based on a large language model, characterized in that, Including: The conversion module is configured to convert the received target voice input by the user into target question text based on the voice recognition module; The input module is configured to input the target question text and a pre-defined intent prompt word template into the first large language model to obtain an intent rating of the target question text, wherein the intent rating indicates the level of accuracy requirement; wherein a plurality of levels of large language models are pre-determined, each level of large language model has a different size, and the larger the large language model, the higher the accuracy of the content output by the large language model, and the first large language model is the most lightweight one among the plurality of levels of large language models; The matching module is configured to match the second large language model and the third large language model in the plurality of levels of large language models based on the intent rating and the target question text, wherein the second large language model serves as a main model and the third large language model serves as an auxiliary model; The input module is further configured to input the target question text and a keyword prompt word template into the third large language model to obtain a summary and a keyword; The output module is configured to generate a first output result based on the summary and the keyword and output the first output result by voice; The input module is further configured to input the target question text and a historical context related to the target question text into the second large language model to obtain a high-quality text stream; The output module is further configured to fuse the high-quality text stream, the summary, and the keyword to obtain a second output result, and continue to output the second output result by voice after the first output result is played; The first input content is the target question text, and the intent prompt word template includes the following content: Role: intent analysis and model selection expert; Background: intent rating of the first input content to determine the level of accuracy requirement; Character portrait: proficient in natural language processing and intent analysis, with deep understanding and rich practical experience of the characteristics and application of multi-level large language models, and capable of determining model requirements according to text characteristics; Skills: professional skills in deep learning, natural language processing, text intent recognition, and model optimization, capable of efficiently combining target question text and model characteristics to achieve accurate intent rating; Objective: to quickly and accurately rate the intent of the target question text and help users select the appropriate large language model according to the intent rating; Constraint: The intent rating should be determined based on the logicality, semantic clarity, and speech rate of the text content, while considering the performance and resource consumption balance of different levels of large language models; Output format: intent rating result; The keyword prompt word template includes: generating an abstract and keywords based on the "first to-be-input content"; determining a model based on the following equation in a primary score for a current task and based on determining the second large language model among a plurality of large language models ； wherein for the model accuracy in a same task; Intended rating; is the numerical stability coefficient; is the time sensitive coefficient; for the model generating a latency; for the model computational cost; a budget of available computing resources for the system; determining a model based on the following equation in the auxiliary score for the current task and based on determining the third large language model in a multi-level large language model ； wherein for the model response time; is the largest response time in the multi-level large language model; For models of assistive functionality completeness; to align with the output of the master model ; is a function of the trade-off coefficient.

Images