An intent recognition method and device based on a general large language model and readable medium

Abstract

This invention discloses an intent recognition method, apparatus, and readable medium based on a general large language model, comprising: combining a basic intent recognition instruction, a candidate intent list, and historical dialogue records to obtain a first intent recognition instruction, and adjusting the general large language model accordingly to obtain a first intent recognition model; if its accuracy is less than a threshold, combining the first intent recognition instruction with input / output examples to obtain a second intent recognition instruction, and adjusting the general large language model accordingly to obtain a second intent recognition model; if its accuracy is less than the threshold, constructing an instruction-supervised fine-tuning dataset based on the first or second intent recognition instruction and training data, and fine-tuning the second intent recognition model accordingly to obtain a fine-tuned intent recognition model; and using the first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an accuracy greater than or equal to the threshold as the large intent recognition model and classifying the intent.

Description

Technical Field

[0001] This invention relates to the field of intent recognition, and more specifically to an intent recognition method, apparatus, and readable medium based on a general large language model. Background Technology

[0002] Currently, large-scale models and their applications are experiencing a golden age of rapid development, profoundly changing the face of various industries. These large-scale machine learning models, with their massive parameter scale, complex network structure, and massive training data, have demonstrated unprecedented learning and generalization capabilities.

[0003] In fields such as natural language processing, computer vision, speech recognition, and recommendation systems, large-scale models have become a key force driving technological innovation and application upgrades. In natural language processing, large language models such as the GPT series and Qianwen have not only achieved significant results in text generation, question-answering systems, and sentiment analysis, but are also gradually penetrating professional fields such as education, healthcare, and law, providing intelligent assistance and decision support. They can understand and generate coherent and logical text, greatly improving the fluency and naturalness of human-computer interaction. However, in the deep integration of large-scale models with business systems, any misunderstanding or error can cause significant problems in the overall workflow of the business system, turning what was once an aid to the large-scale model and business system into a burden. Therefore, how to quickly and accurately interpret the intent of language text has become a key technology for combining large-scale models with applications.

[0004] As the informatization level of various business departments continues to improve, the business systems they build are becoming increasingly diversified. With the increasing integration of these systems, the demands on their intelligence also present greater challenges. A significant characteristic is that the number of intent branches that the system needs to identify is increasing, and the intent recognition results are becoming more and more similar. Traditional intent recognition based on large models only utilizes the capabilities of the large model itself and fails to effectively integrate specific business scenarios. For example, when considering opening a specific video versus opening a specific video pre-setup, traditional intent recognition will miss the mark between the video and menu intent categories, resulting in an accuracy rate of generally around 70%. This accuracy fails to meet application requirements, leading to a poor user experience. Summary of the Invention

[0005] The purpose of this application is to propose an intent recognition method, apparatus, and readable medium based on a general large language model to address the aforementioned technical problems.

[0006] In a first aspect, the present invention provides an intent recognition method based on a general large language model, comprising the following steps:

[0007] Obtain the input text, its corresponding historical dialogue records, and the output text to construct training data;

[0008] Construct a general large language model, build basic instructions for intent recognition based on business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list.

[0009] The first intent recognition instruction is obtained by combining the basic intent recognition command, the candidate intent list, and the historical dialogue record corresponding to the input text. The first intent recognition instruction is then used to adjust the general large language model to obtain the first intent recognition model. In response to the determination that the intent recognition accuracy of the first intent recognition model is less than the accuracy threshold, input-output samples are constructed. The first intent recognition instruction is combined with the input-output samples to obtain the second intent recognition instruction. The second intent recognition instruction is then used to adjust the general large language model to obtain the second intent recognition model. In response to the determination that the intent recognition accuracy of the second intent recognition model is less than the accuracy threshold, an instruction-supervised fine-tuning dataset is constructed based on the first or second intent recognition instruction and the training data. The instruction-supervised fine-tuning dataset is then used to fine-tune the second intent recognition model to obtain the fine-tuned intent recognition model.

[0010] The system obtains the current input text or the current input text and its corresponding historical dialogue records, and constructs a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold are used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

[0011] Preferably, intent recognition test cases are constructed based on the candidate intent list corresponding to the input text through enumeration and in combination with written or spoken language. The intent recognition test cases include a total of N pairs of test input texts and test output texts. The test input texts in the intent recognition test cases are input into the first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model to obtain the predicted output text. The intent recognition accuracy is the number of times the predicted output text is the same as the test output text divided by N.

[0012] Preferably, the first intent recognition instruction is combined with the input / output sample to obtain the second intent recognition instruction, which specifically includes:

[0013] In response to determining that the number of input / output samples is greater than or equal to a threshold, the similarity between the embedding vector of the input / output sample and the embedding vector of the input text is calculated. The similarity includes cosine similarity or Euclidean distance. Input / output samples whose similarity between the embedding vector of the input / output sample and the embedding vector of the input text is greater than the similarity threshold are selected as filtered input / output samples. The filtered input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0014] In response to determining that the number of input / output samples is less than a quantity threshold, the direct input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0015] Preferably, an instruction-supervised fine-tuning dataset is constructed based on the first intent recognition instruction or the second intent recognition instruction and the training data, specifically including:

[0016] The candidate intent list in the first intent recognition instruction or the second intent recognition instruction is adjusted to include the output text and an enhanced candidate intent list containing several intent categories randomly selected from the candidate intent list, thus obtaining the data-enhanced first intent recognition instruction or the second intent recognition instruction.

[0017] The first or second intent recognition instruction augmented with data is combined with the corresponding training data to form an instruction-supervised fine-tuning dataset.

[0018] As a preferred option, it also includes:

[0019] In response to determining that the intent recognition accuracy of the first intent recognition model is greater than or equal to the accuracy threshold, the first intent recognition model is used as the large intent recognition model.

[0020] In response to determining that the intent recognition accuracy of the second intent recognition model is greater than or equal to the accuracy threshold, the second intent recognition model is used as the large intent recognition model.

[0021] In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold, the fine-tuned intent recognition model is taken as the large intent recognition model.

[0022] In response to the determination that the intent recognition accuracy of the fine-tuned intent recognition model is less than the accuracy threshold, the input and output samples and / or instructions are continuously adjusted to supervise the fine-tuning dataset until the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold.

[0023] As a preferred option, the general large language model includes the Transformer model, and the candidate intent list also includes intent descriptions, with an option for intent category "other" added to the candidate intent list.

[0024] Secondly, the present invention provides an intent recognition device based on a general large language model, comprising:

[0025] The training data construction module is configured to acquire input text and its corresponding historical dialogue records and output text to construct training data.

[0026] The model building module is configured to build a general large language model, build basic instructions for intent recognition according to business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list.

[0027] The model adjustment module is configured to combine the basic intent recognition command, the candidate intent list, and the historical dialogue record corresponding to the input text to obtain a first intent recognition command. The first intent recognition command is then used to adjust the general large language model to obtain a first intent recognition model. In response to determining that the intent recognition accuracy of the first intent recognition model is less than an accuracy threshold, input-output samples are constructed, and the first intent recognition command is combined with the input-output samples to obtain a second intent recognition command. The second intent recognition command is then used to adjust the general large language model to obtain a second intent recognition model. In response to determining that the intent recognition accuracy of the second intent recognition model is less than an accuracy threshold, an instruction-supervised fine-tuning dataset is constructed based on the first or second intent recognition command and training data. The instruction-supervised fine-tuning dataset is then used to fine-tune the second intent recognition model to obtain a fine-tuned intent recognition model.

[0028] The recognition module is configured to acquire the current input text or the current input text and its corresponding historical dialogue records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold is used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

[0029] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0030] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0031] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) The intent recognition method based on the general large language model proposed in this invention not only lowers the threshold of intent recognition but also greatly improves the accuracy of intent recognition in business systems by combining the contextual capabilities of the general large language model with input and output examples, threshold vector matching degree and instruction fine-tuning datasets and other multi-dimensional adjustment methods.

[0034] (2) The intent recognition method based on the general large language model proposed in this invention retains the capabilities of the general large language model, while allowing users to select the corresponding candidate intent list according to their business category to complete the intent recognition of their respective business, making the business system more intelligent. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the intent recognition method based on a general large language model, as an embodiment of this application.

[0037] Figure 2 A flowchart illustrating the intent recognition method based on a general large language model, as an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of an intent recognition device based on a general large language model, which is an embodiment of this application.

[0039] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] Figure 1 illustrates an intention recognition method based on a general large language model provided by the embodiments of the present application, comprising the following steps:

[0042] S1: Acquire the input text, its corresponding historical dialogue records and output text to construct training data.

[0043] Specifically, refer to Figure 2 , collect a number of input texts, their corresponding historical dialogue records and output texts. The output texts here are the intention categories of the input texts or the input texts and their corresponding historical dialogue records, and finally the training data is constructed. Acquire the historical dialogue record γ, for example, γ = "<History Dialogue>< / History Dialogue>". Normally, it is sufficient to acquire the first 5 rounds of historical dialogues, and in other embodiments, the number can be adjusted according to specific scenarios.

[0044] S2: Construct a general large language model, construct a basic intention recognition instruction according to business requirements, and construct a candidate intention list according to the business categories mentioned in the input text and its historical dialogue records. The basic intention recognition instruction requires that the intention recognition be carried out by comprehensively considering the input text and its corresponding historical dialogue records, and requires that the output text of the general large language model must be an intention category in the candidate intention list.

[0045] In a specific embodiment, the general large language model includes a Transformer model, the candidate intention list further includes intention descriptions, and an option with the intention category of "Others" is also added to the candidate intention list.

[0046] Specifically, the general large language model in the embodiments of the present application may adopt a Transformer model. Specifically, an appropriate large language model is selected according to existing resources, taking into account multiple requirements such as computing power resources, response requirements, and recognition accuracy, and a model with relatively small parameters is recommended, such as a 7B model.

[0047] Construct the basic intention recognition instruction α according to business requirements. In the construction process of the basic intention recognition instruction, the output text must be forcibly constrained to avoid the phenomenon of random responses from the large language model, which leads to a decrease in the accuracy of intention recognition. For example: "Please note that your output must be in this candidate intention list, only output the name of the intention category and do not output the intention description".

[0048] The following gives a specific example of a basic intent recognition instruction: α="<Instruction>You are an intent recognition expert. Your job is to perform intent recognition on the content input by the user, which requires comprehensive consideration of historical conversation records and the user's current input text. If the user's question can find an answer from the historical conversation records, please return others. The known candidate intent list is as follows. Please note that your output must be within this candidate intent list, only output the name of the intent category and do not output the intent description.< / Instruction>".

[0049] Construct a candidate intent list β according to a specific scenario, where each intent represents a service branch, and the intent category is related to the service category; add an "others" option to the candidate intent list to reserve the question-and-answer function of the large language model itself, so as to avoid the disadvantage that the model evolves to only be capable of intent recognition, for example: β="<Classification List>1. Make a call\n 2. Open video\n3. Open menu\n…99. Others< / Classification List>".

[0050] S3, combining the basic intent recognition instruction, the candidate intent list and the historical conversation records corresponding to the input text to obtain a first intent recognition instruction, and using the first intent recognition instruction to perform instruction adjustment on the general large language model to obtain a first intent recognition model; in response to determining that the intent recognition accuracy of the first intent recognition model is less than the accuracy threshold, constructing input and output samples, combining the first intent recognition instruction with the input and output samples to obtain a second intent recognition instruction, and using the second intent recognition instruction to perform instruction adjustment on the general large language model to obtain a second intent recognition model; in response to determining that the intent recognition accuracy of the second intent recognition model is less than the accuracy threshold, constructing an instruction supervised fine-tuning data set according to the first intent recognition instruction or the second intent recognition instruction and the training data, and using the instruction supervised fine-tuning data set to fine-tune the second intent recognition model to obtain a fine-tuned intent recognition model.

[0051] Specifically, the above basic intent recognition instruction α, candidate intent list β and historical conversation record γ are spliced to obtain the first intent recognition instruction x1=α+β+γ. The first intent recognition instruction x1 is directly connected to the general large language model to obtain the first intent recognition model. The complexity of intent recognition scenarios varies: the more complex, the more similar, and the more difficult to distinguish. For simple scenarios, the first intent recognition model applying the first intent recognition instruction x1 can already meet the requirements.

[0052] In a specific embodiment, intent recognition test cases are constructed based on the candidate intent list corresponding to the input text through enumeration and in combination with written or spoken language. The intent recognition test cases include a total of N pairs of test input texts and test output texts. The test input texts in the intent recognition test cases are input into the first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model to obtain the predicted output text. The intent recognition accuracy is the number of times the predicted output text is the same as the test output text divided by N.

[0053] Specifically, intent recognition test cases are constructed. Based on the candidate intent list, intent recognition test cases are constructed using enumeration methods and combined with different methods such as written language and spoken language. Special attention is paid to the construction of spoken language test cases. In daily questions, the proportion of spoken language is much greater than that of written language. For example, if the intent category is: obtain tidal data, the written language is: check today's tidal information; the spoken language is: when is the tide going out today.

[0054] Furthermore, the intent recognition accuracy of the first intent recognition model is tested using intent recognition test cases. The intent recognition accuracy is calculated as the number of times the predicted output text generated by the first intent recognition model is the same as the test output text in the intent recognition test cases, divided by the total number of intent recognition test cases N.

[0055] In a specific embodiment, the first intent recognition instruction is combined with the input / output sample to obtain the second intent recognition instruction, which specifically includes:

[0056] In response to determining that the number of input / output samples is greater than or equal to a threshold, the similarity between the embedding vector of the input / output sample and the embedding vector of the input text is calculated. The similarity includes cosine similarity or Euclidean distance. Input / output samples whose similarity between the embedding vector of the input / output sample and the embedding vector of the input text is greater than the similarity threshold are selected as filtered input / output samples. The filtered input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0057] In response to determining that the number of input / output samples is less than a quantity threshold, the direct input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0058] Specifically, if the intent recognition accuracy of the first intent recognition model fails to meet the accuracy requirement, that is, is less than the accuracy threshold, the input-output example δ is added on the basis of the first intent recognition instruction; the input-output example is generally divided into two parts, "input" and "output", for example, δ="<example><input>his legal person's phone number< / input><output>make a phone call< / output><input>I want to check the accident case< / input><output>call up menu< / output>< / example>". If there are too many input-output examples, the input-output examples whose embedding vectors are relatively close to the embedding vector of the input text are screened out by calculating cosine similarity, Euclidean distance and other methods, so as to obtain the screened input-output examples, and the first intent recognition instruction is spliced with the screened input-output examples or the input-output example δ to obtain the second intent recognition instruction x2=α+β+γ+δ.

[0059] Further, the second intent recognition instruction x2 is directly used to interface with a general large language model to obtain a second intent recognition model, and intent recognition test examples are used to test the intent recognition accuracy of the second intent recognition model. At this time, the intent recognition accuracy is the number of predicted output texts generated by inputting the test input texts in the intent recognition test examples into the second intent recognition model that are the same as the test output texts, divided by the total number N of the intent recognition test examples. If the intent recognition accuracy of the second intent recognition model fails to meet the accuracy requirement, that is, is less than the accuracy threshold, model fine-tuning is performed through the instruction supervised fine-tuning dataset.

[0060] In a specific embodiment, the instruction supervised fine-tuning dataset is constructed according to the first intent recognition instruction or the second intent recognition instruction and training data, which specifically includes:

[0061] The candidate intent list in the first intent recognition instruction or the second intent recognition instruction is adjusted to an enhanced candidate intent list that includes the output text and several intent categories randomly selected from the candidate intent list, so as to obtain a data-augmented first intent recognition instruction or second intent recognition instruction;

[0062] The data-augmented first intent recognition instruction or second intent recognition instruction is respectively combined with corresponding training data to form the instruction supervised fine-tuning dataset.

[0063] Specifically, by constructing an instruction fine-tuning dataset based on intent sampling, the system prevents the model from collapsing after fine-tuning. For example, if the original candidate intent list contains 10 intents, only 6-7 different intents can be selected when constructing the instruction fine-tuning dataset. An example from an embodiment of this application is as follows: "{"instruction":"You are a professional intent recognition expert. Based on the following candidate intent list, and by fully combining historical dialogue records and the current user's input text, accurately select the name of the intent category that best matches the user's intent from the candidate intent list. \nCandidate Intent List:\n1. Access Video: Open, call, view, or query video, video plan, monitoring, or recording.\n2. Enterprise Profile: Query relevant enterprise profile information.\n3. Send Notification: Send a notification or message to the user or system.\n4. Upload File: Upload a file to the system.\n5. Access Menu: Call, open, or query a specific function menu.\n6. Other: If none of the above intents apply, then it is "Other".\nRestrictions:\n-The output must be the name of the intent category in the given candidate intent list; you may not create or modify the category yourself." \n- Output only the name of the intent category, without adding any explanation, description or other text. ","input":"input:View menu for highlight videos","output":"Induce menu"}.

[0064] The embodiment of this application provides another example as follows: "{"instruction":"You are a professional intent recognition expert. Based on the following candidate intent list, fully combine historical dialogue records and the current user's input text to make a comprehensive judgment, accurately select the name of the intent category that best matches the user's intent from the candidate intent list.\nCandidate intent list:\n1. Monitoring environment: View monitoring information of physical or virtual environment.\n2. Adjust video: Open, call, view, or query video, video plan, monitoring, or recording.\n3. Adjust menu: Call, open, or query a function menu.\n4. Other: If none of the following intents are met, it is "Other".\nRestrictions:\n-The output must be the name of the intent category in the given candidate intent list. You may not create or modify the category yourself.\n-Only output the name of the intent category, without adding any explanation, description, or other text.","input":"input: Call the video plan of Xiamen Bridge","output":"Adjust video"}.

[0065] The second intent recognition model was fine-tuned using the above instruction fine-tuning dataset to obtain the fine-tuned intent recognition model.

[0066] In specific embodiments, it also includes:

[0067] In response to determining that the intent recognition accuracy of the first intent recognition model is greater than or equal to the accuracy threshold, the first intent recognition model is used as the large intent recognition model.

[0068] In response to determining that the intent recognition accuracy of the second intent recognition model is greater than or equal to the accuracy threshold, the second intent recognition model is used as the large intent recognition model.

[0069] In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold, the fine-tuned intent recognition model is taken as the large intent recognition model.

[0070] In response to the determination that the intent recognition accuracy of the fine-tuned intent recognition model is less than the accuracy threshold, the input and output samples and / or instructions are continuously adjusted to supervise the fine-tuning dataset until the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold.

[0071] Specifically, if the intent recognition accuracy of the first intent recognition model can meet the accuracy requirement, that is, greater than or equal to the intent recognition accuracy, then the first intent recognition model will be directly used as the large intent recognition model.

[0072] If the intent recognition accuracy of the first intent recognition model cannot meet the accuracy requirement, while the intent recognition accuracy of the second intent recognition model can meet the accuracy requirement, then the second intent recognition model will be used as the larger intent recognition model.

[0073] If the intent recognition accuracy of both the first and second intent recognition models fails to meet the accuracy requirements, but the intent recognition accuracy of the fine-tuned intent recognition model meets the accuracy requirements, then the fine-tuned intent recognition model will be used as the large intent recognition model.

[0074] If the intent recognition accuracy of the fine-tuned intent recognition model fails to meet the accuracy requirements, the above steps are repeated to continuously adjust the input / output samples and the instruction fine-tuning dataset until the intent recognition accuracy of the fine-tuned intent recognition model meets the accuracy requirements. In one embodiment, after testing in a business system, the intent recognition accuracy reached over 95%, which basically meets the application needs of most business systems and can be well integrated into the business system.

[0075] S4. Obtain the current input text or the current input text and its corresponding historical dialogue records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. Use the first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold as the large intent recognition model. Input the current input text or the current input text and its corresponding historical dialogue records and the candidate intent list into the large intent recognition model to generate the corresponding intent category.

[0076] Specifically, as the number of business systems increases and the degree of information integration becomes higher, for example, a system may need to recognize dozens of different intent categories such as video, menus, duty shifts, and command and dispatch (including the dispatch of various command elements). Therefore, deploying the finalized intent recognition model in the business system can more accurately obtain more refined intent categories. By obtaining the user's current input text or the current input text and its corresponding historical dialogue records, as well as the constructed candidate intent list, and inputting them into the intent recognition model, the intent category corresponding to the current input text or the current input text and its corresponding historical dialogue records can be generated.

[0077] Further reference Figure 3 As an implementation of the methods shown in the above figures, this application provides an embodiment of an intent recognition device based on a general large language model. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0078] This application provides an intent recognition device based on a general large language model, comprising:

[0079] Training data construction module 1 is configured to acquire input text and its corresponding historical dialogue records and output text to construct training data;

[0080] Model building module 2 is configured to build a general large language model, build basic instructions for intent recognition according to business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list.

[0081] Model adjustment module 3 is configured to combine the basic intent recognition instruction, the candidate intent list, and the historical dialogue record corresponding to the input text to obtain a first intent recognition instruction; use the first intent recognition instruction to adjust the general large language model to obtain a first intent recognition model; in response to determining that the intent recognition accuracy of the first intent recognition model is less than the accuracy threshold, construct input-output samples, combine the first intent recognition instruction with the input-output samples to obtain a second intent recognition instruction, use the second intent recognition instruction to adjust the general large language model to obtain a second intent recognition model; in response to determining that the intent recognition accuracy of the second intent recognition model is less than the accuracy threshold, construct an instruction-supervised fine-tuning dataset based on the first intent recognition instruction or the second intent recognition instruction and the training data, and use the instruction-supervised fine-tuning dataset to fine-tune the second intent recognition model to obtain a fine-tuned intent recognition model.

[0082] The recognition module 4 is configured to acquire the current input text or the current input text and its corresponding historical dialogue records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold is used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

[0083] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 4 As shown, the electronic device in this embodiment includes a processor 401 and a memory 402; wherein the memory 402 is used to store computer execution instructions; and the processor 401 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0084] Alternatively, the memory 402 can be either standalone or integrated with the processor 401.

[0085] When the memory 402 is set up independently, the electronic device also includes a bus 403 for connecting the memory 402 and the processor 401.

[0086] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 401, implement the above method.

[0087] This invention also provides a computer program product, including a computer program that, when executed by a processor 401, implements the above-described method.

[0088] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0089] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0090] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0091] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 401 to execute some steps of the methods of the various embodiments of this application.

[0092] It should be understood that the processor 401 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 401 can be any conventional processor 401. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 401 executing the steps, or as a combination of hardware and software modules within the processor 401 executing the steps.

[0093] The memory 402 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0094] Bus 403 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 403 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 403 in the accompanying drawings of this application is not limited to only one bus 403 or one type of bus 403.

[0095] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0096] An exemplary storage medium is coupled to processor 401, enabling processor 401 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of processor 401. Processor 401 and storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, processor 401 and storage medium can exist as discrete components in an electronic device or host device.

[0097] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intent recognition method based on a general large language model, characterized in that, Includes the following steps: Obtain the input text, its corresponding historical dialogue records, and the output text to construct training data; Construct a general large language model, build basic instructions for intent recognition according to business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list. The intent recognition basic instruction, the candidate intent list, and the historical dialogue record corresponding to the input text are combined to obtain a first intent recognition instruction. The first intent recognition instruction is used to adjust the instruction of the general large language model to obtain a first intent recognition model. In response to determining that the intent recognition accuracy of the first intent recognition model is less than an accuracy threshold, input-output samples are constructed. The first intent recognition instruction is combined with the input-output samples to obtain a second intent recognition instruction. The second intent recognition instruction is used to adjust the instruction of the general large language model to obtain a second intent recognition model. In response to determining that the intent recognition accuracy of the second intent recognition model is less than an accuracy threshold, an instruction-supervised fine-tuning dataset is constructed based on the first or second intent recognition instruction and training data, specifically including: The candidate intent list in the first intent recognition instruction or the second intent recognition instruction is adjusted to include the output text and an enhanced candidate intent list containing several intent categories randomly selected from the candidate intent list, to obtain the data-enhanced first intent recognition instruction or the second intent recognition instruction. The first or second intent recognition instruction augmented by the data is combined with the corresponding training data to form an instruction-supervised fine-tuning dataset. The second intent recognition model is then fine-tuned using the instruction-supervised fine-tuning dataset to obtain the fine-tuned intent recognition model. The system obtains the current input text or the current input text and its corresponding historical dialogue records, and constructs a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold are used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

2. The intent recognition method based on a general large language model according to claim 1, characterized in that, Based on the candidate intent list corresponding to the input text, intent recognition test cases are constructed by enumeration and combined with written or spoken language. The intent recognition test cases include a total of N pairs of test input texts and test output texts. The test input texts in the intent recognition test cases are input into the first intent recognition model, the second intent recognition model, or a fine-tuned intent recognition model to obtain the predicted output text. The intent recognition accuracy is the number of times the predicted output text is the same as the test output text divided by N.

3. The intent recognition method based on a general large language model according to claim 1, characterized in that, The first intent recognition instruction is combined with the input / output sample to obtain the second intent recognition instruction, specifically including: In response to determining that the number of input / output samples is greater than or equal to a number threshold, the similarity between the embedding vector of the input / output sample and the embedding vector of the input text is calculated. The similarity includes cosine similarity or Euclidean distance. Input / output samples whose similarity between the embedding vector of the input / output sample and the embedding vector of the input text is greater than the similarity threshold are selected as filtered input / output samples. The filtered input / output samples are then concatenated with the first intent recognition instruction to obtain the second intent recognition instruction. In response to determining that the number of input / output samples is less than a quantity threshold, the input / output samples are directly concatenated with the first intent recognition instruction to obtain a second intent recognition instruction.

4. The intent recognition method based on a general large language model according to claim 1, characterized in that, Also includes: In response to determining that the intent recognition accuracy of the first intent recognition model is greater than or equal to the accuracy threshold, the first intent recognition model is used as the large intent recognition model. In response to determining that the intent recognition accuracy of the second intent recognition model is greater than or equal to the accuracy threshold, the second intent recognition model is used as the large intent recognition model. In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to an accuracy threshold, the fine-tuned intent recognition model is used as the large intent recognition model. In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is less than an accuracy threshold, the input / output samples and / or the instruction-supervised fine-tuning dataset are continuously adjusted until the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold.

5. The intent recognition method based on a general large language model according to claim 1, characterized in that, The general large language model includes the Transformer model, the candidate intent list also includes intent descriptions, and the candidate intent list also includes an option with the intent category "other".

6. An intent recognition device based on a general large language model, characterized in that, include: The training data construction module is configured to acquire input text and its corresponding historical dialogue records and output text to construct training data. The model building module is configured to build a general large language model, build basic instructions for intent recognition according to business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list. The model adjustment module is configured to combine the basic intent recognition instruction, the candidate intent list, and the historical dialogue record corresponding to the input text to obtain a first intent recognition instruction; use the first intent recognition instruction to adjust the instruction of the general large language model to obtain a first intent recognition model; in response to determining that the intent recognition accuracy of the first intent recognition model is less than an accuracy threshold, construct input-output samples, combine the first intent recognition instruction with the input-output samples to obtain a second intent recognition instruction, use the second intent recognition instruction to adjust the instruction of the general large language model to obtain a second intent recognition model; in response to determining that the intent recognition accuracy of the second intent recognition model is less than an accuracy threshold, construct an instruction-supervised fine-tuning dataset based on the first intent recognition instruction or the second intent recognition instruction and training data, specifically including: The candidate intent list in the first intent recognition instruction or the second intent recognition instruction is adjusted to include the output text and an enhanced candidate intent list containing several intent categories randomly selected from the candidate intent list, to obtain the data-enhanced first intent recognition instruction or the second intent recognition instruction. The first or second intent recognition instruction augmented by the data is combined with the corresponding training data to form an instruction-supervised fine-tuning dataset. The second intent recognition model is then fine-tuned using the instruction-supervised fine-tuning dataset to obtain the fine-tuned intent recognition model. The recognition module is configured to acquire the current input text or the current input text and its corresponding historical dialogue records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold is used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

7. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images