General large language model-based intention recognition method and device and readable medium

Abstract

The invention discloses an intention recognition method and device based on a general big language model and a readable medium, and the method comprises the steps: combining an intention recognition basic instruction, a candidate intention list and a historical dialogue record to obtain a first intention recognition instruction, and carrying out the instruction adjustment of the general big language model to obtain a first intention recognition model; if the accuracy is smaller than a threshold value, combining the first intention recognition instruction with an input and output sample to obtain a second intention recognition instruction, and performing instruction adjustment on the general large language model to obtain a second intention recognition model; if the accuracy is smaller than a threshold value, constructing an instruction supervision fine tuning data set according to the first intention recognition instruction or the second intention recognition instruction and training data, and performing fine tuning on the second intention recognition model to obtain a fine-tuned intention recognition model; and taking the first intention recognition model, the second intention recognition model or the fine-tuned intention recognition model of which the accuracy is greater than or equal to a threshold value as a large intention recognition model, and performing intention classification.

Description

Technical Field

[0001] This invention relates to the field of intent recognition, and more specifically to an intent recognition method, apparatus, and readable medium based on a general large language model. Background Technology

[0002] Currently, large-scale models and their applications are experiencing a golden age of rapid development, profoundly changing the face of various industries. These large-scale machine learning models, with their massive parameter scale, complex network structure, and massive training data, have demonstrated unprecedented learning and generalization capabilities.

[0003] In fields such as natural language processing, computer vision, speech recognition, and recommendation systems, large-scale models have become a key force driving technological innovation and application upgrades. In natural language processing, large language models such as the GPT series and Qianwen have not only achieved significant results in text generation, question-answering systems, and sentiment analysis, but are also gradually penetrating professional fields such as education, healthcare, and law, providing intelligent assistance and decision support. They can understand and generate coherent and logical text, greatly improving the fluency and naturalness of human-computer interaction. However, in the deep integration of large-scale models with business systems, any misunderstanding or error can cause significant problems in the overall workflow of the business system, turning what was once an aid to the large-scale model and business system into a burden. Therefore, how to quickly and accurately interpret the intent of language text has become a key technology for combining large-scale models with applications.

[0004] As the informatization level of various business departments continues to improve, the business systems they build are becoming increasingly diversified. With the increasing integration of these systems, the demands on their intelligence also present greater challenges. A significant characteristic is that the number of intent branches that the system needs to identify is increasing, and the intent recognition results are becoming more and more similar. Traditional intent recognition based on large models only utilizes the capabilities of the large model itself and fails to effectively integrate specific business scenarios. For example, when considering opening a specific video versus opening a specific video pre-setup, traditional intent recognition will miss the mark between the video and menu intent categories, resulting in an accuracy rate of generally around 70%. This accuracy fails to meet application requirements, leading to a poor user experience. Summary of the Invention

[0005] The purpose of this application is to propose an intent recognition method, apparatus, and readable medium based on a general large language model to address the aforementioned technical problems.

[0006] In a first aspect, the present invention provides an intent recognition method based on a general large language model, comprising the following steps:

[0007] Obtain the input text, its corresponding historical dialogue records, and the output text to construct training data;

[0008] Construct a general large language model, build basic instructions for intent recognition based on business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list.

[0009] The first intent recognition instruction is obtained by combining the basic intent recognition command, the candidate intent list, and the historical dialogue record corresponding to the input text. The first intent recognition instruction is then used to adjust the general large language model to obtain the first intent recognition model. In response to the determination that the intent recognition accuracy of the first intent recognition model is less than the accuracy threshold, input-output samples are constructed. The first intent recognition instruction is combined with the input-output samples to obtain the second intent recognition instruction. The second intent recognition instruction is then used to adjust the general large language model to obtain the second intent recognition model. In response to the determination that the intent recognition accuracy of the second intent recognition model is less than the accuracy threshold, an instruction-supervised fine-tuning dataset is constructed based on the first or second intent recognition instruction and the training data. The instruction-supervised fine-tuning dataset is then used to fine-tune the second intent recognition model to obtain the fine-tuned intent recognition model.

[0010] The system obtains the current input text or the current input text and its corresponding historical dialogue records, and constructs a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold are used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

[0011] Preferably, intent recognition test cases are constructed based on the candidate intent list corresponding to the input text through enumeration and in combination with written or spoken language. The intent recognition test cases include a total of N pairs of test input texts and test output texts. The test input texts in the intent recognition test cases are input into the first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model to obtain the predicted output text. The intent recognition accuracy is the number of times the predicted output text is the same as the test output text divided by N.

[0012] Preferably, the first intent recognition instruction is combined with the input / output sample to obtain the second intent recognition instruction, which specifically includes:

[0013] In response to determining that the number of input / output samples is greater than or equal to a threshold, the similarity between the embedding vector of the input / output sample and the embedding vector of the input text is calculated. The similarity includes cosine similarity or Euclidean distance. Input / output samples whose similarity between the embedding vector of the input / output sample and the embedding vector of the input text is greater than the similarity threshold are selected as filtered input / output samples. The filtered input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0014] In response to determining that the number of input / output samples is less than a quantity threshold, the direct input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0015] Preferably, an instruction-supervised fine-tuning dataset is constructed based on the first intent recognition instruction or the second intent recognition instruction and the training data, specifically including:

[0016] The candidate intent list in the first intent recognition instruction or the second intent recognition instruction is adjusted to include the output text and an enhanced candidate intent list containing several intent categories randomly selected from the candidate intent list, thus obtaining the data-enhanced first intent recognition instruction or the second intent recognition instruction.

[0017] The first or second intent recognition instruction augmented with data is combined with the corresponding training data to form an instruction-supervised fine-tuning dataset.

[0018] As a preferred option, it also includes:

[0019] In response to determining that the intent recognition accuracy of the first intent recognition model is greater than or equal to the accuracy threshold, the first intent recognition model is used as the large intent recognition model.

[0020] In response to determining that the intent recognition accuracy of the second intent recognition model is greater than or equal to the accuracy threshold, the second intent recognition model is used as the large intent recognition model.

[0021] In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold, the fine-tuned intent recognition model is taken as the large intent recognition model.

[0022] In response to the determination that the intent recognition accuracy of the fine-tuned intent recognition model is less than the accuracy threshold, the input and output samples and / or instructions are continuously adjusted to supervise the fine-tuning dataset until the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold.

[0023] As a preferred option, the general large language model includes the Transformer model, and the candidate intent list also includes intent descriptions, with an option for intent category "other" added to the candidate intent list.

[0024] Secondly, the present invention provides an intent recognition device based on a general large language model, comprising:

[0025] The training data construction module is configured to acquire input text and its corresponding historical dialogue records and output text to construct training data.

[0026] The model building module is configured to build a general large language model, build basic instructions for intent recognition according to business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list.

[0027] The model adjustment module is configured to combine the basic intent recognition command, the candidate intent list, and the historical dialogue record corresponding to the input text to obtain a first intent recognition command. The first intent recognition command is then used to adjust the general large language model to obtain a first intent recognition model. In response to determining that the intent recognition accuracy of the first intent recognition model is less than an accuracy threshold, input-output samples are constructed, and the first intent recognition command is combined with the input-output samples to obtain a second intent recognition command. The second intent recognition command is then used to adjust the general large language model to obtain a second intent recognition model. In response to determining that the intent recognition accuracy of the second intent recognition model is less than an accuracy threshold, an instruction-supervised fine-tuning dataset is constructed based on the first or second intent recognition command and training data. The instruction-supervised fine-tuning dataset is then used to fine-tune the second intent recognition model to obtain a fine-tuned intent recognition model.

[0028] The recognition module is configured to acquire the current input text or the current input text and its corresponding historical dialogue records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold is used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

[0029] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0030] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0031] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) The intent recognition method based on the general large language model proposed in this invention not only lowers the threshold of intent recognition but also greatly improves the accuracy of intent recognition in business systems by combining the contextual capabilities of the general large language model with input and output examples, threshold vector matching degree and instruction fine-tuning datasets and other multi-dimensional adjustment methods.

[0034] (2) The intent recognition method based on the general large language model proposed in this invention retains the capabilities of the general large language model, while allowing users to select the corresponding candidate intent list according to their business category to complete the intent recognition of their respective business, making the business system more intelligent. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the intent recognition method based on a general large language model, as an embodiment of this application.

[0037] Figure 2 A flowchart illustrating the intent recognition method based on a general large language model, as an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of an intent recognition device based on a general large language model, which is an embodiment of this application.

[0039] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0041] Figure 1 The following shows a method for intent recognition based on a general large language model provided by an embodiment of the present application, including the following steps:

[0042] S1. Obtain the input text, its corresponding historical conversation record, and the output text to construct training data.

[0043] Specifically, refer to Figure 2 , collect a number of input texts, their corresponding historical conversation records, and output texts. Here, the output text is the intent category of the input text or the input text and its corresponding historical conversation record, and finally, training data is constructed. Obtain the historical conversation record γ, for example: γ = "<Historical conversation>< / Historical conversation>", usually, the first 5 rounds of historical conversations can be obtained. In other embodiments, it can be adjusted according to specific scenarios.

[0044] S2. Construct a general large language model, construct a basic instruction for intent recognition according to business requirements, and construct a candidate intent list according to the business categories mentioned in the input text and its historical conversation record. The basic instruction for intent recognition requires comprehensive consideration of the input text and its corresponding historical conversation record for intent recognition, and requires that the output text of the general large language model must be the intent category in the candidate intent list.

[0045] In a specific embodiment, the general large language model includes a Transformer model. The candidate intent list also includes an intent description, and an option with the intent category of "other" is also added to the candidate intent list.

[0046] Specifically, the general large language model in the embodiment of the present application can adopt a Transformer model. Specifically, select a suitable large language model according to existing resources, taking into account multiple requirements such as computing power resources, response requirements, and recognition accuracy. It is recommended to select a model with smaller parameters, such as a 7B model.

[0047] Construct a basic instruction for intent recognition α according to business requirements. During the construction of the basic instruction for intent recognition, the output text must be forced to be restricted to avoid the phenomenon of the large language model answering randomly, resulting in a decrease in the intent recognition accuracy. For example: "Please note that your output must be in this candidate intent list, and only output the name of the intent category without outputting the intent description".

[0048] The following gives a specific example of the basic instruction for intent recognition: α = "<Instruction>You are an intent recognition expert. Your job is to perform intent recognition on the content entered by the user, and you need to comprehensively consider the historical conversation record and the user's current input text. If the user's question can be found in the historical conversation record, please return 'Other'. The following is the known list of candidate intents. Note that your output must be in this list of candidate intents, and only the name of the intent category should be output without the intent description.< / Instruction>".

[0049] Construct a list of candidate intents β according to the specific scenario. Each intent represents a business branch, and the intent category is related to the business category; add the option of "Other" to the list of candidate intents to reserve the Q&A function of the large language model itself and avoid the drawback that the model evolution can only be used for intent recognition. For example: β = "<Classification List>1. Make a call\n 2. Open a video\n3. Open a menu\n…99. Other< / Classification List>".

[0050] S3. Combine the basic instruction for intent recognition, the list of candidate intents, and the historical conversation record corresponding to the input text to obtain the first intent recognition instruction. Use the first intent recognition instruction to adjust the instructions of the general large language model to obtain the first intent recognition model; in response to determining that the intent recognition accuracy of the first intent recognition model is less than the accuracy threshold, construct input-output examples, combine the first intent recognition instruction with the input-output examples to obtain the second intent recognition instruction, and use the second intent recognition instruction to adjust the instructions of the general large language model to obtain the second intent recognition model; in response to determining that the intent recognition accuracy of the second intent recognition model is less than the accuracy threshold, construct an instruction supervised fine-tuning data set according to the first intent recognition instruction or the second intent recognition instruction and the training data, and use the instruction supervised fine-tuning data set to fine-tune the second intent recognition model to obtain the fine-tuned intent recognition model.

[0051] Specifically, concatenate the above basic instruction for intent recognition α, the list of candidate intents β, and the historical conversation record γ to obtain the first intent recognition instruction x1 = α + β + γ. Directly use this first intent recognition instruction x1 to interface with the general large language model to obtain the first intent recognition model. The complexity of the intent recognition scenario is different. The more complex, the more similar, and the more difficult to distinguish. For simple scenarios, the first intent recognition model using the first intent recognition instruction x1 can already meet the requirements.

[0052] In a specific embodiment, intent recognition test cases are constructed based on the candidate intent list corresponding to the input text through enumeration and in combination with written or spoken language. The intent recognition test cases include a total of N pairs of test input texts and test output texts. The test input texts in the intent recognition test cases are input into the first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model to obtain the predicted output text. The intent recognition accuracy is the number of times the predicted output text is the same as the test output text divided by N.

[0053] Specifically, intent recognition test cases are constructed. Based on the candidate intent list, intent recognition test cases are constructed using enumeration methods and combined with different methods such as written language and spoken language. Special attention is paid to the construction of spoken language test cases. In daily questions, the proportion of spoken language is much greater than that of written language. For example, if the intent category is: obtain tidal data, the written language is: check today's tidal information; the spoken language is: when is the tide going out today.

[0054] Furthermore, the intent recognition accuracy of the first intent recognition model is tested using intent recognition test cases. The intent recognition accuracy is calculated as the number of times the predicted output text generated by the first intent recognition model is the same as the test output text in the intent recognition test cases, divided by the total number of intent recognition test cases N.

[0055] In a specific embodiment, the first intent recognition instruction is combined with the input / output sample to obtain the second intent recognition instruction, which specifically includes:

[0056] In response to determining that the number of input / output samples is greater than or equal to a threshold, the similarity between the embedding vector of the input / output sample and the embedding vector of the input text is calculated. The similarity includes cosine similarity or Euclidean distance. Input / output samples whose similarity between the embedding vector of the input / output sample and the embedding vector of the input text is greater than the similarity threshold are selected as filtered input / output samples. The filtered input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0057] In response to determining that the number of input / output samples is less than a quantity threshold, the direct input / output samples are concatenated with the first intent recognition instruction to obtain the second intent recognition instruction.

[0058] Specifically, if the intention recognition accuracy rate of the first intention recognition model fails to meet the accuracy requirement, i.e., is less than the accuracy threshold, then input-output examples δ are appended to the first intention recognition instruction; the input-output examples are generally divided into two parts: "input" and "output". For example, δ = "<example><input>His corporate phone number< / input><output>Make a call< / output><input>I want to see accident cases< / input><output>Adjust the menu< / output>< / example>". If there are too many input-output examples, the input-output examples whose embedding vectors are relatively close to the embedding vector of the input text are screened out by calculating cosine similarity, Euclidean distance, etc., so as to obtain the screened input-output examples. The first intention recognition instruction is concatenated with the screened input-output examples or the input-output examples δ to obtain the second intention recognition instruction x2 = α + β + γ + δ.

[0059] Furthermore, directly use this second intention recognition instruction x2 to interface with the general large language model to obtain the second intention recognition model. Use intention recognition test examples to test the intention recognition accuracy rate of the second intention recognition model. At this time, the intention recognition accuracy rate is the number of the predicted output texts generated by inputting the test input texts in the intention recognition test examples into the second intention recognition model that are the same as the test output texts divided by the total number N of the intention recognition test examples. If the intention recognition accuracy rate of the second intention recognition model fails to meet the accuracy requirement, i.e., is less than the accuracy threshold, then the model is fine-tuned through the instruction supervision fine-tuning data set.

[0060] In a specific embodiment, an instruction supervision fine-tuning data set is constructed according to the first intention recognition instruction or the second intention recognition instruction and the training data, which specifically includes:

[0061] Adjust the candidate intention list in the first intention recognition instruction or the second intention recognition instruction to an enhanced candidate intention list that includes the output text and several intention categories randomly selected from the candidate intention list, so as to obtain the data-augmented first intention recognition instruction or the second intention recognition instruction;

[0062] The data-augmented first intention recognition instruction or the second intention recognition instruction and the corresponding training data are respectively used to form an instruction supervision fine-tuning data set.

[0063] Specifically, by constructing an instruction fine-tuning dataset based on intent sampling, the system prevents the model from collapsing after fine-tuning. For example, if the original candidate intent list contains 10 intents, only 6-7 different intents can be selected when constructing the instruction fine-tuning dataset. An example from an embodiment of this application is as follows: "{"instruction":"You are a professional intent recognition expert. Based on the following candidate intent list, and by fully combining historical dialogue records and the current user's input text, accurately select the name of the intent category that best matches the user's intent from the candidate intent list. \nCandidate Intent List:\n1. Access Video: Open, call, view, or query video, video plan, monitoring, or recording.\n2. Enterprise Profile: Query relevant enterprise profile information.\n3. Send Notification: Send a notification or message to the user or system.\n4. Upload File: Upload a file to the system.\n5. Access Menu: Call, open, or query a specific function menu.\n6. Other: If none of the above intents apply, then it is "Other".\nRestrictions:\n-The output must be the name of the intent category in the given candidate intent list; you may not create or modify the category yourself." \n- Output only the name of the intent category, without adding any explanation, description or other text. ","input":"input:View menu for highlight videos","output":"Induce menu"}.

[0064] The embodiment of this application provides another example as follows: "{"instruction":"You are a professional intent recognition expert. Based on the following candidate intent list, fully combine historical dialogue records and the current user's input text to make a comprehensive judgment, accurately select the name of the intent category that best matches the user's intent from the candidate intent list.\nCandidate intent list:\n1. Monitoring environment: View monitoring information of physical or virtual environment.\n2. Adjust video: Open, call, view, or query video, video plan, monitoring, or recording.\n3. Adjust menu: Call, open, or query a function menu.\n4. Other: If none of the following intents are met, it is "Other".\nRestrictions:\n-The output must be the name of the intent category in the given candidate intent list. You may not create or modify the category yourself.\n-Only output the name of the intent category, without adding any explanation, description, or other text.","input":"input: Call the video plan of Xiamen Bridge","output":"Adjust video"}.

[0065] The second intent recognition model was fine-tuned using the above instruction fine-tuning dataset to obtain the fine-tuned intent recognition model.

[0066] In specific embodiments, it also includes:

[0067] In response to determining that the intent recognition accuracy of the first intent recognition model is greater than or equal to the accuracy threshold, the first intent recognition model is used as the large intent recognition model.

[0068] In response to determining that the intent recognition accuracy of the second intent recognition model is greater than or equal to the accuracy threshold, the second intent recognition model is used as the large intent recognition model.

[0069] In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold, the fine-tuned intent recognition model is taken as the large intent recognition model.

[0070] In response to the determination that the intent recognition accuracy of the fine-tuned intent recognition model is less than the accuracy threshold, the input and output samples and / or instructions are continuously adjusted to supervise the fine-tuning dataset until the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold.

[0071] Specifically, if the intent recognition accuracy of the first intent recognition model can meet the accuracy requirement, that is, greater than or equal to the intent recognition accuracy, then the first intent recognition model will be directly used as the large intent recognition model.

[0072] If the intent recognition accuracy of the first intent recognition model cannot meet the accuracy requirement, while the intent recognition accuracy of the second intent recognition model can meet the accuracy requirement, then the second intent recognition model will be used as the larger intent recognition model.

[0073] If the intent recognition accuracy of both the first and second intent recognition models fails to meet the accuracy requirements, but the intent recognition accuracy of the fine-tuned intent recognition model meets the accuracy requirements, then the fine-tuned intent recognition model will be used as the large intent recognition model.

[0074] If the intent recognition accuracy of the fine-tuned intent recognition model fails to meet the accuracy requirements, the above steps are repeated to continuously adjust the input / output samples and the instruction fine-tuning dataset until the intent recognition accuracy of the fine-tuned intent recognition model meets the accuracy requirements. In one embodiment, after testing in a business system, the intent recognition accuracy reached over 95%, which basically meets the application needs of most business systems and can be well integrated into the business system.

[0075] S4. Obtain the current input text or the current input text and its corresponding historical dialogue records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. Use the first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold as the large intent recognition model. Input the current input text or the current input text and its corresponding historical dialogue records and the candidate intent list into the large intent recognition model to generate the corresponding intent category.

[0076] Specifically, as the number of business systems increases and the degree of information integration becomes higher, for example, a system may need to recognize dozens of different intent categories such as video, menus, duty shifts, and command and dispatch (including the dispatch of various command elements). Therefore, deploying the finalized intent recognition model in the business system can more accurately obtain more refined intent categories. By obtaining the user's current input text or the current input text and its corresponding historical dialogue records, as well as the constructed candidate intent list, and inputting them into the intent recognition model, the intent category corresponding to the current input text or the current input text and its corresponding historical dialogue records can be generated.

[0077] Further reference Figure 3 As an implementation of the methods shown in the above figures, this application provides an embodiment of an intent recognition device based on a general large language model. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0078] This application provides an intent recognition device based on a general large language model, comprising:

[0079] Training data construction module 1 is configured to acquire input text and its corresponding historical dialogue records and output text to construct training data;

[0080] Model building module 2 is configured to build a general large language model, build basic instructions for intent recognition according to business requirements, and build a candidate intent list based on the business categories mentioned in the input text and its historical dialogue records. The basic instructions for intent recognition require comprehensive consideration of the input text and its corresponding historical dialogue records for intent recognition, and require that the output text of the general large language model must be the intent category in the candidate intent list.

[0081] Model adjustment module 3 is configured to combine the basic intent recognition instruction, the candidate intent list, and the historical dialogue record corresponding to the input text to obtain a first intent recognition instruction; use the first intent recognition instruction to adjust the general large language model to obtain a first intent recognition model; in response to determining that the intent recognition accuracy of the first intent recognition model is less than the accuracy threshold, construct input-output samples, combine the first intent recognition instruction with the input-output samples to obtain a second intent recognition instruction, use the second intent recognition instruction to adjust the general large language model to obtain a second intent recognition model; in response to determining that the intent recognition accuracy of the second intent recognition model is less than the accuracy threshold, construct an instruction-supervised fine-tuning dataset based on the first intent recognition instruction or the second intent recognition instruction and the training data, and use the instruction-supervised fine-tuning dataset to fine-tune the second intent recognition model to obtain a fine-tuned intent recognition model.

[0082] The recognition module 4 is configured to acquire the current input text or the current input text and its corresponding historical dialogue records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical dialogue records. The first intent recognition model, the second intent recognition model, or the fine-tuned intent recognition model with an intent recognition accuracy greater than or equal to the accuracy threshold is used as the large intent recognition model. The current input text or the current input text and its corresponding historical dialogue records and the candidate intent list are input into the large intent recognition model to generate the corresponding intent category.

[0083] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 4 As shown, the electronic device in this embodiment includes a processor 401 and a memory 402; wherein the memory 402 is used to store computer execution instructions; and the processor 401 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0084] Alternatively, the memory 402 can be either standalone or integrated with the processor 401.

[0085] When the memory 402 is set up independently, the electronic device also includes a bus 403 for connecting the memory 402 and the processor 401.

[0086] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 401, implement the above method.

[0087] This invention also provides a computer program product, including a computer program, which, when executed by a processor 401, implements the above-described method.

[0088] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0089] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0090] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0091] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 401 to execute some steps of the methods of the various embodiments of this application.

[0092] It should be understood that the processor 401 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 401 can be any conventional processor 401. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 401 executing the steps, or as a combination of hardware and software modules within the processor 401 executing the steps.

[0093] The memory 402 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0094] Bus 403 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 403 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 403 in the accompanying drawings of this application is not limited to only one bus 403 or one type of bus 403.

[0095] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0096] An exemplary storage medium is coupled to processor 401, enabling processor 401 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of processor 401. Processor 401 and storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, processor 401 and storage medium can exist as discrete components in an electronic device or host device.

[0097] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intent recognition based on a general large language model, characterized in that: The following steps are involved: Obtain input text and its corresponding historical conversation records and output text to construct training data; Construct a general large language model, construct basic instructions for intent recognition according to business requirements, and construct a candidate intent list according to the business categories mentioned in the input text and its historical conversation records. The basic instructions for intent recognition require that the input text and its corresponding historical conversation records be considered comprehensively for intent recognition, and require that the output text of the general large language model must be an intent category in the candidate intent list; The basic intention recognition instruction, the candidate intention list and the historical conversation record corresponding to the input text are combined to obtain a first intention recognition instruction, and the first intention recognition instruction is used to adjust the general large language model to obtain a first intention recognition model; in response to determining that the intention recognition accuracy of the first intention recognition model is less than the accuracy threshold, input and output samples are constructed, the first intention recognition instruction is combined with the input and output samples to obtain a second intention recognition instruction, and the second intention recognition instruction is used to adjust the general large language model to obtain a second intention recognition model; in response to determining that the intention recognition accuracy of the second intention recognition model is less than the accuracy threshold, an instruction supervision fine-tuning dataset is constructed according to the first intention recognition instruction or the second intention recognition instruction and the training data, and the second intention recognition model is fine-tuned using the instruction supervision fine-tuning dataset to obtain a fine-tuned intention recognition model; Obtain the current input text or the current input text and its corresponding historical conversation records, and build a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical conversation records, use the first intent recognition model, the second intent recognition model or the fine-tuned intent recognition model whose intent recognition accuracy is greater than or equal to the accuracy threshold as the intent recognition big model, input the current input text or the current input text and its corresponding historical conversation records and the candidate intent list into the intent recognition big model, and generate a corresponding intent category.

2. The method for identifying intentions based on a general large language model according to claim 1, characterized in that: An intent recognition test sample is constructed according to the candidate intent list corresponding to the input text by enumeration method and in combination with written or spoken language. The intent recognition test sample includes a total of N pairs of test input texts and test output texts. The test input text in the intent recognition test sample is input into the first intent recognition model, the second intent recognition model or the fine-tuned intent recognition model to obtain a predicted output text. The intent recognition accuracy rate is the number of the predicted output texts that are the same as the test output texts divided by N.

3. The method for intention recognition based on a general large language model according to claim 1, characterized in that: Combining the first intent recognition instruction with the input and output samples to obtain a second intent recognition instruction specifically includes: In response to determining that the number of the input and output samples is greater than or equal to the number threshold, calculating the similarity between the embedding vector of the input and output sample and the embedding vector of the input text, the similarity including cosine similarity or Euclidean distance, screening out input and output samples whose similarity between the embedding vector of the input and output sample and the embedding vector of the input text is greater than the similarity threshold as screened input and output samples, and splicing the screened input and output samples with the first intent recognition instruction to obtain a second intent recognition instruction; In response to determining that the number of the input-output samples is less than the number threshold, the input-output samples are directly concatenated with the first intent recognition instruction to obtain a second intent recognition instruction.

4. The method for identifying intentions based on a general large language model according to claim 1, characterized in that: Constructing an instruction supervision fine-tuning dataset according to the first intent recognition instruction or the second intent recognition instruction and the training data, specifically comprising: The candidate intent list in the first intent recognition instruction or the second intent recognition instruction is adjusted to an enhanced candidate intent list including the output text and a plurality of intent categories randomly selected from the candidate intent list, to obtain a data-enhanced first intent recognition instruction or a second intent recognition instruction; The data-enhanced first intent recognition instruction or the second intent recognition instruction is respectively combined with the corresponding training data to form an instruction supervision fine-tuning data set.

5. The method for identifying intentions based on a general large language model according to claim 1, characterized in that: Also includes: In response to determining that the intent recognition accuracy of the first intent recognition model is greater than or equal to an accuracy threshold, using the first intent recognition model as a large intent recognition model; In response to determining that the intent recognition accuracy of the second intent recognition model is greater than or equal to an accuracy threshold, using the second intent recognition model as a large intent recognition model; In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold, using the fine-tuned intent recognition model as a large intent recognition model; In response to determining that the intent recognition accuracy of the fine-tuned intent recognition model is less than an accuracy threshold, the input and output samples and / or the instruction supervision fine-tuning data set are continuously adjusted until the intent recognition accuracy of the fine-tuned intent recognition model is greater than or equal to the accuracy threshold.

6. The method for intention recognition based on a general large language model according to claim 1, characterized in that: The general large language model includes a Transformer model, the candidate intent list also includes intent descriptions, and an option with intent category "other" is added to the candidate intent list.

7. An intention recognition device based on a general large language model, characterized in that: include: A training data construction module is configured to obtain input text and its corresponding historical conversation record and output text to construct training data; A model building module is configured to build a general large language model, build an intent recognition basic instruction according to business requirements, and build a candidate intent list according to the business categories mentioned in the input text and its historical conversation records, wherein the intent recognition basic instruction requires comprehensive consideration of the input text and its corresponding historical conversation records for intent recognition, and requires that the output text of the general large language model must be an intent category in the candidate intent list; A model adjustment module is configured to combine the basic intention recognition instruction, the candidate intention list and the historical conversation record corresponding to the input text to obtain a first intention recognition instruction, and use the first intention recognition instruction to adjust the general large language model to obtain a first intention recognition model; in response to determining that the intention recognition accuracy of the first intention recognition model is less than an accuracy threshold, construct an input and output sample, combine the first intention recognition instruction with the input and output sample to obtain a second intention recognition instruction, and use the second intention recognition instruction to adjust the general large language model to obtain a second intention recognition model; in response to determining that the intention recognition accuracy of the second intention recognition model is less than the accuracy threshold, construct an instruction supervision fine-tuning dataset based on the first intention recognition instruction or the second intention recognition instruction and the training data, and use the instruction supervision fine-tuning dataset to fine-tune the second intention recognition model to obtain a fine-tuned intention recognition model; The recognition module is configured to obtain the current input text or the current input text and its corresponding historical conversation records, and construct a corresponding candidate intent list based on the current input text or the current input text and its corresponding historical conversation records, use the first intent recognition model, the second intent recognition model or the fine-tuned intent recognition model whose intent recognition accuracy is greater than or equal to the accuracy threshold as the intent recognition big model, input the current input text or the current input text and its corresponding historical conversation records and the candidate intent list into the intent recognition big model, and generate a corresponding intent category.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images