A method and device for constructing a training set for large model intent recognition fine-tuning, and related products

Abstract

This application provides a method, apparatus, and related products for constructing a training set for fine-tuning intent recognition of a large model, relating to the field of artificial intelligence technology. The method constructs and generates a training set to fine-tune the intent recognition of a third-large model. The model parameters of this third-large model are smaller than those of the first-large model, allowing the fine-tuned third-large model to replace the online analysis and inference work of the first-large model, saving computational resources. Furthermore, each training data sample in this application's training set uses intent category-level labels, effectively increasing the distinguishability between different categories during training, thereby improving the accuracy of intent recognition. In addition, each training data sample in this application's training set uses multiple intent discrimination reasons corresponding to intent category-level labels, employing a synonym random sampling mode, which not only overcomes the problem of homogeneity in the description of the same intent discrimination reason but also enhances the model's expressive power.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus and related products for constructing a training set for fine-tuning intent recognition of large models. Background Technology

[0002] In recent years, with the development of AI (Artificial Intelligence), numerous large-scale modeling technologies have been applied to automated question-answering scenarios. For example, in the customer service field, relatively mature scenarios such as insurance after-sales service have already introduced chatbot question-answering systems. In current agent-based question-answering systems, intent recognition is a key issue; only by accurately recognizing the user's intent can the relevant agent be accurately invoked to execute tasks and respond to the user's conversation.

[0003] In existing technologies, business rules and dialogue logic are extracted manually and written into prompts to call a large model for responses. However, as dialogue cases accumulate, the prompts become very bloated, which is not only detrimental to project maintenance but also affects the response time during Q&A. Therefore, this technical problem urgently needs to be solved. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide a method, apparatus, and related products for constructing a training set for fine-tuning intent recognition of a large model that overcomes or at least partially solves the above problems. The technical solution is as follows: Firstly, a method for constructing a training set for fine-tuning intent recognition in large models is provided, the method comprising: Collect online interaction data; the interaction data is generated based on the first major model and user interaction. Based on the interaction data, multiple text pairs including historical interaction sessions and current user sessions are identified, along with preliminary intent annotations for the current user session within each text pair. The second major model is invoked to clean and supplement the preliminary intent annotation of the current user session in each of the multiple text pairs, determine the intent category hierarchical label of the current user session in the text pair, and generate multiple intent discrimination reasons corresponding to the intent category hierarchical label; wherein, the intent category hierarchical label includes a first category identifier and a second category identifier; wherein, the model parameters of the second major model are smaller than those of the first major model. Based on the text pair, the intent category hierarchical label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchical label, construct a training data sample corresponding to the text pair; A training set is generated based on the training data samples corresponding to multiple text pairs; the training set is used to fine-tune the intent recognition of the third major model; the model parameters of the third major model are smaller than those of the first major model.

[0005] In one possible implementation, a training data sample corresponding to the text pair is constructed based on the text pair, the intent category hierarchical label of the current user session in the text pair, and multiple intent discrimination reasons corresponding to the intent category hierarchical label, including: Obtain a pre-created prompt template; wherein the prompt template includes a prompt sentence indicating intent recognition of the current user session based on historical interaction sessions, the fill position of the historical interaction sessions, and the fill position of the current user session; Based on the prompt word template, fill the historical interaction sessions in the text pair into the fill position of the historical interaction sessions, and fill the current user session in the text pair into the fill position of the current user session to obtain the model input data; The intent category hierarchy label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchy label, are used as the model output data; Based on the model input data and model output data, construct a training data sample corresponding to the text pair.

[0006] In one possible implementation, the method further includes: When fine-tuning the intent recognition of the third model using the training set, one intent discrimination reason is randomly selected from multiple intent discrimination reasons in the training data samples to generate a target training sample, and the intent recognition of the third model is fine-tuned using the target training sample.

[0007] In one possible implementation, the method further includes: Multiple preset intent categories are hierarchically divided into parent category layers and sub-category layers; Assign a unique first category identifier and a second category identifier to each intent category; the first category identifier is used to identify the category in the parent category layer, and the second category identifier is used to identify the category in the child category layer.

[0008] In one possible implementation, the multiple intent discrimination reasons corresponding to the intent category level labels of the current user session in the text pair are sentences with the same semantics but different descriptions.

[0009] Secondly, an apparatus for constructing a training set for fine-tuning intent recognition of a large model is provided, the apparatus comprising: The collection unit is used to collect online interaction data; the interaction data is generated based on the first model and user interaction. The first determining unit is used to determine, based on the interaction data, multiple text pairs including historical interaction sessions and current user sessions, and preliminary intent annotations for the current user session in each text pair. The second determining unit is used to call the second major model. For each text pair in the multiple text pairs, it cleans and supplements the preliminary intent annotation of the current user session in the text pair, determines the intent category level label of the current user session in the text pair, and generates multiple intent discrimination reasons corresponding to the intent category level label. The intent category level label includes a first category identifier and a second category identifier. The model parameters of the second major model are smaller than those of the first major model. The construction unit is used to construct a training data sample corresponding to the text pair based on the text pair, the intent category hierarchical label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchical label; The generation unit is used to generate a training set based on the corresponding training data samples of multiple text pairs; the training set is used to fine-tune the intent recognition of the third major model; the model parameters of the third major model are smaller than those of the first major model.

[0010] In one possible implementation, the building unit is also used for: Obtain a pre-created prompt template; wherein the prompt template includes a prompt sentence indicating intent recognition of the current user session based on historical interaction sessions, the fill position of the historical interaction sessions, and the fill position of the current user session; Based on the prompt word template, fill the historical interaction sessions in the text pair into the fill position of the historical interaction sessions, and fill the current user session in the text pair into the fill position of the current user session to obtain the model input data; The intent category hierarchy label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchy label, are used as the model output data; Based on the model input data and model output data, construct a training data sample corresponding to the text pair.

[0011] Thirdly, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method for constructing a training set for fine-tuning large model intent recognition as described in any of the preceding claims.

[0012] Fourthly, a storage medium is provided that stores a computer program, wherein the computer program is configured to execute, at runtime, the method for constructing a training set for fine-tuning intent recognition of a large model as described in any of the preceding claims.

[0013] Fifthly, a computer program product is provided, including a computer program configured to execute, at runtime, the method for constructing a training set for fine-tuning intent recognition of a large model as described in any of the preceding claims.

[0014] Using the above technical solutions, this application provides a method, apparatus, and related products for constructing a training set for fine-tuning intent recognition of a large model. This method generates a training set and fine-tunes the intent recognition of a third large model. The model parameters of the third large model are smaller than those of the first large model, allowing the fine-tuned third large model to replace the online analysis and inference work of the first large model, saving computational resources. Furthermore, each training data sample in this application's training set uses intent category-level labels, which effectively increases the distinguishability between different categories during training, thereby improving the accuracy of intent recognition. In addition, each training data sample in this application's training set uses multiple intent discrimination reasons corresponding to intent category-level labels, and can adopt a synonym random sampling mode, which not only overcomes the problem of homogeneity in the description of the same intent discrimination reason but also greatly enhances the model's expressive power. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0016] Figure 1 A flowchart illustrating a method for constructing a training set for fine-tuning intent recognition of a large model, as provided in an embodiment of this application, is shown. Figure 2 This paper shows a structural diagram of a device for constructing a training set for fine-tuning intent recognition of a large model, according to an embodiment of this application. Figure 3 A structural diagram of a device for constructing a training set for fine-tuning large model intent recognition, according to another embodiment of this application, is shown. Figure 4 A structural diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0017] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0019] To address the aforementioned technical problems, embodiments of this application provide a method for constructing a training set for fine-tuning intent recognition in large models, such as... Figure 1 As shown, the method for constructing the training set for fine-tuning the intent recognition of a large model may include the following steps S101 to S105: Step S101: Collect online accumulated interaction data; wherein, the interaction data is generated based on the interaction between the first model and users; Step S102: Based on the interaction data, determine multiple text pairs including historical interaction sessions and current user sessions, and the preliminary intent annotation of the current user session in each text pair. Step S103: Call the second major model. For each text pair in the multiple text pairs, clean and supplement the preliminary intent annotation of the current user session in the text pair, determine the intent category level label of the current user session in the text pair, and generate multiple intent discrimination reasons corresponding to the intent category level label; wherein, the intent category level label includes the first category identifier and the second category identifier; wherein, the model parameters of the second major model are smaller than the model parameters of the first major model. Step S104: Based on the text pair, the intent category hierarchical label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchical label, construct a training data sample corresponding to the text pair; Step S105: Generate a training set based on the training data samples corresponding to each of the multiple text pairs; wherein the training set is used to fine-tune the intent recognition of the third major model; wherein the model parameters of the third major model are smaller than the model parameters of the first major model.

[0020] The large-scale model in this embodiment has three characteristics: first, as the name suggests, it is large in scale, with network parameters reaching tens of billions, hundreds of billions, trillions, or even more; second, it is general, meaning it is not limited to specific problems or domains; and third, it is emergent, meaning it generates unexpected new capabilities.

[0021] The parameter size of the large model can be flexibly configured according to actual needs, such as different orders of magnitude like 235 billion, 72 billion, 4 billion, or 1.5 billion. The choice of parameter size mainly depends on computing resource conditions and model performance requirements, and this embodiment does not impose any restrictions on this. The first, second, and third large models can all be open-source large models.

[0022] This embodiment constructs and generates a training set to fine-tune the intent recognition of the third major model. The model parameters of the third major model are smaller than those of the first major model, so the fine-tuned third major model can replace the online analysis and inference work of the first major model, saving computing resources. Furthermore, each training data sample in this application's training set adopts an intent category hierarchical label, which can effectively increase the distinguishability between different categories during training, thereby improving the accuracy of intent recognition. In addition, each training data sample in this application's training set adopts multiple intent discrimination reasons corresponding to intent category hierarchical labels, and a synonym random sampling mode can be used, which not only overcomes the problem of homogeneity in the description of the same intent discrimination reason, but also greatly enhances the expressive power of the model.

[0023] This application embodiment provides a possible implementation method. Step S104 constructs a training data sample corresponding to the text pair based on the text pair, the intent category hierarchical label of the current user session in the text pair, and multiple intent discrimination reasons corresponding to the intent category hierarchical label. Specifically, it may include the following steps A1 to A4: Step A1: Obtain a pre-created prompt word template; wherein, the prompt word template includes a prompt sentence indicating intent recognition of the current user session based on historical interaction sessions, the fill position of the historical interaction sessions, and the fill position of the current user session; Step A2: Based on the prompt word template, fill the historical interaction sessions in the text pair into the fill position of the historical interaction sessions, and fill the current user session in the text pair into the fill position of the current user session to obtain the model input data. Step A3: Use the intent category hierarchy label of the current user session in the text pair and the multiple intent discrimination reasons corresponding to the intent category hierarchy label as model output data; Step A4: Based on the model input data and model output data, construct a training data sample corresponding to the text pair.

[0024] In this embodiment, when constructing training data samples, the prompts only describe the most basic information about the data without any manual logical prompts. This minimalist prompt design, driven by data, reduces the maintenance cost of the prompts in existing fine-tuning methods.

[0025] This application provides a possible implementation method, which may further include the following step B1: Step B1: When fine-tuning the intent recognition of the third model using the training set, randomly select one intent discrimination reason from multiple intent discrimination reasons in the training data samples to generate a target training sample, and use the target training sample to fine-tune the intent recognition of the third model.

[0026] In this embodiment, when fine-tuning the intent recognition of the third major model using the training set, one intent discrimination reason is selected from multiple intent discrimination reasons in the training data samples to generate a target training sample. The target training sample is then used to fine-tune the intent recognition of the third major model, which can increase the richness of the generated language.

[0027] This application provides a possible implementation method, which may further include the following steps C1 and C2: Step C1: Divide the preset multiple intent categories into hierarchical layers to form a parent category layer and a sub-category layer; Step C2: Assign a unique first category identifier and a second category identifier to each intent category; the first category identifier is used to identify the category in the parent category layer, and the second category identifier is used to identify the category in the child category layer.

[0028] This embodiment uses a generative paradigm for classification, employing a parent-child class pattern to construct supervised labels, thereby improving the accuracy of intent recognition.

[0029] This application provides a possible implementation method in which multiple intent discrimination reasons corresponding to the intent category level tags of the current user session in the text pair are sentences with the same semantics but different descriptions, thus increasing the richness of the generated language.

[0030] The above introduces Figure 1 The embodiments shown have various implementation methods for each step. The following will further explain the method for constructing the training set for fine-tuning large model intent recognition in this application through specific embodiments.

[0031] In a specific implementation, taking the insurance after-sales scenario as an example, a minimalist prompt word design is adopted, with hierarchical setting of category labels and multiple descriptions for the same intent discrimination reason. This not only simplifies the cost of prompt word maintenance but also improves the accuracy of classifying easily confused intents and the richness of intent reasons.

[0032] (1) Data organization.

[0033] 1.1) Collect online interaction data; where the interaction data is generated based on the interaction between the first major model and users; the model parameters of the first major model can be 235 billion.

[0034] 1.2) Based on the interaction data, determine multiple text pairs including historical interaction sessions and current user sessions, and the preliminary intent annotation of the current user session in each text pair.

[0035] 1.3) The second major model is invoked. For each text pair among multiple text pairs, the initial intent annotation of the current user session in that text pair is cleaned and supplemented to determine the intent category hierarchy label of the current user session in that text pair, and generate multiple intent discrimination reasons corresponding to the intent category hierarchy label. The intent category hierarchy label includes a first category identifier and a second category identifier. The model parameters of the second major model are fewer than those of the first major model. Here, the first category identifier can be a parent category identifier, and the second category identifier can be a child category identifier. The model parameters of the second major model can be 72 billion.

[0036] (2) Data construction.

[0037] 2.1) Prompt word template: “Please perform intent recognition for the current user session based on the historical interaction sessions: [].

[0038] [] indicates the specific corpus to be added, such as historical interaction sessions or current user sessions.

[0039] 2.2) Category hierarchy: Collect all existing intentions and categorize them (parent categories). For example, divide all intentions into three main categories: consultation, insurance cancellation, and others.

[0040] Taking consulting and other categories as examples, the main subcategories are as follows: Consultation services: Consultations on one's own insurance policy, consultations on other people's insurance policies, general inquiries, etc. Other: casual conversation, backing up, goodbye, etc.

[0041] 2.3) Tag encoding, for example: 1) Parent category coding: Consultation category is 1, Insurance Refund category is 2, others are 3; 2) Inquire about my policy, code (1, 1); 3) Consult other people's insurance policies, coded as (1, 2); 4) Catcher in the bag, coded as (3, 2); All intent categories are coded according to the rules above.

[0042] 2.4) Reasons for determining intent: The second main model is invoked to clean and supplement the initial intent annotations of the current user session within each of the multiple text pairs. This determines the intent category hierarchical label for the current user session within the text pair and generates multiple intent discrimination reasons corresponding to the intent category hierarchical label. These multiple intent discrimination reasons corresponding to the intent category hierarchical label for the current user session within the text pair are sentences with the same semantics but different descriptions.

[0043] 2.5) Construction of training data samples: Taking consultation-related corpora as an example, the original corpus is: Customer: Hello.

[0044] Customer Service: Hello, this is XX Insurance customer service. What insurance-related questions can I answer for you? Customer: I just bought insurance for my family yesterday. When will it take effect? Fine-tuning the data: Input (as model input data): "Based on the historical interaction conversation: [Customer: Hello. Customer Service: Hello, this is XX Insurance customer service. Can I help you with any insurance-related questions?], perform intent recognition on the current user conversation: [Customer: I just bought insurance for my family yesterday, when will it take effect?]".

[0045] Label (as model output data): {Parent category of intent: 1; Subcategory of intent: 2; Reason for intent determination: Customer inquires about purchased products}; The candidate set of reasons for determining intent (1,2): [ 1) Customer inquiries about purchased products; 2) What information does the customer want to inquire about regarding a newly purchased insurance product? ...... ] Based on the example above, all corpora are processed to construct the training set D.

[0046] (3) Model training.

[0047] The training set D is used to fine-tune the intent recognition of the third-largest model, which can have 4 billion parameters.

[0048] When fine-tuning the intent recognition of the third model using the training set, one intent discrimination reason is selected from multiple intent discrimination reasons in the training data samples to generate a target training sample, and the intent recognition of the third model is fine-tuned using the target training sample.

[0049] This embodiment proposes a simplified training data construction scheme for fine-tuning a large model and verifies its effectiveness through experiments. For example, when constructing training data, the prompts only describe the most basic information about the data, without any manual logical prompts. Therefore, during fine-tuning, the logic for distinguishing data comes entirely from data-driven approaches, allowing the model to learn semantic information and business logic from the data. In label design, considering the hierarchical relationship between intent categories, a tiered design is adopted for the categories, i.e., a parent category and child category pattern, so that a label contains two layers of category meaning, ensuring increased data distinguishability and reduced error rate during training. Another component of the label is the intent discrimination reason. In this embodiment, for multiple sentences with the same semantics but different expressions for the same intent candidate, during training, the intent discrimination reason corresponding to any example data in a batch (a subset of data used in a single iteration of machine learning training) is randomly selected from its candidate set to increase the richness of the generated language. Considering these aspects, this scheme not only simplifies model input but also facilitates the model's mining of key information, improving model performance.

[0050] It should be noted that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In practical applications, all the above possible implementation methods can be arbitrarily combined in a combined manner to form possible embodiments of this application, which will not be described in detail here.

[0051] Based on the above embodiments, which provide a method for constructing a training set for fine-tuning intent recognition of a large model, this application also provides an apparatus for constructing a training set for fine-tuning intent recognition of a large model, based on the same inventive concept.

[0052] Figure 2 This is a structural diagram of a device for constructing a training set for fine-tuning intent recognition of a large model, as provided in an embodiment of this application. Figure 2 As shown, the apparatus for constructing a training set for fine-tuning intent recognition of a large model may specifically include a collection unit 210, a first determination unit 220, a second determination unit 230, a construction unit 240, and a generation unit 250.

[0053] Collection unit 210 is used to collect online accumulated interaction data; wherein, the interaction data is generated based on the interaction between the first model and the user; The first determining unit 220 is used to determine, based on the interaction data, multiple text pairs including historical interaction sessions and current user sessions, and preliminary intent annotations of the current user session in each text pair. The second determining unit 230 is used to call the second major model to clean and supplement the preliminary intent annotation of the current user session in each of the multiple text pairs, determine the intent category level label of the current user session in the text pair, and generate multiple intent discrimination reasons corresponding to the intent category level label; wherein, the intent category level label includes a first category identifier and a second category identifier; wherein, the model parameters of the second major model are smaller than the model parameters of the first major model; The construction unit 240 is used to construct a training data sample corresponding to the text pair based on the text pair, the intent category hierarchical label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchical label; The generation unit 250 is used to generate a training set based on the training data samples corresponding to multiple text pairs; the training set is used to fine-tune the intent recognition of the third model; the model parameters of the third model are smaller than those of the first model.

[0054] This application embodiment provides a possible implementation, wherein the construction unit 240 is further configured to: Obtain a pre-created prompt template; wherein the prompt template includes a prompt sentence indicating intent recognition of the current user session based on historical interaction sessions, the fill position of the historical interaction sessions, and the fill position of the current user session; Based on the prompt word template, fill the historical interaction sessions in the text pair into the fill position of the historical interaction sessions, and fill the current user session in the text pair into the fill position of the current user session to obtain the model input data; The intent category hierarchy label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchy label, are used as the model output data; Based on the model input data and model output data, construct a training data sample corresponding to the text pair.

[0055] This application provides one possible implementation method, such as... Figure 3 As shown above, Figure 2 The device shown may also include a training unit 310 for: When fine-tuning the intent recognition of the third model using the training set, one intent discrimination reason is randomly selected from multiple intent discrimination reasons in the training data samples to generate a target training sample, and the intent recognition of the third model is fine-tuned using the target training sample.

[0056] This application embodiment provides a possible implementation, wherein the second determining unit 230 is further configured to: Multiple preset intent categories are hierarchically divided into parent category layers and sub-category layers; Assign a unique first category identifier and a second category identifier to each intent category; the first category identifier is used to identify the category in the parent category layer, and the second category identifier is used to identify the category in the child category layer.

[0057] This application provides a possible implementation method in which multiple intent discrimination reasons corresponding to the intent category level tags of the current user session in the text pair are sentences with the same semantics but different descriptions.

[0058] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for constructing a training set for fine-tuning large model intent recognition of any of the above embodiments.

[0059] In an exemplary embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 400 includes a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, for example, via a bus 402. Optionally, the electronic device 400 may also include a transceiver 404. It should be noted that in practical applications, the transceiver 404 is not limited to one type, and the structure of this electronic device 400 does not constitute a limitation on the embodiments of this application.

[0060] Processor 401 may be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0061] Bus 402 may include a pathway for transmitting information between the aforementioned components. Bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 402 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0062] The memory 403 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0063] The memory 403 stores computer program code that executes the scheme of this application, and its execution is controlled by the processor 401. The processor 401 executes the computer program code stored in the memory 403 to implement the content shown in the foregoing method embodiments.

[0064] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0065] Based on the same inventive concept, embodiments of this application also provide a storage medium storing a computer program, wherein the computer program is configured to execute, at runtime, the method for constructing a training set for fine-tuning large model intent recognition of any of the above embodiments.

[0066] Based on the same inventive concept, embodiments of this application also provide a computer program product, including a computer program configured to execute, at runtime, the method for constructing a training set for fine-tuning large model intent recognition of any of the above embodiments.

[0067] Those skilled in the art will clearly understand that the specific working process of the systems, devices, and modules described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0068] Those skilled in the art will understand that the technical solution of this application, or all or part of it, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several program instructions to cause an electronic device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of this application when running the program instructions. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0069] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as electronic devices like personal computers, servers, or network devices) associated with program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of this application.

[0070] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of this application, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to leave the protection scope of this application.

Claims

1. A method for constructing a training set for fine-tuning intent recognition in large models, characterized in that, The method includes: Collect online interaction data; the interaction data is generated based on the first major model and user interaction. Based on the interaction data, multiple text pairs including historical interaction sessions and current user sessions are identified, along with preliminary intent annotations for the current user session within each text pair. The second major model is invoked to clean and supplement the preliminary intent annotation of the current user session in each of the multiple text pairs, determine the intent category hierarchical label of the current user session in the text pair, and generate multiple intent discrimination reasons corresponding to the intent category hierarchical label; wherein, the intent category hierarchical label includes a first category identifier and a second category identifier; wherein, the model parameters of the second major model are smaller than those of the first major model. Based on the text pair, the intent category hierarchical label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchical label, construct a training data sample corresponding to the text pair; A training set is generated based on the training data samples corresponding to multiple text pairs; the training set is used to fine-tune the intent recognition of the third major model; the model parameters of the third major model are smaller than those of the first major model.

2. The method according to claim 1, characterized in that, Based on the text pair, the intent category hierarchy labels of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchy labels, a training data sample corresponding to the text pair is constructed, including: Obtain a pre-created prompt template; wherein the prompt template includes a prompt sentence indicating intent recognition of the current user session based on historical interaction sessions, the fill position of the historical interaction sessions, and the fill position of the current user session; Based on the prompt word template, fill the historical interaction sessions in the text pair into the fill position of the historical interaction sessions, and fill the current user session in the text pair into the fill position of the current user session to obtain the model input data; The intent category hierarchy label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchy label, are used as the model output data; Based on the model input data and model output data, construct a training data sample corresponding to the text pair.

3. The method according to claim 1, characterized in that, The method further includes: When fine-tuning the intent recognition of the third model using the training set, one intent discrimination reason is randomly selected from multiple intent discrimination reasons in the training data samples to generate a target training sample, and the intent recognition of the third model is fine-tuned using the target training sample.

4. The method according to claim 1, characterized in that, The method further includes: Multiple preset intent categories are hierarchically divided into parent category layers and sub-category layers; Assign a unique first category identifier and a second category identifier to each intent category; the first category identifier is used to identify the category in the parent category layer, and the second category identifier is used to identify the category in the child category layer.

5. The method according to claim 1, characterized in that, The multiple intents in this text pair corresponding to the intent category hierarchical labels of the current user session are identified by sentences with the same semantics but different descriptions.

6. An apparatus for constructing a training set for fine-tuning intent recognition in large models, characterized in that, The device includes: The collection unit is used to collect online interaction data; the interaction data is generated based on the first model and user interaction. The first determining unit is used to determine, based on the interaction data, multiple text pairs including historical interaction sessions and current user sessions, and preliminary intent annotations for the current user session in each text pair. The second determining unit is used to call the second major model. For each text pair in the multiple text pairs, it cleans and supplements the preliminary intent annotation of the current user session in the text pair, determines the intent category level label of the current user session in the text pair, and generates multiple intent discrimination reasons corresponding to the intent category level label. The intent category level label includes a first category identifier and a second category identifier. The model parameters of the second major model are smaller than those of the first major model. The construction unit is used to construct a training data sample corresponding to the text pair based on the text pair, the intent category hierarchical label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchical label; The generation unit is used to generate a training set based on the corresponding training data samples of multiple text pairs; the training set is used to fine-tune the intent recognition of the third major model; the model parameters of the third major model are smaller than those of the first major model.

7. The apparatus according to claim 6, characterized in that, Building units are also used for: Obtain a pre-created prompt template; wherein the prompt template includes a prompt sentence indicating intent recognition of the current user session based on historical interaction sessions, the fill position of the historical interaction sessions, and the fill position of the current user session; Based on the prompt word template, fill the historical interaction sessions in the text pair into the fill position of the historical interaction sessions, and fill the current user session in the text pair into the fill position of the current user session to obtain the model input data; The intent category hierarchy label of the current user session in the text pair, and the multiple intent discrimination reasons corresponding to the intent category hierarchy label, are used as the model output data; Based on the model input data and model output data, construct a training data sample corresponding to the text pair.

8. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method for constructing a training set for fine-tuning large model intent recognition as described in any one of claims 1 to 5.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute, at runtime, the method for constructing a training set for fine-tuning large model intent recognition as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is configured to execute, at runtime, the method for constructing a training set for fine-tuning intent recognition of a large model as described in any one of claims 1 to 5.

Images