Intention recognition method based on large language model

Abstract

The invention provides an intention recognition method based on a large language model, and relates to the technical field of artificial intelligence, in particular to the technical fields of large language models, deep learning and the like. The LLM-based training data construction method comprises the steps of obtaining real data of a target field; performing intention recognition on the real data by adopting LLM to obtain an initial intention; verifying the initial intention by adopting the LLM to obtain a target intention; generating a query statement based on the target intention by adopting the LLM; and constructing training data based on the query statement and the target intention. The quality of the training data of the intention recognition task can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of large language models and deep learning, and specifically to an intent recognition method based on a large language model. Background Technology

[0002] Training intent recognition models requires high-quality training data, but in vertical fields (such as finance, healthcare, and government affairs), there is a lack of high-quality training data. Summary of the Invention

[0003] This disclosure provides an intent recognition method based on a large language model, as well as related methods and products.

[0004] According to one aspect of this disclosure, a training data construction method based on LLM is provided, comprising: acquiring real data in a target domain; using LLM to perform intent recognition on the real data to obtain an initial intent; using the LLM to verify the initial intent to obtain a target intent; using the LLM to generate a query statement based on the target intent; and constructing training data based on the query statement and the target intent.

[0005] According to another aspect of this disclosure, an intent recognition model training method based on LLM is provided, comprising: acquiring training data; and training an intent recognition model using the training data; wherein the training data is generated using the method described in any of the preceding claims.

[0006] According to another aspect of this disclosure, an intent recognition method based on LLM is provided, comprising: obtaining a query statement to be recognized; and performing intent recognition on the query statement using an intent recognition model; wherein the intent recognition model is trained using the method described in any of the preceding claims.

[0007] According to another aspect of this disclosure, an LLM-based training data construction apparatus is provided, comprising: an acquisition module for acquiring real data in a target domain; an identification module for performing intent identification on the real data using LLM to obtain an initial intent; a verification module for verifying the initial intent using the LLM to obtain a target intent; a generation module for generating a query statement based on the target intent using the LLM; and a construction module for constructing training data based on the query statement and the target intent.

[0008] According to another aspect of this disclosure, an intent recognition model training apparatus based on LLM is provided, comprising: an acquisition module for acquiring training data; and a training module for training an intent recognition model using the training data; wherein the training data is generated using the method described in any of the preceding claims.

[0009] According to another aspect of this disclosure, an intent recognition device based on LLM is provided, comprising: an acquisition module for acquiring a query statement to be recognized; and an identification module for performing intent recognition on the query statement using an intent recognition model; wherein the intent recognition model is trained using the method described in any of the preceding claims.

[0010] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to said at least one processor; wherein the memory stores instructions executable by said at least one processor, said instructions being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the foregoing aspects.

[0011] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any of the preceding aspects.

[0012] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to any of the preceding aspects.

[0013] According to embodiments of this disclosure, the quality of training data for intent recognition tasks can be improved.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0017] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0018] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0019] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0020] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0021] Figure 6 This is a schematic diagram according to the sixth embodiment of the present disclosure;

[0022] Figure 7 This is a schematic diagram according to the seventh embodiment of the present disclosure.

[0023] Figure 8 This is a schematic diagram of an electronic device used to implement the LLM-based training data construction method, the LLM-based intent recognition model training method, or the LLM-based intent recognition method according to the embodiments of this disclosure. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] In related technologies, training data for intent recognition tasks can be generated based on knowledge graphs or large language models (LLMs).

[0026] The knowledge graph-based approach requires the construction of a knowledge graph, which is costly and lacks flexibility.

[0027] The LLM-based approach typically involves directly instructing the LLM to generate intents, such as inputting commands like "Generate 20 user intent categories in the banking sector" or "Generate 10 user queries for the 'transfer' intent," relying on the general knowledge stored within the LLM to generate data out of thin air. However, this direct generation method based on the LLM is prone to creating "illusions," lacks business authenticity, and has poor controllability.

[0028] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure. This embodiment provides a method for constructing training data based on LLM. Figure 1 As shown, the method includes:

[0029] 101. Obtain real data in the target field.

[0030] 102. Use LLM to perform intent recognition on the real data to obtain the initial intent.

[0031] 103. The initial intent is verified using the LLM to obtain the target intent.

[0032] 104. Use the LLM to generate a query statement based on the target intent.

[0033] 105. Based on the query statement and the target intent, construct training data.

[0034] The target domain refers to the vertical domain in which the intent recognition task is applied, such as finance, healthcare, or government affairs.

[0035] Real data refers to relevant data that actually exists in the target field, such as official document data, user comment data, or forum data.

[0036] The aforementioned real data can be pre-configured or obtained through web searches.

[0037] After obtaining the real data, the LLM can be instructed to identify the intent based on the prompt information to obtain the initial intent.

[0038] After obtaining the initial intent, the LLM can be instructed to perform self-verification, and the verified initial intent can be used as the target intent.

[0039] Once the target intent is obtained, the LLM can be instructed to generate a query statement based on the target intent.

[0040] Then, the query statement and its corresponding target intent can be used as sample pairs to obtain training data.

[0041] In this embodiment, the initial intent is obtained based on real data in the target domain, ensuring that the intent category originates from real business, avoiding the illusion problem caused by the LLM's arbitrary generation, and improving the authenticity and reliability of the business. By verifying the initial intent to obtain the target intent, the accuracy of the intent can be improved. Therefore, training data is generated based on the target intent and query statement, which can improve the authenticity, reliability and accuracy of the training data, thereby obtaining high-quality training data.

[0042] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure. This embodiment provides a method for constructing training data based on LLM. Figure 2 As shown, the method includes:

[0043] 201. Send a first prompt message to the LLM, the first prompt message containing the target domain and company constraint information, so that the LLM calls the network search interface to obtain a list of companies in the target domain.

[0044] For example, the first prompt message is:

[0045] "Please search and list companies in the {domain} domain. ... Search requirements: 1. Only return companies that actually exist and have real business operations in the {domain} domain; 2. Prioritize searching for well-known companies with publicly available information..."

[0046] The "domain" field mentioned above refers to the target domain, which is the domain where the task of identifying the target domain is to be applied. This field can be specified by the user. The search requirements mentioned above are company-specific constraints and can be set according to actual needs.

[0047] Based on the initial information provided above, the authenticity and reliability of the company list can be guaranteed.

[0048] 202. For each company in the company list, send a second prompt message to the LLM, the second prompt message containing company information and product constraint information, so that the LLM can call the network search interface to obtain the application list of each company.

[0049] For example, the second prompt message is:

[0050] "Please search and list all major mobile and desktop clients developed by {company.name}. ... Search requirements: 1. Only return applications that are indeed developed or officially released by {company.name}; 2. Exclude third-party developed applications that use {company.name} services; 3. Prioritize information from official app stores and the company's official website..."

[0051] The "company.name" field should be filled with the company name, which can be any company from the list above. The search criteria are product constraint information and can be set according to actual needs.

[0052] Based on the second piece of information mentioned above, the authenticity and reliability of the application list can be guaranteed.

[0053] 203. For each application in the application list, send a third prompt message to the LLM. The third prompt message contains application information and document constraint information, so that the LLM can call the network search interface to obtain the document data of each application as the real data of the target domain.

[0054] For example, the third prompt message is:

[0055] Please search for the official product documentation for the app named {app.name} from company {app.company}. Search requirements: 1. Only return documentation from official sources (company website, developer blog, official help center). 2. Prioritize searching for complete user manuals, product descriptions, API documentation, etc.

[0056] The "app.name" field above should be the application name, which can be any application name from the above application list. The "app.company" field should be the name of the company to which the application belongs. The search requirements mentioned above are document constraints and can be set according to actual needs.

[0057] Based on the aforementioned third piece of information, the authenticity and reliability of the application list can be guaranteed.

[0058] In this embodiment, real data is obtained through web search, eliminating the need for manual configuration and enabling efficient and accurate acquisition of real data.

[0059] 204. Send a fourth prompt message to the LLM, the fourth prompt message containing the real data and intent constraint information, so that the LLM generates an initial intent.

[0060] For example, the fourth prompt message is:

[0061] "Based on the following app's product documentation, generate all possible user intent categories. These intent categories are categorized into three levels, connected by '-', such as 'Level 1 Intent - Level 2 Intent - Level 3 Intent'. ... App Information: App Name: {documentation.app_name}; Development Company: {documentation.company}; Document Content: {documentation}... Analysis Requirements: 1. Analyze only based on the document content; do not rely on guesswork; 2. Each intent category must be related to the functionality mentioned in the document..."

[0062] In the "documentation" section of the above document content, fill in the actual data. For the application name and development company, fill in the application and company to which the document content belongs. The analysis requirements are intent constraint information, which can be set according to actual needs. In addition, intent constraint information can also include the granularity of intent categories, such as three-level classification, so as to achieve more fine-grained classification.

[0063] In this embodiment, based on the fourth prompt information, the initial intent originating from the real business can be automatically obtained through LLM.

[0064] 205. Send a fifth prompt message to the LLM, the fifth prompt message containing the initial intent and verification constraint information, so that the LLM obtains a verification result; and, in response to the verification result being passed, use the initial intent as the target intent.

[0065] For example, the fifth prompt message is:

[0066] Role and Task: You are a rigorous quality assurance expert. Your task is to review an "intent category" initially extracted from product documentation and determine whether it is explicitly supported in the application functionality described in the document. Review Information: Intent Category to be Reviewed: {intent.category}; Review Requirements: Explicitly mention...; Provide your judgment process: First, give your review conclusion: Yes or No. Then, provide the review basis: If yes,..., if no,...

[0067] The "intent.category" field contains the initial intent obtained from the LLM. The review information may also include information such as the application corresponding to the initial intent. The review requirements field contains specific requirements, such as the explicit mentions mentioned above. The roles, tasks, and review requirements mentioned above are verification constraint information, which can be set according to actual needs. Self-verification is performed based on this verification constraint information.

[0068] After verification, the LLM provides the verification results, such as the audit conclusion mentioned above. Then, the initial intent with an audit conclusion of "yes" can be used as the target intent.

[0069] In this embodiment, based on the fifth prompt information, the LLM can be instructed to perform self-verification, thereby improving the accuracy, authenticity, and reliability of the target intent.

[0070] 206. Send a sixth prompt message to the LLM, the sixth prompt message containing the target intent and query constraint information, so that the LLM generates a query statement.

[0071] For example, the sixth prompt message is:

[0072] "Generate search queries that real users might use based on the following intent categories: intent category {intent.category}; ... Generation requirements: 1. Queries must reflect the search habits of real users; 2. Diversity: Include queries of different lengths and expressions..."

[0073] The aforementioned intent category refers to the verified target intent, which may also include the application name and intent description information corresponding to the target intent; the generation requirements are query constraint information, which can be set according to actual needs.

[0074] In this embodiment, based on the sixth instruction information, the LLM can be instructed to automatically generate query statements, thereby efficiently obtaining diverse query statements.

[0075] In addition, the LLMs described in the above steps can use large language models with the same or different structures.

[0076] 207. Based on the query statement and the target intent, construct training data.

[0077] For example, each target intent and its corresponding query statement can be grouped into a sample pair, and multiple sample pairs can be used to form training data.

[0078] The following example, using a financial scenario, illustrates the above points:

[0079] The target sector can be specified as "banking," and a list of companies can be obtained through web search:

[0080] [Bank X, Bank Y, ...]. X and Y are different bank names.

[0081] For each bank, such as Bank X, search for the official app of "Bank X": "Bank X App", and download its user manual.

[0082] LLM extracts intent categories from the manual, such as ["Credit Card Bill Inquiry", "Current Account to Fixed Deposit", "Purchase Financial Products", "Apply for Loan"...], as the initial intent.

[0083] After verification, LLM's "hypothetical" intentions not explicitly mentioned in the manual may be filtered out, such as "international gold price inquiry". Then, the target intention is determined based on self-verification.

[0084] For a specific target intent, such as "buying wealth management products," LLM generates queries such as: "What are some recommended wealth management products?", "How do I buy some low-risk wealth management products?", and "Can I buy funds on the China Merchants Bank app?"

[0085] Assemble into sample pairs (“What financial management options are recommended?”, “Purchase financial products”).

[0086] Using thousands of such synthesized sample pairs, an intent recognition model is trained. This model can then be deployed in a target domain, such as in Bank X's intelligent customer service system, to accurately understand the true intent behind user input.

[0087] Figure 3 This is a schematic diagram based on the third embodiment of the present disclosure. This embodiment provides a method for training an intent recognition model based on LLM. Figure 3 As shown, the method includes:

[0088] 301. Obtain training data.

[0089] 302. Using the training data, train the intent recognition model.

[0090] The training data was generated using any of the above embodiments.

[0091] For example, through the above embodiments, a query statement and its corresponding target intent can be obtained, which can be used as training data. Based on the above training data, an intent recognition model can be trained.

[0092] In this embodiment, a better intent recognition model can be obtained based on the training data described above, thereby improving the intent recognition performance of the model.

[0093] Figure 4 This is a schematic diagram based on the fourth embodiment of the present disclosure. This embodiment provides an intent recognition method based on LLM. Figure 4 As shown, the method includes:

[0094] 401. Obtain the query statement to be identified.

[0095] 402. Use an intent recognition model to perform intent recognition on the query statement.

[0096] The intent recognition model is generated using any of the above embodiments.

[0097] For example, the above intent recognition model can be deployed in an intelligent customer service system in the target domain. After the system receives the query statement to be recognized by the user, it can use the intent recognition model to obtain the intent category of the query statement to be recognized.

[0098] In this embodiment, intention recognition based on the above-described intention recognition model can improve the accuracy of the obtained intention categories.

[0099] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure, which provides an LLM-based training data construction apparatus.

[0100] like Figure 5 As shown, the device 500 includes: an acquisition module 501, an identification module 502, a verification module 503, a generation module 504, and a construction module 505.

[0101] The acquisition module 501 is used to acquire real data in the target domain; the recognition module 502 is used to perform intent recognition on the real data using LLM to obtain an initial intent; the verification module 503 is used to verify the initial intent using the LLM to obtain a target intent; the generation module 504 is used to generate a query statement based on the target intent using the LLM; and the construction module 505 is used to construct training data based on the query statement and the target intent.

[0102] In this embodiment, the initial intent is obtained based on real data in the target domain, ensuring that the intent category originates from real business, avoiding the illusion problem caused by the LLM's arbitrary generation, and improving the authenticity and reliability of the business. By verifying the initial intent to obtain the target intent, the accuracy of the intent can be improved. Therefore, training data is generated based on the target intent and query statement, which can improve the authenticity, reliability and accuracy of the training data, thereby obtaining high-quality training data.

[0103] In some embodiments, the acquisition module 501 is further configured to:

[0104] Send a first prompt message to the LLM, the first prompt message containing the target domain and company constraint information, so that the LLM calls the network search interface to obtain a list of companies in the target domain;

[0105] For each company in the company list, a second prompt message is sent to the LLM, which contains company information and product constraint information, so that the LLM can call the network search interface to obtain the application list of each company;

[0106] For each application in the application list, a third prompt message is sent to the LLM. The third prompt message contains application information and document constraint information, so that the LLM calls the network search interface to obtain the document data of each application as the actual data.

[0107] In this embodiment, real data is obtained through web search, eliminating the need for manual configuration and enabling efficient and accurate acquisition of real data.

[0108] In some embodiments, the identification module 502 is further configured to:

[0109] A fourth prompt message is sent to the LLM, the fourth prompt message containing the real data and intent constraint information, so that the LLM generates the initial intent.

[0110] In this embodiment, based on the fourth prompt information, the initial intent originating from the real business can be automatically obtained through LLM.

[0111] In some embodiments, the verification module 503 is further configured to:

[0112] A fifth prompt message is sent to the LLM, the fifth prompt message containing the initial intent and verification constraint information, so that the LLM can obtain the verification result;

[0113] In response to the verification result being passed, the initial intent is taken as the target intent.

[0114] In this embodiment, based on the fifth prompt information, the LLM can be instructed to perform self-verification, thereby improving the accuracy, authenticity, and reliability of the target intent.

[0115] In some embodiments, the generation module 504 is further configured to:

[0116] A sixth prompt message is sent to the LLM, the sixth prompt message containing the target intent and query constraint information, so that the LLM generates the query statement.

[0117] In this embodiment, based on the sixth instruction information, the LLM can be instructed to automatically generate query statements, thereby efficiently obtaining diverse query statements.

[0118] Figure 6 This is a schematic diagram according to the sixth embodiment of the present disclosure, which provides an intent recognition model training device based on LLM.

[0119] like Figure 6 As shown, the device 600 includes an acquisition module 601 and a training module 602.

[0120] The acquisition module 601 is used to acquire training data; the training module 602 is used to train the intent recognition model using the training data; wherein the training data is generated using any of the above embodiments.

[0121] In this embodiment, a better intent recognition model can be obtained based on the training data described above, thereby improving the intent recognition performance of the model.

[0122] Figure 7 This is a schematic diagram according to the seventh embodiment of the present disclosure, which provides an intent recognition device based on LLM.

[0123] like Figure 7 As shown, the device 700 includes an acquisition module 701 and an identification module 702.

[0124] The acquisition module 701 is used to acquire the query statement to be identified; the identification module 702 is used to perform intent identification on the query statement using an intent identification model; wherein the intent identification model is generated using any of the above embodiments.

[0125] In this embodiment, intention recognition based on the above-described intention recognition model can improve the accuracy of the obtained intention categories.

[0126] It is understood that the same or similar content in different embodiments of this disclosure can be referred to each other.

[0127] It is understood that the terms "first" and "second" in the embodiments of this disclosure are only used for distinction and do not indicate the degree of importance or the order of events.

[0128] It is understandable that, unless otherwise specified, the order of steps in the process indicates that the temporal relationship between these steps is not limited.

[0129] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0130] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0131] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0132] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0133] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0134] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as an LLM-based training data construction method, an intent recognition model training method, or an intent recognition method. For example, in some embodiments, the LLM-based training data construction method, the LLM-based intent recognition model training method, or the LLM-based intent recognition method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the LLM-based training data construction method, the LLM-based intent recognition model training method, or the LLM-based intent recognition method described above can be performed. Alternatively, in other embodiments, computing unit 801 can be configured to perform the LLM-based training data construction method, the LLM-based intent recognition model training method, or the LLM-based intent recognition method by any other suitable means (e.g., by means of firmware).

[0135] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable task processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0139] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0140] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0141] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0142] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for constructing training data based on Large Language Model (LLM), comprising: Obtain real-world data in the target domain; LLM is used to perform intent recognition on the real data to obtain the initial intent; The initial intent is verified using the LLM to obtain the target intent; The LLM is used to generate a query statement based on the target intent; Training data is constructed based on the query statement and the target intent.

2. The method according to claim 1, wherein, The acquisition of real data in the target domain includes: Send a first prompt message to the LLM, the first prompt message containing the target domain and company constraint information, so that the LLM calls the network search interface to obtain a list of companies in the target domain; For each company in the company list, a second prompt message is sent to the LLM, which contains company information and product constraint information, so that the LLM can call the network search interface to obtain the application list of each company; For each application in the application list, a third prompt message is sent to the LLM. The third prompt message contains application information and document constraint information, so that the LLM calls the network search interface to obtain the document data of each application as the actual data.

3. The method according to claim 1, wherein, The step of using LLM to perform intent recognition on the real data to obtain the initial intent includes: A fourth prompt message is sent to the LLM, the fourth prompt message containing the real data and intent constraint information, so that the LLM generates the initial intent.

4. The method according to claim 1, wherein, The step of using the LLM to verify the initial intent to obtain the target intent includes: A fifth prompt message is sent to the LLM, the fifth prompt message containing the initial intent and verification constraint information, so that the LLM can obtain the verification result; In response to the verification result being passed, the initial intent is taken as the target intent.

5. The method according to claim 1, wherein, The step of generating a query statement based on the target intent using the LLM includes: A sixth prompt message is sent to the LLM, the sixth prompt message containing the target intent and query constraint information, so that the LLM generates the query statement.

6. A method for training an intent recognition model based on LLM, comprising: Obtain training data; The training data is used to train the intent recognition model; The training data is generated using the method described in any one of claims 1-5.

7. An intent recognition method based on LLM, comprising: Retrieve the query statement to be identified; An intent recognition model is used to identify the intent of the query statement; The intent recognition model is trained using the method described in claim 6.

8. A training data construction device based on LLM, comprising: The acquisition module is used to acquire real data in the target domain; The identification module is used to perform intent identification on the real data using LLM to obtain the initial intent; The verification module is used to verify the initial intent using the LLM to obtain the target intent; The generation module is used to generate query statements based on the target intent using the LLM; A construction module is used to construct training data based on the query statement and the target intent.

9. A training device for an intent recognition model based on LLM, comprising: The acquisition module is used to acquire training data; The training module is used to train the intent recognition model using the training data; The training data is generated using the method described in any one of claims 1-5.

10. An intent recognition device based on LLM, comprising: The acquisition module is used to acquire the query statement to be identified; The identification module is used to identify the intent of the query statement using an intent identification model. The intent recognition model is trained using the method described in claim 6.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images