Label classification method, apparatus and electronic device

Abstract

The present disclosure discloses a label classification method, device and electronic equipment, the method comprising: retrieving a similar sample set of a target predicted utterance from a sample set corresponding to each label element of a preset label space and determining the label element to which each sample in the similar sample set belongs; based on each sample in the similar sample set and the label element to which each sample belongs, generating a prompt and inputting it into a target large language model to obtain a classification label corresponding to the target predicted utterance, the prompt carrying the target predicted utterance, a label prediction range composed of the label elements to which each sample in the similar sample set belongs, and reference examples composed of each sample in the similar sample set and the label element to which each sample belongs.

Description

Technical Field

[0001] This document pertains to the field of artificial intelligence technology, and in particular relates to a label classification method, apparatus, and electronic device. Background Technology

[0002] To improve the sales skills of insurance planners, it is necessary to evaluate the quality of their sales scripts. A common approach is to define numerous key sales points throughout the sales process and assess whether the planner's sales scripts include these points to determine their professional competence.

[0003] However, as sales script evaluation becomes increasingly refined, the number of defined key points also increases. How to quickly identify a large number of defined key issues from sales planners' sales scripts and improve identification efficiency has become an urgent problem to be solved. Summary of the Invention

[0004] This disclosure provides a label classification method, apparatus, and electronic device that can improve the efficiency of label classification and recognition of discourse in specific scenarios.

[0005] To solve the above-mentioned technical problems, the embodiments of this disclosure are implemented as follows:

[0006] Firstly, a label classification method is proposed, including:

[0007] Retrieve a set of similar samples of the target predicted discourse from the sample set corresponding to each label element in the preset label space;

[0008] The merged label set, which consists of the label elements of each sample in the similar sample set, is determined and used as the label prediction range of the target predicted discourse in the target large language model.

[0009] Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0010] Secondly, a label classification device is proposed, comprising:

[0011] The retrieval module retrieves a set of similar samples of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space.

[0012] The label determination module determines the merged label set composed of the label elements of each sample in the similar sample set and uses it as the label prediction range of the target predicted discourse in the target large language model.

[0013] The prompt generation module generates a prompt based on each sample in the similar sample set and the label elements to which each sample belongs. The prompt carries a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0014] The large language model processing module feeds the input prompt into the target large language model to obtain the classification label corresponding to the target predicted utterance.

[0015] Thirdly, an electronic device is proposed, comprising: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the following operations:

[0016] Retrieve a set of similar samples of the target predicted discourse from the sample set corresponding to each label element in the preset label space;

[0017] The merged label set, which consists of the label elements of each sample in the similar sample set, is determined and used as the label prediction range of the target predicted discourse in the target large language model.

[0018] Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0019] Fourthly, a computer-readable storage medium is provided that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the following operations:

[0020] Retrieve a set of similar samples of the target predicted discourse from the sample set corresponding to each label element in the preset label space;

[0021] The merged label set, which consists of the label elements of each sample in the similar sample set, is determined and used as the label prediction range of the target predicted discourse in the target large language model.

[0022] Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0023] In this embodiment of the disclosure, by pre-setting a label space and setting discourse samples for each label element, and then retrieving a set of similar samples of the target predicted discourse based on the discourse samples corresponding to the label elements in the label space, and determining the label elements corresponding to the samples in the set of similar samples, the predicted label range of the target large language model is determined. Then, based on the samples in the set of similar samples and the corresponding label elements, a prompt of the target large language model is generated, and the classification label corresponding to the target predicted discourse is predicted according to the prompt. This can greatly reduce the range of predicted labels in extreme multi-label scenarios, thereby greatly improving the label classification and recognition efficiency of discourse. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0025] Figure 1 This is a flowchart of a label classification method according to an embodiment of the present disclosure.

[0026] Figure 2 A second flowchart illustrating the label classification method of this disclosure.

[0027] Figure 3 This is a schematic diagram of the structure of the model infringement detection device according to an embodiment of the present disclosure.

[0028] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0030] As mentioned earlier, with increasingly refined sales script evaluations, the number of defined key points also increases, making it increasingly difficult to identify a large number of defined key points from sales scripts. Of course, it should be understood that similar identification challenges exist in other business scenarios besides sales, such as medical consultations, after-sales service, and complaint hotlines. Furthermore, in today's society, the ability to solve a problem is not the primary focus; efficiency is what matters. If solution A can solve a problem in one hour, while solution B can solve it in ten minutes, then solution B is the higher-value solution compared to solution A. In this efficiency-driven era, improving the efficiency of labeling and identifying sales scripts has become a pressing issue.

[0031] To address the aforementioned issues, this disclosure aims to propose a label classification method, apparatus, and electronic device to improve the efficiency of label classification and recognition of discourse.

[0032] Figure 1 This is a flowchart of a label classification method according to an embodiment of this disclosure. Figure 1 The method shown may include the following steps:

[0033] S102, retrieve a set of similar samples of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space and determine the tag element to which each sample in the similar sample set belongs.

[0034] In this embodiment, a tag space can be pre-defined, which includes all category tags for the target business scenario. Let's assume this tag space is L, containing tag elements such as l1, l2, ..., ln. It should be understood that this embodiment supports various orders of magnitude of category tags, such as hundreds, thousands, tens of thousands, or even hundreds of thousands, or more; this embodiment does not impose any limitations on this. Of course, for scenarios with tens of thousands or more extreme tags, the solution in this embodiment offers a more significant efficiency improvement compared to conventional solutions.

[0035] Furthermore, after setting up the label space, a certain number of discourse samples can be collected for each label element in the label space, such as dozens, hundreds, or more. The collected samples can be represented using various vector representation methods, such as One-Hot Encoding, Bag of Words (BoW), TF-IDF (Term Frequency-Inverse Document Frequency), Word Embeddings, N-gram models, hierarchical models (such as Doc2Vec), etc. Taking Word Embeddings as an example, M3E (Multi-task Multi-view Multi-granularity Embedding) can be used for vector representation.

[0036] Of course, it should be understood that a discourse may belong to multiple label elements at the same time. For example, discourse L can be classified into label category A or label category B. In this case, the discourse samples corresponding to label category A contain discourse L, and the discourse samples corresponding to label category B also contain discourse L.

[0037] In this embodiment of the disclosure, for a target predicted discourse, similar samples corresponding to the target predicted discourse can be retrieved from the discourse sample set corresponding to the tag elements in the tag space, and their union can be taken to obtain a set of similar samples.

[0038] Figure 2 This is a schematic diagram illustrating a specific application scenario of an embodiment of this disclosure. The following is in conjunction with… Figure 2 The sales script shown in the sales scenario illustrates the method of this embodiment.

[0039] Assuming the query tag space L in a sales scenario is a point pool, it includes the following point tag elements:

[0040] 1. The housekeeper asked who it was for;

[0041] 2. The housekeeper inquires about the buyer's age;

[0042] 3. The housekeeper inquires about the user's health status;

[0043] 4. The administrator reminds users that they need to provide a health declaration;

[0044] 5. The administrator inquires about the user's previous insurance coverage;

[0045] 6. The butler inquires about the user's budget;

[0046] 7. The butler explains the product's expiration date;

[0047] 8. Critical illness insurance with savings or restitution type.

[0048] Of course, it should be understood that in practical applications, the number of tags in the tag space far exceeds the number of key tags contained in the Point Pool.

[0049] Meanwhile, assume that each tag element in the Point Pool has been configured with the corresponding discourse sample q, and assume that the requirement is described as: select which sales points are included in the current discourse (query) from the point pool.

[0050] Suppose the query that needs to predict the key tag is: Are you planning to prepare some money for your child?

[0051] For this query, a similar set of samples can be found from the utterance samples q corresponding to each tag element in the Point Pool.

[0052] Of course, it should be understood that similar discourse samples can be retrieved in one or more ways in this embodiment of the disclosure.

[0053] For samples retrieved using only one method, the set of retrieval results constitutes the set of similar samples.

[0054] For samples retrieved using multiple methods, a set of similar samples can be determined based on the sets retrieved using the various methods.

[0055] Optionally, in step S102, retrieving a set of similar samples of the target predicted discourse from the sample set corresponding to each label element in the preset label space can be implemented as follows:

[0056] Using the first similarity retrieval method, the first similar sample set of the target predicted discourse is retrieved from the sample set corresponding to each tag element in the preset tag space;

[0057] The second similarity retrieval method is used to retrieve the second similarity sample set of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space;

[0058] The similar sample set is determined based on the first similar sample set and the second similar sample set.

[0059] It should be understood that the first set of similar samples is a first preset number of samples ranked first in similarity; the second set of similar samples is a second preset number of samples ranked first in similarity. The first and second preset numbers can be set according to actual circumstances, and can be equal or unequal. For example, the first and second preset numbers can both be set to 5, 10, 15, etc.; or the first preset number can be set to 8, and the second preset number to 10, etc. This embodiment of the present disclosure does not limit this.

[0060] Furthermore, the union of the first and second similar sample sets can be determined as the set of similar samples. By taking the union of the two as the set of similar samples, we can avoid missing similar samples.

[0061] Alternatively, the intersection of the first and second similar sample sets can be determined as the similar sample set. By taking the intersection of the two as the similar sample set, the similarity between the samples in the similar sample set and the target predicted discourse can be guaranteed to a large extent.

[0062] Taking the query "Are you planning to prepare some money for your child?" as an example, assuming that the first similar sample set obtained by the first similarity retrieval method includes:

[0063] 1. Are you planning to invest a sum of money in your child's education?

[0064] 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition period until your child reaches adulthood?

[0065] 3. Are you looking to prepare insurance for your parents? How old are they?

[0066] Suppose that the set of second similar samples obtained by the second similarity retrieval method includes:

[0067] 1. Are you planning to invest a sum of money in your child's education?

[0068] 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition period until your child reaches adulthood?

[0069] 3. Are you planning to view this for yourself, or is it because family members also have plans to configure it?

[0070] If we take the union of the first set of similar samples and the second set of similar samples, then the set of similar samples is:

[0071] 1. Are you planning to invest a sum of money in your child's education?

[0072] 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition period until your child reaches adulthood?

[0073] 3. Are you looking to prepare insurance for your parents? How old are they?

[0074] 4. Are you planning to view this for yourself, or is it because family members also have plans to configure it?

[0075] If we take the intersection of the first set of similar samples and the second set of similar samples, then the set of similar samples is:

[0076] 1. Are you planning to invest a sum of money in your child's education?

[0077] 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition period until your child reaches adulthood?

[0078] Of course, it should be understood that the above is just an example of retrieving similar samples using two retrieval methods and then determining the set of similar samples based on the retrieved similar samples. In this embodiment of the disclosure, similar samples can also be retrieved using three or more retrieval methods and then the set of similar samples can be determined based on the retrieved similar samples.

[0079] Optionally, retrieving a set of similar samples of the target predicted discourse from the sample set corresponding to each label element in the preset label space may further include:

[0080] Before determining the similar sample set based on the first and second similar sample sets, the third similar sample set of the target predicted discourse is retrieved from the sample set corresponding to each tag element in the preset tag space using a third similarity retrieval method.

[0081] The process of determining the similar sample set based on the first similar sample set and the second similar sample set includes: determining the similar sample set based on the first similar sample set, the second similar sample set, and the third similar sample set.

[0082] Similar to the previous one, in this case, the intersection or union of the first set of similar samples, the second set of similar samples, and the third set of similar samples can be used to obtain the set of similar samples.

[0083] Furthermore, it should be understood that the retrieval methods mentioned above in this disclosure embodiment may include various specific implementation methods, such as vector retrieval methods, literal similarity retrieval methods, and so on.

[0084] Vector retrieval refers to a retrieval method that relies on vector databases for vector retrieval, such as Milvus, Pinecone, Chroma, etc.

[0085] Literal similarity retrieval refers to retrieval methods based on literal similarity, such as Elasticsearch algorithm, Simhash algorithm, Sequence Similarity algorithm, Jaccard similarity algorithm, Cosine Similarity algorithm, Levenshtein distance algorithm, N-gram similarity algorithm, etc.

[0086] S104. Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries the target predicted utterance, the label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0087] After obtaining the set of similar samples and the corresponding label elements, the label prediction range and reference examples can be determined based on this information.

[0088] Specifically, determining the label prediction range may include:

[0089] The union of all label elements corresponding to each target sample in the similar sample set is taken to obtain the merged label set;

[0090] The label prediction range of the target predicted discourse is determined based on the merged label set.

[0091] Take, for example, each discourse sample and its corresponding key tag in the following similar sample set:

[0092] 1. Are you planning to invest a sum of money in your child's education?

[0093] Key tags:

[0094] The manager inquires about the user's purpose for purchasing insurance;

[0095] 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition period until the child reaches adulthood?

[0096] Key tag: The administrator inquires about the user's purpose for purchasing insurance;

[0097] 3. Are you looking to prepare insurance for your parents? How old are they?

[0098] Key tags: The concierge asks who the purchase is for; the concierge asks the buyer's age;

[0099] 4. Are you planning to view this for yourself, or is it because family members also have plans to configure it?

[0100] Key tag: The housekeeper asks who it's for;

[0101] At this point, the merged tag set includes:

[0102] 1. The administrator inquires about the user's purpose for purchasing insurance;

[0103] 2. The housekeeper asked who it was for;

[0104] 3. The housekeeper asks the buyer about their age.

[0105] Once the merged label set is determined, it can be set as the range of predicted labels for the target prediction statement (query: Do you want to prepare some money for your child?) in the target large language model.

[0106] In this embodiment, based on the tag elements of the merged tag set, the predicted tag range of the target large language model can be determined, and the predicted tag of the target predicted sentence is selected from this predicted tag range. Furthermore, in this embodiment, several reference examples are generated based on each sample in the similar sample set and the tag elements to which each sample belongs, to assist the target large language model in inference and prediction to obtain the classification tag of the target predicted sentence.

[0107] Or with Figure 2 For example, in a prompt generated by an embodiment of this disclosure, the predicted label range is as follows: Figure 2 The "##List of Defined Points" in the prompt is shown below:

[0108] ## List of Defined Key Points

[0109] A: The homeowner inquired about the user's purpose for purchasing insurance.

[0110] b: The housekeeper asked who it was for.

[0111] c: The housekeeper inquires about the buyer's age

[0112] In addition, see the example below. Figure 2 The prompt's "## Discourse Examples" section shows:

[0113] ##Discourse Examples

[0114] Example 1

[0115] Input: Are you looking to invest in your child's education?

[0116] Output: The administrator inquires about the user's purpose for purchasing insurance.

[0117] [Example 2]

[0118] Input: Do you plan to provide lifelong security for your child, or do you want to prepare a transition period until your child reaches adulthood?

[0119] Output: The administrator inquires about the user's purpose for purchasing insurance.

[0120] [Example 3]

[0121] Input: Are you looking to prepare insurance for your parents? How old are they?

[0122] Output: The butler asks who the purchase is for | The butler asks the purchaser's age

[0123] [Example 4]

[0124] Input: Are you planning to see this for yourself? Or are you considering that family members also have plans to configure it?

[0125] Output: The butler asks who to buy it for.

[0126] In addition, its target prediction statement is as follows: Figure 2 The prompt in the middle is shown as "##Current Phrase":

[0127] ##Current Discourse

[0128] Are you planning to prepare some money for your child?

[0129] Of course, it should be understood that the generated prompt may also include other content, such as... Figure 2 The "requirements" in the prompt, etc.

[0130] In this embodiment of the disclosure, by pre-setting a label space and setting discourse samples for each label element, and then retrieving a set of similar samples of the target predicted discourse based on the discourse samples corresponding to the label elements in the label space, and determining the label elements corresponding to the samples in the set of similar samples, the predicted label range of the target large language model is determined. Then, based on the samples in the set of similar samples and the corresponding label elements, a prompt of the target large language model is generated, and the classification label corresponding to the target predicted discourse is predicted according to the prompt. This can greatly narrow the range of predicted labels, thereby greatly improving the label classification and recognition efficiency of discourse.

[0131] It should be understood that, in specific applications, the methods of the embodiments of this disclosure may have multiple implementations.

[0132] Alternatively, in one specific implementation, the method can be derived from, for example... Figure 2 The example shown is an implementation of the retrieval-generate-frame framework. For instance... Figure 2As shown, a query and a key point pool can be input into the Retrival-Generate-Frame, where the key point pool includes key point tag elements and corresponding utterance sample sets; the Retrival-Generate-Frame can call a retrieval library (e.g., a retriever) through the retriever. Figure 2 The `urterance.lib` file is used to perform a search to obtain the corresponding search hits, which include similar samples and their corresponding labels. Based on the search hits, `Retrival-Generate-Frame` generates a hint for the target large language model, which can then be input into the target large language model to obtain the classification label.

[0133] Figure 3 This is a schematic diagram of the structure of a label sorting device according to an embodiment of this disclosure. Figure 3 As shown, the label sorting device 300 may include:

[0134] The retrieval module 310 retrieves a set of similar samples of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space and determines the tag element to which each sample in the similar sample set belongs.

[0135] The generation module 320 generates a prompt based on each sample in the similar sample set and the label elements to which each sample belongs. The prompt carries the target prediction statement, the label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0136] The large language model processing module 330 inputs the prompt into the target large language model to obtain the classification label corresponding to the target predicted discourse.

[0137] In this embodiment, the label classification device 300 pre-sets a label space and sets discourse samples for each label element. Then, based on the discourse samples corresponding to the label elements in the label space, it retrieves a set of similar samples of the target predicted discourse and determines the label elements corresponding to the samples in the similar sample set. This determines the predicted label range of the target large language model. Then, based on the samples in the similar sample set and the corresponding label elements, it generates a prompt for the target large language model. Based on the prompt, it predicts the classification label corresponding to the target predicted discourse. This greatly narrows the range of predicted labels and thus greatly improves the label classification and recognition efficiency of discourse.

[0138] In this embodiment, the modules in the apparatus can also implement the method steps provided in the method embodiments. Alternatively, the apparatus provided in this embodiment may include other modules besides those described above to implement the method steps provided in the method embodiments. Furthermore, the apparatus provided in this embodiment can achieve the technical effects achievable by the method embodiments.

[0139] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this disclosure. Please refer to... Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0140] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0141] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0142] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming the aforementioned tag classification device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0143] Retrieve a set of similar samples of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space and determine the tag element to which each sample in the similar sample set belongs;

[0144] Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries the target predicted utterance, the label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0145] It should be understood that the label sorting device in this embodiment can be used as... Figure 1 The execution body of the method shown is therefore able to achieve... Figure 1 The steps and functions of the method shown will not be elaborated here.

[0146] The above is as disclosed in this disclosure. Figure 1 The methods disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in one or more embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in one or more embodiments of this disclosure can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0147] Of course, in addition to software implementation, the electronic device disclosed herein does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0148] This disclosure also provides a computer-readable storage medium for storing one or more programs. The one or more programs include instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0149] Retrieve a set of similar samples of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space and determine the tag element to which each sample in the similar sample set belongs;

[0150] Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries the target predicted utterance, the label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0151] Furthermore, embodiments of this disclosure also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to execute:

[0152] Retrieve a set of similar samples of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space and determine the tag element to which each sample in the similar sample set belongs;

[0153] Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries the target predicted utterance, the label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0154] In summary, the above description is merely a preferred embodiment of this disclosure and is not intended to limit the scope of protection of this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this disclosure should be included within the scope of protection of one or more embodiments of this disclosure.

[0155] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0156] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0158] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

Claims

1. A label classification method, comprising: Using the first similarity retrieval method, the first similar sample set of the target predicted discourse is retrieved from the sample set corresponding to each tag element in the preset tag space; Using a second similarity retrieval method, a second similarity sample set of the target predicted discourse is retrieved from the sample set corresponding to each tag element in the preset tag space; the similarity sample set is determined based on the first similarity sample set and the second similarity sample set. Determine the label element to which each sample belongs in the set of similar samples; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain the classification label corresponding to the target predicted utterance. The prompt carries the target predicted utterance, the label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

2. The method as described in claim 1, wherein the first set of similar samples is a first preset number of samples ranked first in similarity; and the second set of similar samples is a second preset number of samples ranked first in similarity.

3. The method of claim 1, wherein before determining the similar sample set based on the first similar sample set and the second similar sample set, the method further comprises: The third similarity retrieval method is used to retrieve the third similarity sample set of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space; Determining the similar sample set based on the first similar sample set and the second similar sample set includes: The similar sample set is determined based on the first similar sample set, the second similar sample set, and the third similar sample set.

4. The method as described in claim 2, wherein the first similarity retrieval method includes a vector retrieval method, and the second similarity retrieval method includes a literal similarity retrieval method.

5. The method as described in claim 1, wherein the preset label space includes more than a third preset number of label elements, each label element is pre-labeled with one or more discourse samples of the target application scenario, and the discourse samples are represented by vectors.

6. A label sorting device, comprising: The retrieval module retrieves a first similar sample set of the target predicted discourse from the sample set corresponding to each tag element in the preset tag space using the first similarity retrieval method. Using a second similarity retrieval method, a second similarity sample set of the target predicted discourse is retrieved from the sample set corresponding to each tag element in the preset tag space; the similarity sample set is determined based on the first similarity sample set and the second similarity sample set. Determine the label element to which each sample belongs in the set of similar samples; The generation module generates a prompt based on each sample in the similar sample set and the label elements to which each sample belongs. The prompt carries the target prediction statement, the label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs. The large language model processing module inputs the prompt into the target large language model to obtain the classification label corresponding to the target predicted utterance.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of the method as described in any one of claims 1 to 5.

Images