Label classification method and device and electronic equipment

Abstract

The invention discloses a label classification method and device and electronic equipment. The method comprises the steps that a similar sample set of a target prediction utterance is retrieved from sample sets corresponding to all label elements in a preset label space, and the label elements to which all samples in the similar sample set belong are determined; generating a prompt based on each sample in the similar sample set and the tag element to which each sample belongs, and inputting the prompt into a target large language model to obtain a classification tag corresponding to the target prediction utterance, the prompt carries the target prediction utterance, a label prediction range formed by label elements to which samples in the similar sample set belong, and reference examples formed by the samples in the similar sample set and the label elements to which the samples belong.

Description

Technical Field

[0001] The present disclosure belongs to the field of artificial intelligence technology, and in particular, relates to a label classification method, device and electronic device. Background Art

[0002] In order to improve the sales level of insurance planners, it is necessary to evaluate the quality of their sales pitch. The usual method is to define a large number of sales points for the entire sales process, and determine the planner's professional level by evaluating whether these points are included in the planner's sales pitch.

[0003] However, as sales pitch evaluation becomes more sophisticated, the number of key points of definition increases. How to quickly identify a large number of key points of definition from sales planners' sales pitches and improve identification efficiency has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present disclosure provide a label classification method, device and electronic device, which can improve the efficiency of label classification and recognition of speech in specific scenarios.

[0005] To solve the above technical problems, the embodiments of the present disclosure are implemented as follows: First, a label classification method is proposed, including: Retrieving a similar sample set of the target predicted utterance from the sample set corresponding to each label element in the preset label space; Determine a merged label set consisting of label elements belonging to each sample in the similar sample set and use it as a label prediction range of the target predicted utterance in the target large language model; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0006] In a second aspect, a label classification device is proposed, comprising: A retrieval module retrieves a similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space; A label determination module, which determines a merged label set consisting of label elements belonging to each sample in the similar sample set and uses it as a label prediction range of the target predicted utterance in the target large language model; A prompt generating module, based on each sample in the similar sample set and the label element to which each sample belongs, generates a prompt, wherein the prompt carries a reference example consisting of each sample in the similar sample set and the label element to which each sample belongs; The large language model processing module prompts the input into the target large language model to obtain a classification label corresponding to the target predicted utterance.

[0007] In a third aspect, an electronic device is provided, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor performs the following operations: Retrieving a similar sample set of the target predicted utterance from the sample set corresponding to each label element in the preset label space; Determine a merged label set consisting of label elements belonging to each sample in the similar sample set and use it as a label prediction range of the target predicted utterance in the target large language model; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0008] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device performs the following operations: Retrieving a similar sample set of the target predicted utterance from the sample set corresponding to each label element in the preset label space; Determine a merged label set consisting of label elements belonging to each sample in the similar sample set and use it as a label prediction range of the target predicted utterance in the target large language model; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0009] In the disclosed embodiment, a label space is pre-set and speech samples of each label element are set, and then a similar sample set of a target predicted speech is retrieved based on the speech samples corresponding to the label elements in the label space and the label elements corresponding to the samples in the similar sample set are determined, thereby determining a predicted label range of a target large language model, and then a prompt of the target large language model is generated based on the samples in the similar sample set and the corresponding label elements, and then a classification label corresponding to the target predicted speech is predicted based on the prompt, which can greatly narrow the range of predicted labels in extreme multi-label scenarios, thereby greatly improving the label classification and recognition efficiency of speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings: Figure 1 The figure is a flow chart of the tag classification method according to an embodiment of the present disclosure.

[0011] Figure 2 A second flowchart of the tag classification method according to an embodiment of the present disclosure.

[0012] Figure 3 Schematic diagram of the structure of the model infringement detection device according to the embodiment of the present disclosure.

[0013] Figure 4 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solutions in the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0015] As mentioned earlier, as the evaluation of sales speech becomes more and more sophisticated, the number of defined key points also increases, and it is increasingly difficult to identify a large number of defined key points from sales speech. Of course, it should be understood that in addition to sales scenarios, similar identification problems also exist in other business scenarios, such as medical consultation scenarios, product after-sales service scenarios, complaint hotline scenarios, and so on. At the same time, in today's society, whether the problem can be solved is not the focus of everyone's attention, but the efficiency of problem handling is what everyone expects. For the same problem, if solution A can be solved in 1 hour and solution B can be solved within 10 minutes, then solution B is a high-value solution relative to solution A. In an efficiency-oriented era, how to improve the efficiency of label classification and recognition of discourse has become an urgent problem to be solved.

[0016] In view of the above problems, the present disclosure aims to propose a label classification method, device and electronic device to improve the efficiency of label classification and recognition of speech.

[0017] Figure 1 is a flowchart of a tag classification method according to an embodiment of the present disclosure. Figure 1 The method shown may include the following steps: S102, retrieving a similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space and determining the label element to which each sample in the similar sample set belongs.

[0018] In the disclosed embodiment, a label space may be preset first, and the label space includes all classification labels of the target business scenario. It may be assumed that the label space is L, which contains label elements such as l1, l2...ln. It should be understood that in the disclosed embodiment, various orders of magnitude of classification labels are supported, such as hundreds, thousands, tens of thousands, or even hundreds of thousands, or more, and the disclosed embodiment does not limit this. Of course, for scenarios with more than tens of thousands of extreme multi-labels, the solution of the disclosed embodiment has a more obvious efficiency improvement compared to the conventional solution.

[0019] In addition, after setting the label space, a certain number of speech samples can be collected for each label element of the label space, for example, dozens, hundreds, or more. For the collected samples, a variety of vector representation methods can be used for representation, such as One-Hot Encoding, Bag of Words (BoW), TF-IDF (Term Frequency-Inverse Document Frequency), Word Embeddings model, N-gram model, hierarchical model (such as Doc2Vec), etc. Taking the Word Embeddings model as an example, M3E (Multi-task Multi-view Multi-granularity Embedding) can be used for vector representation.

[0020] Of course, it should be understood that an utterance may belong to multiple label elements at the same time. For example, utterance L can be classified as label category A or label category B. In this case, the utterance samples corresponding to label category A include utterance L, and the utterance samples corresponding to label category B also include utterance L.

[0021] In the disclosed embodiment, for a target predicted utterance, similar samples corresponding to the target predicted utterance may be retrieved from a utterance sample set corresponding to a label element in a label space, and their union may be taken to obtain a similar sample set.

[0022] Figure 2 This is a schematic diagram of a specific application scenario of the embodiment of the present disclosure. Figure 2 A sales script in the sales scenario shown is used to illustrate the method of the embodiment of the present disclosure.

[0023] Assume that the query label space L in the sales scenario is a point pool, which includes the following key label elements: 1. The housekeeper asks who you want to buy it for; 2. The housekeeper asks the buyer's age; 3. The housekeeper asks the user about his / her health condition; 4. The housekeeper reminds the user that health notification is required; 5. The housekeeper asks the user about his previous insurance situation; 6. The housekeeper asks the user about the budget; 7. The butler explains the product deadline; 8. Savings type critical illness or return type critical illness.

[0024] Of course, it should be understood that in actual applications, the number of labels in the label space far exceeds the number of key point labels contained in the Point Pool.

[0025] At the same time, it is assumed that each label element of the Point Pool has been configured with the corresponding speech sample q, and it is assumed that the demand description is: select which sales points the current speech (query) contains from the point pool.

[0026] Suppose the query that needs to predict the key point label is: Do you want to prepare some money for your child? For this query, a similar sample set can be found from the discourse samples q corresponding to each label element of the Point Pool.

[0027] Of course, it should be understood that in the embodiments of the present disclosure, similar speech samples may be retrieved in one or more ways.

[0028] For searches using only one method, the set of search results is the similar sample set.

[0029] For retrieval using multiple methods, similar sample sets can be determined based on the sets retrieved using the multiple retrieval methods.

[0030] Optionally, in step S102, retrieving a similar sample set of the target predicted utterance from the sample set corresponding to each label element in the preset label space can be implemented as follows: Retrieving a first similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space by a first similarity retrieval method; Retrieving a second similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space by a second similarity retrieval method; The similar sample set is determined based on the first similar sample set and the second similar sample set.

[0031] It should be understood that the first similar sample set is a first preset number of samples ranked at the top in similarity; the second similar sample set is a second preset number of samples ranked at the top in similarity. The first preset number and the second preset number can be set according to actual conditions, and can be equal or unequal. For example, the first preset number and the second preset number can both be set to 5, 10, 15, etc.; or the first preset number can be set to 8, the second preset number can be set to 10, etc., which is not limited in the embodiments of the present disclosure.

[0032] Furthermore, the union of the first similar sample set and the second similar sample set may be determined as the similar sample set. By taking the union of the two as the similar sample set, similar samples may be avoided from being missed.

[0033] Alternatively, further, the intersection of the first similar sample set and the second similar sample set may be determined as the similar sample set. By taking the intersection of the two as the similar sample set, the similarity between the samples in the similar sample set and the target predicted utterance can be guaranteed to a large extent.

[0034] Take the query "Do you want to prepare some money for your child?" as an example. Assume that the first similar sample set retrieved by the first similarity search method includes: 1. Do you want to invest some education funds for your child? 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition for your child until he or she reaches adulthood? 3. Do you want to prepare insurance for your parents? How old are you? Assume that the second similar sample set retrieved by the second similarity retrieval method includes: 1. Do you want to invest some education funds for your child? 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition for your child until he or she reaches adulthood? 3. Are you planning to use it for yourself? Or are you considering that your family members may also have configuration plans? If the first similar sample set and the second similar sample set are taken as a union, the similar sample set is: 1. Do you want to invest some education funds for your child? 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition for your child until he or she reaches adulthood? 3. Do you want to prepare insurance for your parents? How old are you? 4. Are you planning to use it for yourself? Or are you considering that your family members may also have configuration plans? If the intersection of the first similar sample set and the second similar sample set is taken, the similar sample set is: 1. Do you want to invest some money in education for your child? 2. Do you plan to provide lifelong security for your child, or do you want to prepare a transition for your child until he or she reaches adulthood? Of course, it should be understood that the above only shows an example of retrieving similar samples through two retrieval methods and then determining a similar sample set based on the retrieved similar samples. The embodiments of the present disclosure can also retrieve similar samples through three or more retrieval methods and then determine a similar sample set based on the retrieved similar samples.

[0035] Optionally, retrieving a similar sample set of the target predicted utterance from the sample set corresponding to each label element in the preset label space may also include: Before determining the similar sample set based on the first similar sample set and the second similar sample set, searching a third similar sample set of the target predicted utterance from sample sets corresponding to each label element in a preset label space by a third similarity retrieval method; The determining of the similar sample set based on the first similar sample set and the second similar sample set includes: determining the similar sample set based on the first similar sample set, the second similar sample set and the third similar sample set.

[0036] Similar to the above, at this time, the first similar sample set, the second similar sample set and the third similar sample set can be obtained by taking the intersection or union to obtain the similar sample set.

[0037] In addition, it should be understood that in the embodiments of the present disclosure, the above-mentioned retrieval method may include a variety of specific implementation methods, such as a vector retrieval method, a literal similarity retrieval method, and the like.

[0038] Vector retrieval method refers to a retrieval method that relies on vector databases for vector retrieval, such as Milvus, Pinecone, Chroma, etc.

[0039] Literal similarity retrieval refers to a retrieval method based on literal similarity, such as elasticsearch algorithm, Simhash algorithm, sequence similarity algorithm, Jaccard similarity algorithm, cosine similarity algorithm, Levenshtein distance algorithm, N-gram similarity algorithm, etc.

[0040] S104, based on each sample in the similar sample set and the label element to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries the target predicted utterance, a label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label element to which each sample belongs.

[0041] After obtaining the similar sample set and the label elements corresponding to the similar sample set, the label prediction range and reference examples can be determined based on this information.

[0042] Specifically, determining the tag prediction range may include: Taking a union of all label elements corresponding to each target sample in the similar sample set to obtain the merged label set; A tag prediction range of the target prediction utterance is determined based on the merged tag set.

[0043] Take the following speech samples and corresponding key point labels in the similar sample set as an example: 1. Do you want to invest some money in education for your child? Key points tags: The housekeeper asks the user about the purpose of insurance; 2. Do you intend to provide lifelong security for your child, or do you want to prepare a transition for your child until he or she reaches adulthood? Key tags: The housekeeper asks the user the purpose of insurance; 3. Are you planning to provide insurance for your parents? How old are you? Key points tags: The butler asks who the purchase is for; The butler asks the purchaser’s age; 4. Are you planning to use it for yourself? Or are you considering purchasing it for your family members? Key points label: The butler asks who to buy it for; At this point, the merge tag set includes: 1. The housekeeper asks the user about the purpose of insurance; 2. The housekeeper asks who you want to buy it for; 3. The housekeeper asks the buyer’s age.

[0044] After determining the merged label set, the merged label set can be set as the prediction label range of the target prediction sentence (query: Do you want to prepare some money for your child) in the target large language model.

[0045] In the present embodiment, based on the label elements of the merged label set, the prediction label range of the target large language model can be determined, and the prediction label of the target prediction sentence is selected from the prediction label range. In addition, in the present embodiment, a fewshot reference example is generated based on each sample in the similar sample set and the label elements to which each sample belongs, to assist the target large language model in performing inference prediction to obtain the classification label of the target prediction sentence.

[0046] Or Figure 2 For example, in a prompt generated by the embodiment of the present disclosure, the predicted label range is as follows Figure 2 The "##Definition of Key Points List" in the prompt is as follows: ## List of defined points a: Ask the user about the purpose of insurance b: The housekeeper asks who you want to buy it for c: The butler asks the buyer's age In addition, reference examples such as Figure 2 As shown in the "##discourse example" of prompt: ##Discourse Examples [Example 1 Input: Do you want to invest in education for your child? Output: The housekeeper asks the user about the purpose of insurance [Example 2] Input: Do you intend to provide lifelong security for your child, or do you want to prepare a transition for your child until he or she reaches adulthood? Output: The housekeeper asks the user about the purpose of insurance [Example 3] Input: Do you want to prepare insurance for your parents? How old are you? Output: The butler asks who to buy it for | The butler asks the buyer's age [Example 4] Input: Are you planning to use it for yourself? Or are you considering purchasing it for your family members? Output: The butler asks who to buy it for In addition, its target prediction statement is as follows Figure 2 The "##Current utterance" in the prompt is as follows: ##Current discourse Do you want to prepare some money for your children? Of course, it should be understood that the generated prompt may also include other content, such as Figure 2 The “requirement” in the prompt, etc.

[0047] In the disclosed embodiment, a label space is pre-set and speech samples of each label element are set, and then a similar sample set of a target predicted speech is retrieved based on the speech samples corresponding to the label elements in the label space and the label elements corresponding to the samples in the similar sample set are determined, thereby determining a predicted label range of a target large language model, and then a prompt of the target large language model is generated based on the samples in the similar sample set and the corresponding label elements, and then a classification label corresponding to the target predicted speech is predicted based on the prompt, which can greatly narrow the range of predicted labels and thereby greatly improve the efficiency of label classification and recognition of speech.

[0048] It should be understood that in specific applications, the method of the embodiment of the present disclosure may be implemented in multiple ways.

[0049] Optionally, in a specific implementation, the method may be performed as follows: Figure 2 The retrieval generation framework (Retrival-Generate-Frame) is implemented as shown in Figure 2As shown, the query and the key point pool can be input into the Retrival-Generate-Frame, where the key point pool includes key point label elements and corresponding speech sample sets; the Retrival-Generate-Frame can call the retrieval library (for example, Figure 2 urterance.lib) shown in the figure to obtain corresponding retrieval hits, which include the retrieved similar samples and corresponding labels; Retrival-Generate-Frame generates prompts for the target large language model based on the retrieval hits, and then the prompts can be input into the target large language model to obtain classification labels.

[0050] Figure 3 Schematic diagram of the structure of the label classification device of the embodiment of the present disclosure. Figure 3 As shown, the tag classification device 300 may include: A retrieval module 310 retrieves a similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space and determines the label element to which each sample in the similar sample set belongs; A generation module 320 generates a prompt based on each sample in the similar sample set and the label element to which each sample belongs, wherein the prompt carries the target predicted speech, the label prediction range consisting of the label element to which each sample in the similar sample set belongs, and the reference example consisting of each sample in the similar sample set and the label element to which each sample belongs; The large language model processing module 330 inputs the prompt into the target large language model to obtain a classification label corresponding to the target predicted utterance.

[0051] In the disclosed embodiment, the label classification device 300 pre-sets a label space and sets speech samples of each label element, then retrieves a similar sample set of the target predicted speech based on the speech samples corresponding to the label elements in the label space and determines the label elements corresponding to the samples in the similar sample set, thereby determining the predicted label range of the target large language model, and then generates a prompt of the target large language model based on the samples in the similar sample set and the corresponding label elements, and then predicts the classification label corresponding to the target predicted speech according to the prompt, which can greatly narrow the range of the predicted label and thereby greatly improve the label classification and recognition efficiency of the speech.

[0052] The above modules in the device provided by the embodiment of the present disclosure can also implement the method steps provided by the above method embodiment. Alternatively, the device provided by the embodiment of the present disclosure can also include other modules in addition to the above modules to implement the method steps provided by the above method embodiment. And the device provided by the embodiment of the present disclosure can achieve the technical effects that can be achieved by the above method embodiment.

[0053] Figure 4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0054] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0055] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0056] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming the above-mentioned label classification device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: Retrieving a similar sample set of the target predicted speech from a sample set corresponding to each label element in a preset label space and determining the label element to which each sample in the similar sample set belongs; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries the target predicted utterance, a label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0057] It should be understood that the label classification device of this embodiment can be used as Figure 1 The execution subject of the method shown can thus realize Figure 1 The steps and functions of the method shown are not repeated here.

[0058] As disclosed above Figure 1 The method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in one or more embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with one or more embodiments of the present disclosure can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0059] Of course, in addition to software implementation, the electronic device disclosed in the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0060] The present disclosure also provides a computer-readable storage medium that stores one or more programs. The one or more programs include instructions that, when executed by a portable electronic device that includes multiple application programs, enable the portable electronic device to execute Figure 1 The method of the embodiment shown is specifically used to perform the following operations: Retrieving a similar sample set of the target predicted speech from a sample set corresponding to each label element in a preset label space and determining the label element to which each sample in the similar sample set belongs; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries the target predicted utterance, a label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0061] In addition, the present disclosure also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute: Retrieving a similar sample set of the target predicted speech from a sample set corresponding to each label element in a preset label space and determining the label element to which each sample in the similar sample set belongs; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries the target predicted utterance, a label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

[0062] In short, the above description is only a preferred embodiment of the present disclosure and is not intended to limit the protection scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present disclosure shall be included in the protection scope of one or more embodiments of the present disclosure.

[0063] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0064] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0065] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0066] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A label classification method, comprising: Retrieving a similar sample set of the target predicted speech from a sample set corresponding to each label element in a preset label space and determining the label element to which each sample in the similar sample set belongs; Based on each sample in the similar sample set and the label elements to which each sample belongs, a prompt is generated and input into the target large language model to obtain a classification label corresponding to the target predicted utterance, wherein the prompt carries the target predicted utterance, a label prediction range consisting of the label elements to which each sample in the similar sample set belongs, and a reference example consisting of each sample in the similar sample set and the label elements to which each sample belongs.

2. The method according to claim 1, retrieving a similar sample set of the target predicted utterance from the sample set corresponding to each label element in the preset label space, comprising: Retrieving a first similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space by a first similarity retrieval method; Retrieving a second similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space by a second similarity retrieval method; The similar sample set is determined based on the first similar sample set and the second similar sample set.

3. The method according to claim 2, wherein the first similar sample set is a first preset number of samples ranked top in similarity; and the second similar sample set is a second preset number of samples ranked top in similarity.

4. The method according to claim 2, before determining the similar sample set based on the first similar sample set and the second similar sample set, the method further comprises: Retrieving a third similar sample set of the target predicted utterance from the sample sets corresponding to each label element in the preset label space by a third similarity retrieval method; Determining the similar sample set based on the first similar sample set and the second similar sample set includes: The similar sample set is determined based on the first similar sample set, the second similar sample set and the third similar sample set.

5. As described in the method of claim 2, the first similarity search method includes a vector search method, and the second similarity search method includes a literal similarity search method.

6. The method as claimed in claim 1, wherein the preset label space includes label elements greater than a third preset number, each element is pre-labeled with one or more speech samples of a target application scenario, and the speech samples are represented by vectors.

7. A label classification device, comprising: A retrieval module retrieves a similar sample set of the target predicted utterance from a sample set corresponding to each label element in a preset label space and determines the label element to which each sample in the similar sample set belongs; A generation module, based on each sample in the similar sample set and the label element to which each sample belongs, generates a prompt, wherein the prompt carries the target predicted speech, the label prediction range consisting of the label element to which each sample in the similar sample set belongs, and the reference example consisting of each sample in the similar sample set and the label element to which each sample belongs; The large language model processing module inputs the prompt into the target large language model to obtain a classification label corresponding to the target predicted utterance.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 6 when executed by the processor.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute part or all of the steps of the method according to any one of claims 1 to 6.

Images