Data classification method, device, storage medium, and program product

Abstract

Embodiments of the present application provide a data classification method, device, storage medium and program product. The method comprises: obtaining an input text to be classified and an original label set containing a large number of classification labels; splitting the original label set into a plurality of sub-label sets; determining, in parallel, initial screening classification labels corresponding to the input text under the plurality of sub-label sets according to the plurality of sub-label sets and a first large language model; and determining a target classification label corresponding to the input text from the initial screening classification labels corresponding to the input text under the plurality of sub-label sets according to a second large language model. The accuracy of the classification result can be improved by the scheme.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data classification method, device, storage medium, and program product. Background Technology

[0002] Data classification is required in many applications, such as classifying and recognizing the content contained in images, or classifying and recognizing text content based on a set of tags. When classifying data (such as images and text), it is necessary to assign a matching classification label to the data to be classified from a given tag set, that is, to determine the classification label corresponding to the data.

[0003] In tagging and classification scenarios such as multi-level tagging, the number of category tags in the tag set may be very large. For example, in a thousand-category task, there may be thousands of category tags.

[0004] Taking text classification as an example, Large Language Models (LLMs) can be used for data classification due to their powerful semantic understanding capabilities. However, directly using an LLM to classify input text with a large label set results in low accuracy. Summary of the Invention

[0005] This application provides a data classification method, device, storage medium, and program product to improve the accuracy of data classification results.

[0006] In a first aspect, embodiments of this application provide a data classification method, the method comprising:

[0007] Obtain the input text to be classified and multiple sub-label sets, wherein the multiple sub-label sets are obtained by splitting the original label set;

[0008] Based on the multiple sub-label sets and the first large language model, the initial screening classification labels corresponding to the input text under the multiple sub-label sets are determined in parallel.

[0009] Based on the second language model, the target classification label corresponding to the input text is determined from the initial screening classification labels corresponding to the input text under the multiple sub-label sets.

[0010] Secondly, embodiments of this application provide a data classification device, the device comprising:

[0011] The acquisition module is used to acquire the input text to be classified and multiple sub-label sets, wherein the multiple sub-label sets are obtained by splitting the original label set;

[0012] The grouping module is used to split the original tag set into multiple sub-tag sets;

[0013] The initial screening module is used to determine the initial screening classification labels corresponding to the input text under the multiple sub-label sets in parallel based on the multiple sub-label sets and the first large language model;

[0014] The fine screening module is used to determine the target classification label corresponding to the input text from the initial screening classification labels corresponding to the input text under the multiple sub-label sets, based on the second language model.

[0015] Thirdly, embodiments of this application provide a data classification method, the method comprising:

[0016] Receive a request triggered by a client device by calling a data classification service, and obtain the input text to be classified and multiple sub-label sets according to the request. The multiple sub-label sets are obtained by splitting the original label set.

[0017] The following steps are performed using the processing resources corresponding to the data classification service:

[0018] Based on the multiple sub-label sets and the first large language model, the initial screening classification labels corresponding to the input text under the multiple sub-label sets are determined in parallel.

[0019] Based on the second language model, the target classification label corresponding to the input text is determined from the initial screening classification labels corresponding to the input text under the multiple sub-label sets respectively;

[0020] The target classification label is fed back to the client device.

[0021] Fourthly, embodiments of this application provide an electronic device, including: a memory, a processor, and a communication interface; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor performs the data classification method as described in the first or third aspect.

[0022] Fifthly, embodiments of this application provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the data classification method as described in the first or third aspect.

[0023] In a sixth aspect, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, is capable of implementing the data classification method as described in the first or third aspect.

[0024] In the data classification scheme provided in this application embodiment, when the number of classification tags contained in the original tag set is large, causing the number of tokens occupied to exceed a set threshold, the original tag set can be split into multiple sub-tag sets. This reduces the number of tokens corresponding to the classification tags in each sub-tag set. Based on this, a multi-path (one sub-tag set corresponds to one path) parallel approach can be adopted. The first large language model performs initial screening of the classification tags corresponding to each sub-tag set for the input text. At this time, since the number of tokens corresponding to each sub-tag set input to the first large language model is significantly reduced, the first large language model can complete the initial screening of classification tags under the corresponding sub-tag set faster and more accurately. Finally, the multiple initial screening classification tags are aggregated to refine the selection of the target classification tags corresponding to the input text using the second large language model. The number of tokens corresponding to the initial screening classification tags input to the second large language model is even smaller, allowing the second large language model to determine the target classification tags faster and more accurately. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram of the hardware execution environment for a data classification method provided in an embodiment of this application;

[0027] Figure 2 This is an application diagram of a cloud computing environment provided in an embodiment of this application;

[0028] Figure 3 A flowchart illustrating a data classification method provided in this application embodiment;

[0029] Figure 4 This is a schematic diagram illustrating the application of a data classification method provided in an embodiment of this application;

[0030] Figure 5 A flowchart illustrating another data classification method provided in this application embodiment;

[0031] Figure 6 This is a schematic diagram illustrating the application of another data classification method provided in the embodiments of this application;

[0032] Figure 7 This is a schematic diagram of the structure of a data classification device provided in an embodiment of this application;

[0033] Figure 8This is a schematic diagram of the structure of an electronic device provided in this embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. In addition, the timing of the steps in the following method embodiments is only an example and not a strict limitation.

[0035] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to large language models or other models) comply with relevant laws and standards.

[0036] First, the terms or concepts involved in the embodiments of this application will be explained:

[0037] Text classification: Using the output of the classification model, one or more specified classification labels are assigned to the input text from a pre-defined closed set of labels.

[0038] Large Language Model (LLM): This refers to a deep learning model trained on massive amounts of text data, possessing powerful expressive and generalization capabilities. It can not only generate natural language text but also deeply understand the meaning of text, handling various natural language tasks such as text summarization, question answering, and classification.

[0039] Prompt Template: This is a pre-defined prompt input to the large language model, enabling it to generate relevant responses based on the prompts. For different classification scenarios, the tag set can be populated within the prompt template to adapt to various situations.

[0040] Multi-level tags: Category tags consist of multiple levels of content. For example, in a product category scenario,

Furniture

Bathroom

Smart Toilet

[0041] Multi-level recall: In multi-level label classification tasks, for input text, each level of classification label is assigned a classification label step by step according to the classification label level, instead of classifying all of them at once.

[0042] Multi-path recall: In classification tasks, for the input text, the label set is divided into multiple subsets, with each subset representing one path of labels. This significantly reduces the number of classification tasks for each path when the model assigns classification labels to the input text. Finally, the classification results from each path are aggregated to produce the final output classification labels.

[0043] Voice of Customers (VOC): Input text for text classification consists of genuine written feedback from customers regarding a brand's products. Assigning multi-level category labels to VOCs helps businesses better process data and analyze authentic customer feedback on products.

[0044] In some real-world applications, to better analyze genuine customer feedback on products, companies need to perform multi-level tagging and classification on collected input text such as VOCs and product titles. In such cases, the number of classification tags in the tag set may reach thousands. These input texts often contain rich semantic information, so using LLM for classification performs better than traditional pattern recognition methods. However, due to the large number of classification tags, direct LLM classification results in poor accuracy and consumes significantly more computational resources.

[0045] Specifically, on the one hand, when the number of category labels in the label set is too large, it becomes difficult to determine the target category label corresponding to the input text from a large number of category labels, resulting in lower classification accuracy compared to determining the target category label from a smaller number of category labels. On the other hand, when using LLM for classification, the entire label set needs to be filled into the prompt word template to serve as the prompt words input to the LLM. This consumes a lot of inference resources—token consumption is high, meaning the LLM needs to understand the meaning of a large number of category labels, which also reduces the efficiency of the LLM's classification processing.

[0046] Therefore, this application provides a data classification scheme to improve the accuracy of data classification results and even reduce the inference resource consumption of LLM when the number of classification tags in the tag set is large: a multi-path recall mechanism and a multi-level recall mechanism. The multi-path recall mechanism can significantly improve the accuracy of classification results without increasing inference resource consumption. When the classification tags in the tag set are in a multi-level format, the multi-level recall mechanism can reduce inference resource consumption and improve the accuracy of classification results.

[0047] The data classification scheme provided in the embodiments of this application will be described below.

[0048] Figure 1 A schematic diagram of the hardware execution environment for a data classification method provided in an embodiment of this application is shown below. Figure 1 As shown, the hardware execution environment for this data classification method can consist of a client device 101 and a server device 102, with the client device 101 and server device 102 communicating with each other. The server device 102 can be a cloud server from a cloud service provider. The client device 101 can be a laptop, tablet, PC, robot, etc.

[0049] In an optional embodiment, the large language model (hereinafter referred to as the first large language model and the second large language model) provided in this application embodiment can be set in the server device 102. In this case, the execution process of the above data classification method can be as follows: the client device 101 sends the input text to be classified and the original tag set to the server device 102. The number of tokens corresponding to the classification tags contained in the original tag set is greater than a first set threshold, that is, the original tag set is a tag set containing a large number of classification tags. The server device 102 splits the original tag set into multiple sub-tag sets. Then, based on the multiple sub-tag sets and the first large language model, it determines the initial screening classification tags corresponding to the input text under each of the multiple sub-tag sets in parallel. Based on the second large language model, it determines the target classification tag corresponding to the input text from the initial screening classification tags corresponding to the input text under each of the multiple sub-tag sets, and feeds back the target classification tag to the client device 101.

[0050] In practical applications, the aforementioned server-side device 102 can be a cloud server maintained by a cloud service provider—referred to as a computing node. In cases such as... Figure 2 The cloud computing environment shown may include several distributed deployments. Figure 2 The diagram illustrates compute nodes (201-1, 201-2, ...), each possessing processing resources such as computing and storage. In a cloud computing environment, multiple compute nodes can be organized to provide a specific service; conversely, a single compute node can provide one or more services. Figure 2 The diagram illustrates services A, B, C, and D. In a cloud computing environment, these services can be provided via external service interfaces, which client device 101 calls to use the corresponding services. Service interfaces can take the form of Software Development Kits (SDKs), Application Programming Interfaces (APIs), etc.

[0051] The services described above are deployed using various virtualization technologies supported by cloud computing environments, such as virtual machine-based and container-based virtualization technologies. Taking container-based virtualization technology as an example, several containers corresponding to a service can be assembled into a container group (pod). For example... Figure 2 The illustrated service B can be configured with one or more pods, and each pod can include a proxy and one or more containers. The one or more containers in the pod are used to handle requests related to one or more corresponding functions of the service, and the proxy in the pod is used to control network functions related to the service, such as routing and load balancing.

[0052] During operation, executing a request from client device 101 may require invoking one or more services in the cloud computing environment, and executing one or more functions of one service may require invoking one or more functions of another service. For example... Figure 2 As shown, after receiving a request from client device 101, service A can call service B, and service B can request service D to perform one or more functions.

[0053] In this embodiment of the application, a cloud service for data classification can be deployed on one or more computing nodes, and the cloud service integrates the large language model mentioned above.

[0054] The execution process of the data classification method provided in this application embodiment is described in detail below with reference to the accompanying drawings. This data classification method can be executed by computing nodes in the aforementioned cloud computing environment.

[0055] Figure 3 This is a flowchart illustrating a data classification method provided in an embodiment of this application. Figure 3 As shown, the method includes the following steps:

[0056] 301. Obtain the input text to be classified and multiple sub-label sets. The multiple sub-label sets are obtained by splitting the original label sets.

[0057] 302. Based on multiple sub-label sets and the first large language model, determine the initial classification labels corresponding to the input text under the multiple sub-label sets in parallel.

[0058] 303. Based on the second language model, determine the target classification label corresponding to the input text from the initial screening classification labels corresponding to the input text under multiple sub-label sets.

[0059] The data classification scheme provided in this application can be applied to data classification scenarios where the number of classification tags in the original tag set is large, including but not limited to cases where the classification tags are in a multi-level form. A large number of classification tags can be reflected in the large number of tokens they occupy, such as exceeding a first set threshold (e.g., 20,000), or the total number of classification tags exceeding a set threshold (e.g., 5,000).

[0060] In practical applications, according to the definition of a token, such as a character or a word, a category label may occupy one or more tokens. Therefore, when the number of category labels is large, the overall token length is also large. If all these tokens are used to fill the prompt word template of a large language model, the large language model will need to read and understand these tokens when classifying the input data, consuming too many resources.

[0061] In different application scenarios, the input text to be classified will be different. For example, in the retail field, the input text can be customer feedback descriptions of products (such as VOCs) or product descriptions. Accordingly, the classification tags in the tag set can be multi-level tags that reflect the category to which the feedback question belongs, or classification tags that are used to classify products.

[0062] To ensure the accuracy of LLM's classification results for input text, this embodiment employs a multi-path recall mechanism.

[0063] Specifically, firstly, the original tag set can be split into multiple sub-tag sets.

[0064] Optionally, the number of sub-tag sets can be preset, and the category tags in the original tag set can be evenly split according to this number. During the even distribution, a random sampling strategy can be used: for example, if each sub-tag set contains 100 category tags based on this number, then 100 category tags can be randomly selected from the original tag set as the first sub-tag set, and another 100 can be randomly selected from the remaining category tags as the second sub-tag set, and so on.

[0065] Alternatively, an upper limit can be pre-set for the number of tokens corresponding to category tags contained in each sub-tag set. When splitting the original tag set into multiple sub-tag sets, the number of tokens corresponding to category tags contained in each sub-tag set is less than or equal to a second set threshold, which is less than the first set threshold mentioned above. In this case, the random sampling strategy described above can still be used.

[0066] Optionally, in practical applications, the number of category tags contained in each sub-tag set can be "equal". Equal does not necessarily mean that the number is completely the same, but only that the difference in number is small.

[0067] Understandably, if the category tags in the original tag set are not in the form of multi-level tags, but consist of several category tags at the same level, then these category tags can be divided into multiple sub-tag sets on an average basis according to the set number of sub-tag sets or the second set threshold mentioned above.

[0068] When the category tags in the original tag set are in the form of multi-level tags, in one optional implementation, the original tag set can be split into multiple sub-tag sets based on the first-level category tags in the original tag set. Each sub-tag set includes at least one first-level category tag from the original tag set, as well as other level category tags associated with that first-level category tag. The number of sub-tag sets or the aforementioned second preset threshold can be preset, and then the first-level category tags in the original tag set are split evenly according to this number or the second preset threshold to obtain multiple sub-tag sets. It is understood that the sub-tag sets obtained according to this partitioning strategy will not have any overlap; that is, the same category tag will not appear repeatedly in different sub-tag sets.

[0069] When the category tags in the original tag set are in the form of multi-level tags, another optional implementation can also perform similar splitting processing on category tags at levels other than the first-level category tags. For example, in this case, random and even distribution can be performed starting from the lowest-level category tags in reverse order: the number of sub-tag sets is pre-set, and the low-level category tags are randomly sampled to be evenly distributed to each sub-tag set. Then, the parent category tags corresponding to these low-level category tags in each sub-tag set are associatedly distributed to the corresponding sub-tag sets. For example, assuming there are 1000 third-level category tags and 3 sub-tag sets in a three-level tag scenario, 300 third-level category tags are randomly selected and added to the first sub-tag set, another 300 are randomly selected and added to the second sub-tag set, and the remaining 400 are added to the third sub-tag set. Then, the level 2 and level 1 category labels corresponding to the 300 category labels in the first sub-label set are added to the first sub-label set. Similarly, the same process is performed on the second and third sub-label sets. It should be noted that, at this point, duplicate category labels may appear in different sub-label sets.

[0070] After obtaining multiple sub-label sets, the initial classification labels corresponding to the input text under each sub-label set are determined in parallel based on these sub-label sets and the first language model. In practical applications, we can set the number of classification labels that the first language model needs to determine from each sub-label set that match the input text, k ≥ 1. That is, for a certain sub-label set, the first language model needs to predict and output the k classification labels that best match the input text (the classification labels that the input text is most likely to belong to), as the initial classification labels.

[0071] In an alternative embodiment, a first large language model can be determined for each sub-tag set, thereby utilizing multiple first large language models to achieve the above-mentioned parallel processing.

[0072] In an optional embodiment, the parallel use of the aforementioned first large language model can be achieved by calling an Application Programming Interface (API). For example, a cloud service providing data classification services may have an API that allows calls to various large language models. Specifically, for the current input text and multiple sub-label sets, call requests corresponding to the multiple sub-label sets can be sent to the API in parallel. This allows the API to call at least one first large language model to determine the initial classification label of the input text under the corresponding sub-label set. Finally, the API is used to obtain the initial classification label output by the at least one first large language model for each sub-label set. Implementing the above parallel processing through API calls reduces the amount of first large language model used, thereby reducing the storage, computing, and other resources required to load multiple first large language models, achieving efficient parallel processing.

[0073] The call requests corresponding to the multiple sub-label sets include a first prompt word, which includes the input text and a corresponding sub-label set. This prompt word is used to prompt the first language model to determine the k matching classification labels from the corresponding sub-label set for the input text.

[0074] In practical applications, optionally, the API can call different primary language models for each sub-label set request, thereby achieving parallel processing of the initial classification labels of the input text under multiple sub-label sets through multiple primary language models; or, it can call the same primary language model to achieve parallel processing of the initial classification labels of the input text under multiple sub-label sets through batch processing.

[0075] Because each sub-label set contains a relatively small number of classification labels, it requires fewer tokens, thus significantly improving the classification processing speed of the first language model. Parallel processing further enhances classification efficiency. Moreover, selecting initial classification labels from fewer sub-label sets makes it easier to obtain accurate results.

[0076] Then, based on the second language model, the target classification label corresponding to the input text is determined from the initial screening classification labels corresponding to the input text under multiple sub-label sets. Optionally, the parameter size of the first language model is smaller than that of the second language model. Using the better-performing second language model can better ensure the accuracy of the target classification label determination result.

[0077] Specifically, a second prompt word can be generated. This second prompt word includes the input text and its initial classification labels across multiple sub-label sets. The second prompt word is then input into a second language model to obtain the target classification label output by the second language model. In essence, the second prompt word prompts the second language model to further identify the target classification label that best matches the input text from the previously selected classification labels. In practical applications, the input text and its initial classification labels across multiple sub-label sets can be filled into a predefined prompt word template to generate the second prompt word.

[0078] The above is the data classification method implemented using a multi-path recall mechanism provided in the embodiments of this application. For ease of understanding, the following is combined with... Figure 4 This illustrates the execution process of the multi-channel recall mechanism.

[0079] exist Figure 4 In this example, assuming the original tag set contains thousands of category tags, it is then split into three sub-tag sets, as shown in the diagram. Each sub-tag set is assumed to contain 7700 tokens corresponding to the category tags. For the current input text, the initial category tags for each sub-tag set can be determined in parallel across all three sets. Specifically, as shown... Figure 4As shown in the diagram, in the first recall process, the input prompt word 1 of the first large language model includes 7700 tokens from the corresponding sub-label set; in the second recall process, the input prompt word 2 of the first large language model includes 7700 tokens from the corresponding sub-label set; and in the third recall process, the input prompt word 3 of the first large language model includes 7700 tokens from the corresponding sub-label set. Assume that each recall process yields k initial screening classification labels. In the subsequent fine screening process, the k initial screening classification labels from each of the three processes are combined (a total of 3*k labels). Assuming that 300 tokens are used at this point, these 3*k initial screening classification labels are used as part of the prompt word 4 of the second large language model. Finally, the second large language model refines and selects a target classification label from these, which becomes the final classification labeling result for the input text.

[0080] As described above, during the initial screening process, parallel execution across multiple paths, with each path containing a smaller number of tokens for its sub-label set, significantly improves inference speed, reduces classification difficulty, and ultimately increases accuracy. However, the total token usage across multiple paths is not significantly reduced compared to directly inputting the original label set into the LLM. Furthermore, during the fine screening process, the number of input initial screening labels is even smaller, resulting in a negligible number of tokens compared to the initial screening process. Therefore, employing this multi-path recall mechanism can significantly improve the efficiency and accuracy of classification processing without substantially increasing token usage.

[0081] When the classification labels in the original label set are in the form of multi-level classification labels, this application embodiment also provides a multi-level recall mechanism. In general, the input text is labeled with classification labels in descending order of classification label level, so as to significantly reduce the total resource consumption (mainly referring to the processing consumption of tokens by the large language model).

[0082] In fact, the multi-path recall mechanism and the multi-level recall mechanism are decoupled and can be used separately or in combination. Ultimately, the combined use of the multi-level and multi-path recall mechanisms can significantly reduce the inference cost of large language models in the classification process and improve classification accuracy.

[0083] The following is combined Figure 5 Explain the multi-level recall mechanism.

[0084] Figure 5 A flowchart illustrating another data classification method provided in an embodiment of this application. For example... Figure 5 As shown, the method includes the following steps:

[0085] 501. Obtain the input text to be classified and the original label set, which contains multi-level classification labels.

[0086] 502. Based on the first-level category labels in the original label set, split the original label set into multiple sub-label sets, wherein each sub-label set includes at least one first-level category label from the original label set and other level category labels associated with that at least one first-level category label.

[0087] 503. For any target sub-label set in multiple sub-label sets, the first large language model is used to determine the initial screening classification label corresponding to the input text in each level of classification label of any target sub-label set step by step. Based on the initial screening classification label corresponding to the input text in each level of classification label of any target sub-label set, the target initial screening classification label corresponding to the input text in any target sub-label set is determined.

[0088] 504. Based on the second language model, determine the target classification label corresponding to the input text from the target initial screening classification labels corresponding to the input text under multiple sub-label sets.

[0089] Optionally, the number of tokens corresponding to the category tags contained in the original tag set is greater than a set threshold.

[0090] In this embodiment, step 502 uses a sub-tag set division method based on the first-level classification tag. As described in the other embodiments above, the division method is not limited to this method. The execution of step 503 can be a detailed expansion of any one of the recall processes in the multi-path recall process executed in parallel in the aforementioned embodiments, in the case of multi-level tags. This one recall process corresponds to any target sub-tag set. That is to say, a multi-level recall mechanism can be introduced in each recall process of the multi-path recall.

[0091] Optionally, in step 503, the initial screening classification label corresponding to the input text in each level of the target sub-label set is determined by using the first major language model step by step. This can be achieved by using one or more first major language models in parallel to identify the matching initial screening classification labels for each level of the target sub-label set. Then, the identification results that can form known multi-level labels are determined from the identification results of each level to form a target initial screening classification label.

[0092] The above recognition process is illustrated using a two-level classification label scenario as an example. When starting the recall corresponding to the target sub-label set, all Level 1 classification labels contained in the target sub-label set are first determined to generate a prompt word x containing these Level 1 classification labels and the input text. The prompt word x is then input into a first-level language model, which determines p Level 1 classification labels that match the input text based on the prompt word x among these Level 1 classification labels, where p is a preset value greater than or equal to 1.

[0093] Simultaneously, all Level 2 category labels contained in the target sub-label set are determined to generate a prompt word y containing these Level 2 category labels and the input text. The prompt word y is then input into another primary language model, which determines q Level 2 category labels that match the input text from these Level 2 category labels based on the prompt word y, where q is a preset value greater than or equal to 1.

[0094] Next, we determine the combinations of known multi-level labels formed by p first-level category labels and q second-level category labels. Specifically, we examine which of the q second-level category labels are the next-level category labels of a certain label in the p first-level labels, thus forming the initial target category labels. For example, if two of the q second-level category labels are the next-level category labels of a certain first-level label in p, then two initial target category labels can be formed.

[0095] Optionally, in step 503, the initial classification label corresponding to the input text in each level of the target sub-label set is determined step by step using the first large language model. This can be implemented as follows:

[0096] Generate a third prompt word, which includes the input text and the first-level category label from the target sub-label set;

[0097] Input the third prompt word into the first language model to obtain the first-level initial classification label output by the first language model;

[0098] Generate a fourth prompt word, which includes the input text and the (i+1)th level classification label corresponding to the i-th level initial screening classification label. The i-th level initial screening classification label is the classification label level that has been identified in the initial screening classification label according to the classification label level order, i≥1;

[0099] Input the fourth prompt word into the first language model to obtain the (i+1)th level initial classification label output by the first language model.

[0100] The generation methods for the third and fourth prompt words mentioned above are the same as those for the second prompt word, and will not be repeated here.

[0101] The following is combined Figure 6The above step-by-step identification process will be illustrated using a two-level classification label example.

[0102] When starting the recall process corresponding to the target sub-label set, the first-level category labels are determined from the target sub-label set to generate a prompt word 'a' containing these first-level category labels and the input text. The prompt word 'a' is then input into the first language model, which determines m first-level category labels that match the input text based on these first-level category labels, where m is a preset value greater than or equal to 1.

[0103] Next, second-level category labels associated with these m first-level category labels are selected from the target sub-label set (assuming m = 2, where each first-level category label is associated with 10 second-level category labels, resulting in a total of 20 second-level category labels). This generates a prompt word b containing these second-level category labels (the aforementioned 20 second-level category labels and their correspondence with the m first-level category labels) and the input text. The prompt word b is then input into the first language model, which uses it to determine n second-level category labels matching the input text from the selected second-level category labels associated with each first-level category label. Here, n is a preset value greater than or equal to 1. For example, if n = 1, then each selected first-level category label will ultimately result in one second-level category label.

[0104] Understandably, at this point, each of the final selected level 2 category labels and its associated level 1 category label will serve as a target initial screening category label corresponding to the input text under the target sub-label set.

[0105] The above example only illustrates the two-level tag system. In fact, the processing of more levels of category tags is the same, and will not be repeated here.

[0106] In summary, employing a multi-level recall mechanism can significantly reduce token consumption. Specifically, assuming the original tag set contains K1 first-level category tags, each of which contains K2 second-level category tags, and each category tag requires one token, the original total token count would be on the order of K1*K2. If all the category tags from this original tag set were directly filled into the LLM's prompts, and the LLM completed the target category tag determination process for the input text in one go, it would mean that the LLM would need to input and process K1*K2 tokens. However, with a multi-level recall mechanism, through the aforementioned... Figure 6As the example illustrates, the multiplicative token consumption can be converted into an additive one: K1 level 1 category labels are filled into prompt word 'a', and the LLM completes one inference (classification) process. Then, in the process of determining the next level 2 category label, at most K2 level 2 category labels are filled into prompt word 'b', and the LLM completes another inference (classification) process. The token consumption in both inference processes is at most on the order of K1 + K2, which significantly reduces token consumption, thereby contributing to improved classification efficiency and accuracy.

[0107] In addition, to further improve the accuracy of classification results, this application embodiment also provides the following prefix continuation optimization method during the execution of the multi-level recall mechanism. Specifically, during the screening process of the first-level classification label, the third prompt word may also include the first prefix content, which instructs the first large language model to determine the first-level initial screening classification label in a thought chain manner. During the screening process of classification labels at subsequent levels such as the second level, the fourth prompt word also includes the second prefix content, which includes the initial screening classification labels obtained before the (i+1)th level initial screening classification label. The second prefix content instructs the first large language model to output the (i+1)th level initial screening classification label based on the second prefix content. For example, during the screening process of the second-level classification label, the corresponding fourth prompt word may include the first-level initial screening classification label, so that the first large language model continues to output the corresponding second-level classification label screening result based on the output result of the first-level initial screening classification label. Similarly, during the screening process of the third-level classification label, the corresponding fourth prompt word may include the first-level initial screening classification label and the second-level initial screening classification label, and so on. By adding the above content to the third and fourth prompt words, the first language model can be guided to output "following text" that is semantically consistent and coherent based on the "preceding text" content. This makes the output following text more reasonable and helps improve the accuracy of the classification results.

[0108] To more intuitively illustrate the performance improvement of the data classification methods using the multi-path recall mechanism and multi-level recall mechanism provided in the embodiments of this application, Table 1 illustrates the performance comparison results of different data classification methods.

[0109] Table 1 shows the original tag set for a specific brand's real-world VOC tagging scenario. The original tag set consists of four levels of tags. For example, a multi-level classification tag example might be: new car delivery / waiting time after order locking / unsatisfactory delivery time / complaints about long waiting times. In this test scenario, the performance of three data classification methods was compared: traditional methods using LLM and the original tag set for input text tagging and classification; methods using the multi-path recall mechanism provided in this application embodiment; and methods using a combination of multi-path recall and multi-level recall mechanisms provided in this application embodiment. The results are shown in Table 1, focusing on classification accuracy, average token consumption, and average response time (i.e., classification processing time). It is evident that the use of multi-path recall and multi-level recall mechanisms can improve the accuracy of classification results, reduce token consumption, and increase classification processing efficiency.

[0110] Table 1

[0111] Data classification methods Classification accuracy Average token consumption Average response time Original tag set + LLM 44.02％ 22513 5.31 seconds Multiple recalls 82.01％ 23215 3.11 seconds Multi-channel + multi-level recall 93.02％ 3735 5.04 seconds

[0112] Figure 7 This is a schematic diagram of the structure of a data classification device provided in an embodiment of this application, as shown below. Figure 7 As shown, the device includes: an acquisition module 11, a grouping module 12, a primary screening module 13, and a fine screening module 14.

[0113] The acquisition module 11 is used to acquire the input text to be classified and multiple sub-label sets, wherein the multiple sub-label sets are obtained by splitting the original label set.

[0114] The initial screening module 12 is used to determine, in parallel, the initial screening classification labels corresponding to the input text under the multiple sub-label sets and the first large language model.

[0115] The fine screening module 13 is used to determine the target classification label corresponding to the input text from the initial screening classification labels corresponding to the input text under the multiple sub-label sets, based on the second language model.

[0116] Optionally, the parameter size of the first largest language model is smaller than the parameter size of the second largest language model.

[0117] Optionally, the initial screening module 12 is specifically used to: send invocation requests corresponding to the plurality of sub-label sets to the set application interface in parallel, so as to invoke at least one of the first large language models through the application interface, each invocation request including a first prompt word, the first prompt word including the input text and a corresponding sub-label set; and obtain the initial screening classification labels output by the at least one first large language model through the application interface.

[0118] Optionally, the fine screening module 13 is specifically used to: generate a second prompt word, the second prompt word including the input text and the preliminary screening classification labels corresponding to the input text under the multiple sub-label sets respectively; input the second prompt word into the second large language model to obtain the target classification label output by the second large language model.

[0119] Optionally, if the original tag set contains multi-level classification tags, the device further includes a grouping module for splitting the original tag set into multiple sub-tag sets based on the first-level classification tags in the original tag set, wherein each sub-tag set includes at least one first-level classification tag from the original tag set and other level classification tags associated with the at least one first-level classification tag.

[0120] At this time, the initial screening module 12 is specifically used to: for any target sub-label set in the plurality of sub-label sets, use the first large language model to determine the initial screening classification label corresponding to the input text in each level of classification label in any target sub-label set; and determine the target initial screening classification label corresponding to the input text in any target sub-label set based on the initial screening classification label corresponding to the input text in each level of classification label in any target sub-label set.

[0121] In the process of using the first large language model to determine the initial screening classification label corresponding to the input text in each level of classification label of any target sub-label set, the initial screening module 12 is specifically used to: generate a third prompt word, the third prompt word including the input text and the first level classification label in the target sub-label set; input the third prompt word into the first large language model to obtain the first level initial screening classification label output by the first large language model; generate a fourth prompt word, the fourth prompt word including the input text and the (i+1)th level classification label corresponding to the i-th level initial screening classification label, the i-th level initial screening classification label being the classification label level that has been identified in the initial screening classification label according to the classification label level order, i≥1; input the fourth prompt word into the first large language model to obtain the (i+1)th level initial screening classification label output by the first large language model.

[0122] Optionally, the third prompt word also includes a first prefix, which instructs the first large language model to determine the first-level initial screening classification label in a thought chain manner; the fourth prompt word also includes a second prefix, which includes the initial screening classification labels obtained before the (i+1)th level initial screening classification label, and the second prefix indicates that the first large language model outputs the (i+1)th level initial screening classification label based on the second prefix.

[0123] Figure 7 The apparatus shown can perform the steps in the data classification method in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0124] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 8 As shown, in practice, this electronic device includes a memory 21 and a processor 22.

[0125] Memory 21 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0126] The processor 22, coupled to the memory 21, is used to execute the computer program in the memory 21 to implement the data classification method provided in the foregoing embodiments.

[0127] Furthermore, such as Figure 8 As shown, the electronic device also includes other components such as a communication component 23, a display 24, a power supply component 25, and an audio component 26. Figure 8 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 8 The components shown are as follows. The electronic device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server device such as a conventional server, cloud server, or server array.

[0128] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0129] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0130] The aforementioned display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0131] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0132] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0133] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, digital video disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium.

[0134] Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data classification method, characterized in that, The method includes: Obtain the input text to be classified and multiple sub-label sets, wherein the multiple sub-label sets are obtained by splitting the original label set; Based on the multiple sub-label sets and the first large language model, the initial screening classification labels corresponding to the input text under the multiple sub-label sets are determined in parallel. Based on the second language model, the target classification label corresponding to the input text is determined from the initial screening classification labels corresponding to the input text under the multiple sub-label sets.

2. The method according to claim 1, characterized in that, The parameter size of the first largest language model is smaller than that of the second largest language model.

3. The method according to claim 1, characterized in that, The step of determining the initial classification labels corresponding to the input text under the multiple sub-label sets and the first large language model in parallel includes: In parallel, call requests corresponding to the plurality of sub-tag sets are sent to the set application interface to call at least one of the first large language models through the application interface. Each call request includes a first prompt word, and the first prompt word includes the input text and a corresponding sub-tag set. The initial classification labels output by the at least one first language model are obtained through the application programming interface.

4. The method according to claim 1, characterized in that, Based on the second major language model, the target classification label corresponding to the input text is determined from the initial screening classification labels corresponding to the input text under the multiple sub-label sets, including: Generate a second prompt word, which includes the input text and the initial screening classification labels corresponding to the input text under the multiple sub-label sets respectively; The second prompt word is input into the second large language model to obtain the target classification label output by the second large language model.

5. The method according to any one of claims 1-4, characterized in that, The original tag set contains multi-level classification tags, and the splitting process of the multiple sub-tag sets includes: Based on the first-level category label in the original label set, the original label set is split into multiple sub-label sets, wherein each sub-label set includes at least one first-level category label from the original label set and other level category labels associated with the at least one first-level category label.

6. The method according to claim 5, characterized in that, The step of determining the initial classification labels corresponding to the input text under the multiple sub-label sets and the first large language model in parallel includes: For any target sub-label set in the plurality of sub-label sets, the first large language model is used to determine the initial screening classification label corresponding to the input text in each level of classification label in any target sub-label set step by step; Based on the initial screening classification label corresponding to the input text in each level of the classification label of any target sub-label set, determine the target initial screening classification label corresponding to the input text in any target sub-label set.

7. The method according to claim 6, characterized in that, The step of using the first large language model to determine the initial classification label corresponding to the input text in each level of classification label of any target sub-label set includes: Generate a third prompt word, which includes the input text and the first-level category label in the target sub-label set; The third prompt word is input into the first large language model to obtain the first level of preliminary classification label output by the first large language model; Generate a fourth prompt word, which includes the input text and the (i+1)th level classification label corresponding to the i-th level initial screening classification label. The i-th level initial screening classification label is the classification label level that has been identified after initial screening, determined according to the classification label level order, i≥1. The fourth prompt word is input into the first large language model to obtain the (i+1)th level initial screening classification label output by the first large language model.

8. The method according to claim 7, characterized in that, The third prompt word also includes a first prefix content, which instructs the first large language model to determine the first-level initial screening classification label in a thought chain manner; The fourth prompt word also includes a second prefix, which includes the initial screening classification labels obtained before the (i+1)th level initial screening classification label. The second prefix indicates that the first large language model outputs the (i+1)th level initial screening classification label based on the second prefix.

9. A data classification method, characterized in that, The method includes: Receive a request triggered by a client device by calling a data classification service, and obtain the input text to be classified and multiple sub-label sets according to the request. The multiple sub-label sets are obtained by splitting the original label set. The following steps are performed using the processing resources corresponding to the data classification service: Based on the multiple sub-label sets and the first large language model, the initial screening classification labels corresponding to the input text under the multiple sub-label sets are determined in parallel. Based on the second language model, the target classification label corresponding to the input text is determined from the initial screening classification labels corresponding to the input text under the multiple sub-label sets respectively; The target classification label is fed back to the client device.

10. An electronic device, characterized in that, include: The system includes a memory, a processor, and a communication interface; wherein the memory stores executable code, which, when executed by the processor, causes the processor to perform the data classification method as described in any one of claims 1 to 9.

11. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the data classification method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that, include: A computer program, when executed by a processor of an electronic device, causes the processor to perform the data classification method as described in any one of claims 1 to 9.

Images