Large language model training method and related method, device, equipment and storage medium

Abstract

The present disclosure provides a large language model training method and related method, device, equipment and storage medium. Including: obtaining classification knowledge text and sample classification label set; based on the classification knowledge text, the sample classification label set and the sample object attribute corresponding to the training sample, a sample classification prompt is constructed; input the sample classification prompt into the large language model, and according to the classification knowledge text and the sample object attribute in the sample classification prompt, the large language model carries out hierarchical multi-label classification processing on the training sample, so as to select the corresponding predicted hierarchical classification label for the training sample from the sample classification label set; according to the difference between the predicted hierarchical classification label corresponding to the training sample and the labeled hierarchical classification label, a target loss value is determined; the target loss value is used to train the large language model, and a trained target large language model is obtained. The method makes the large language model learn the classification knowledge corresponding to the hierarchical multi-label classification task, which can effectively improve the classification accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a method for training large language models and related methods, apparatus, devices and storage media. Background Technology

[0002] Hierarchical Multi-label Classification (HMC) not only allows instances to belong to multiple categories (multi-label classification) when classifying instances, but also considers the hierarchical relationships between these categories. These hierarchical relationships can present a tree-like organizational structure. More specific categories are located at the lower levels of the tree, while more abstract or generalized categories are located at the upper levels.

[0003] Therefore, each instance can belong to one or more labels simultaneously, and these labels may be distributed across different levels of the tree. Hierarchical multi-label classification is implemented in text-based instance classification, where instances can be assigned to multiple hierarchical categories by analyzing the text content associated with them. Hierarchical multi-label classification is widely used in text classification, bioinformatics, and recommender systems.

[0004] Currently, existing technologies involve inputting text data into a feature extraction network to extract text feature vectors, which are then used in subsequent text classification models to complete the classification task. However, this classification method, which combines various functional neural network models, suffers from poor accuracy due to variations in model architecture and training data. This is particularly problematic when dealing with hierarchical multi-label classification tasks with complex hierarchical relationships, where the model is prone to misunderstandings or prediction errors, leading to inaccurate classification results. Summary of the Invention

[0005] This disclosure provides a method for training a large language model, as well as related methods, apparatus, devices, and storage media, which can improve the accuracy of large language models in hierarchical multi-label classification.

[0006] According to one aspect of this disclosure, a method for training a large language model is provided. The method includes: acquiring classification knowledge text and a sample classification label set; constructing a sample classification prompt based on the classification knowledge text, the sample classification label set, and sample object attributes corresponding to the training samples; inputting the sample classification prompt into the large language model, whereby the large language model performs hierarchical multi-label classification processing on the training samples according to the classification knowledge text and sample object attributes in the sample classification prompt, to select corresponding predicted hierarchical classification labels for the training samples from the sample classification label set; determining a target loss value based on the difference between the predicted hierarchical classification labels corresponding to the training samples and the labeled hierarchical classification labels corresponding to the training samples; and training the large language model using the target loss value to obtain a trained target large language model.

[0007] According to another aspect of this disclosure, a classification method is provided, comprising: acquiring target classification knowledge text and a target classification tag set; constructing a target classification prompt based on the target classification knowledge text, the target classification tag set, and target object attributes corresponding to the target classification object; inputting the target classification prompt into a target large language model, wherein the target large language model performs hierarchical multi-tag classification processing on the target classification object according to the target classification knowledge text and target object attributes in the target classification prompt, so as to select a corresponding target hierarchical classification tag for the target classification object from the target classification tag set; wherein the target large language model is trained according to the above-described large language model training method.

[0008] According to another aspect of this disclosure, a training apparatus for a large language model is provided. The apparatus includes: a sample acquisition module, a prompt construction module, a sample classification module, a loss determination module, and a model training module. The sample acquisition module is used to acquire classification knowledge text and a set of sample classification labels. The prompt construction module is used to construct sample classification prompts based on the classification knowledge text, the set of sample classification labels, and the sample object attributes corresponding to the training samples. The sample classification module is used to input the sample classification prompts into the large language model, whereby the large language model performs hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the sample classification prompts, so as to select corresponding predicted hierarchical classification labels for the training samples from the set of sample classification labels. The loss determination module is used to determine a target loss value based on the difference between the predicted hierarchical classification label corresponding to the training sample and the labeled hierarchical classification label corresponding to the training sample. The model training module is used to train the large language model using the target loss value to obtain a trained target large language model.

[0009] Optionally, the sample classification module may include: a classification processing submodule, a label determination submodule, and a label marking submodule. The classification processing submodule is used to perform hierarchical multi-label classification processing based on the classification knowledge text in the sample classification prompt and the sample object attributes, selecting at least one test hierarchical classification label for the training sample from the sample classification label set. The label determination submodule is used to determine the test hierarchical classification labels belonging to the sample classification label set as the correct hierarchical classification labels corresponding to the training sample. The label marking submodule is used to perform parent class label marking processing on each correct hierarchical classification label to obtain the predicted hierarchical classification label corresponding to the training sample, which includes the parent class label.

[0010] Optionally, the classification knowledge text includes at least terminology standardization text, label explanation text, and classification standard text; the classification processing submodule can be specifically used to: perform terminology standardization processing on the sample object attributes according to the terminology standardization text to obtain standardized object attributes; perform classification range determination processing on each sample classification label in the sample classification label set based on the label explanation text to obtain the label classification range corresponding to each sample classification label; and use the classification standard text to select at least one corresponding test-level classification label from the sample classification label set for the training sample according to the standardized object attributes and label classification range.

[0011] Optionally, the sample classification module may further include a common output submodule and a direct output submodule. The common output submodule is used to jointly output the associated hierarchical classification label and the predicted hierarchical classification label when it is detected that the predicted hierarchical classification label has an associated hierarchical classification label; the direct output submodule is used to output the predicted hierarchical classification label when it is detected that the predicted hierarchical classification label does not have an associated hierarchical classification label.

[0012] Optionally, the prompt building module can be specifically used to: obtain classification task instructions and classification processing steps; perform template generation processing according to the classification task instructions and the classification processing steps to obtain a classification prompt template; and fill the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples into the classification prompt template to obtain sample classification prompts.

[0013] Optionally, the model training module may include: a supervised fine-tuning submodule, a sample selection submodule, and a reinforcement training submodule. The supervised fine-tuning submodule is used to perform supervised fine-tuning of the large language model using the target loss value, resulting in a supervised fine-tuned large language model. The sample selection submodule is used to perform hierarchical multi-label classification processing on the training samples based on the fine-tuned large language model, and to select erroneous samples based on the classification results. The erroneous samples represent training samples corresponding to hierarchical classification labels that do not match the labeled hierarchical classification labels in the classification results. The reinforcement training submodule is used to perform reinforcement training on the fine-tuned large language model using the erroneous samples, resulting in a reinforced target large language model.

[0014] Optionally, the large language model includes a feature extraction layer and a hierarchical classification layer, wherein the hierarchical classification layer is used for hierarchical multi-label classification based on the output of the feature extraction layer; the supervised fine-tuning submodule may further include a parameter fixing unit and a parameter fine-tuning unit. The parameter fixing unit is used to fix the first model parameters of the feature extraction layer corresponding to the large language model; the parameter fine-tuning unit is used to perform supervised fine-tuning of the second model parameters of the hierarchical classification layer corresponding to the large language model using the target loss value, thereby obtaining the trained supervised fine-tuned large language model.

[0015] Optionally, the hierarchical classification layer includes a preset number of sub-hierarchical classification layers, where the preset number is the number of labels in the first-level classification label set included in the sample classification label set, and each sub-hierarchical classification layer corresponds to one first-level classification label. The parameter fine-tuning unit can be specifically used to: determine the target sub-hierarchical classification layer from the preset number of sub-hierarchical classification layers based on the first-level classification label corresponding to the predicted hierarchical classification label; fix the other layer model parameters of the other sub-hierarchical classification layers besides the target sub-hierarchical classification layer in the preset number of sub-hierarchical classification layers; perform supervised fine-tuning of the target layer model parameters of the target sub-hierarchical classification layer corresponding to the large language model using the target loss value, to obtain the trained supervised fine-tuned large language model; wherein, the other layer model parameters and the target layer model parameters belong to the second model parameters corresponding to the hierarchical classification layer.

[0016] Optionally, the reinforcement training submodule may include: a prompt construction unit, an error classification unit, a scoring processing unit, and a reinforcement training unit. The prompt construction unit is used to construct prompts based on the sample attribute information corresponding to the error sample, obtaining a classification judgment prompt corresponding to the error sample; the error classification unit is used to input the classification judgment prompt into the fine-tuned large language model for hierarchical multi-label classification processing, obtaining an error-level classification label corresponding to the error sample; the scoring processing unit is used to score the error-level classification label according to a label scoring model, obtaining a classification score corresponding to the error-level classification label; the reinforcement training unit is used to perform reinforcement training on the fine-tuned large language model using the classification score, obtaining a target large language model after reinforcement training.

[0017] Optionally, the reinforcement training unit can be specifically used to: determine the reinforcement loss value based on the difference between the classification score and the preset cumulative score; perform reinforcement training on the fine-tuned large language model using the reinforcement loss value until the reinforcement training termination condition is met, and obtain the target large language model after reinforcement training.

[0018] According to another aspect of this disclosure, a classification apparatus is provided, comprising: an object acquisition module for acquiring target classification knowledge text and a target classification tag set; a prompt generation module for constructing a target classification prompt based on the target classification knowledge text, the target classification tag set, and target object attributes corresponding to the target classification object; and an object classification module for inputting the target classification prompt into a target large language model, wherein the target large language model performs hierarchical multi-tag classification processing on the target classification object according to the target classification knowledge text and target object attributes in the target classification prompt, so as to select a corresponding target hierarchical classification tag for the target classification object from the target classification tag set; wherein the target large language model is trained according to the large language model training method provided in the above embodiments.

[0019] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program, wherein the computer program, when executed by a processor, performs the above-described large language model training method and related methods.

[0020] According to another aspect of this disclosure, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, which, when invoked by the processor, executes the above-described large language model training method and related methods.

[0021] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program stored in a storage medium; a processor of a computer device reads the computer program from the storage medium and executes the computer program, causing the computer device to perform the above-described large language model training method and related methods.

[0022] While general-purpose large language models possess powerful language understanding capabilities, they are not inherently specialized models designed specifically for classification tasks. To enable large language models to perform hierarchical multi-label classification, this embodiment of the disclosure specifically introduces classification knowledge text during the training of the large language model. This allows the large language model to learn the classification knowledge required for hierarchical multi-label classification from the classification knowledge text. To this end, after obtaining a pre-set classification knowledge text and a sample classification label set, a sample classification prompt can be constructed based on the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples. Further, the sample classification prompt is input into the large language model, which then performs hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the sample classification prompt, selecting the corresponding predicted hierarchical classification label for the training samples from the sample classification label set. Since the classification knowledge text describes the classification knowledge, i.e., the classification logic, for hierarchical multi-label classification, the large language model can select an appropriate predicted hierarchical classification label for the training samples from the sample classification label set based on the learned classification logic and the analysis of the sample object attributes of the training samples. Furthermore, based on the difference between the predicted hierarchical classification labels corresponding to the training samples and the labeled hierarchical classification labels corresponding to the training samples, the target loss value is determined, and then the large language model is trained using the target loss value to obtain the trained target large language model.

[0023] Thus, by fine-tuning the large language model based on sample classification prompts, and by incorporating classification knowledge text into these prompts, clear classification knowledge for hierarchical multi-label classification is provided to the large language model during training. This allows the trained target large language model to quickly master the complex classification logic combining "hierarchical classification" and "multi-classification" in hierarchical multi-label classification tasks. Compared to classification methods that combine various functional neural network models, this embodiment allows the large language model to learn specific classification knowledge for hierarchical multi-label classification tasks. This enables the trained target large language model to accurately select appropriate hierarchical classification labels for the objects to be classified, thereby effectively improving the accuracy of hierarchical multi-label classification. Furthermore, since the classification knowledge text in the sample classification prompts can be customized according to actual application needs, this embodiment's fine-tuning method based on sample classification prompts allows the large language model to more flexibly handle hierarchical multi-label classification tasks with varying requirements, thereby improving the generalization ability of the trained target large language model across different hierarchical multi-label classification tasks. Furthermore, the embodiments of this disclosure use sample classification prompts to fine-tune the model training of the large language model. Unlike traditional neural network model training, which requires iterative training based on a large number of samples to learn classification rules from scratch, this method uses predefined classification knowledge text to enable the large language model to gain a deep understanding of hierarchical multi-label classification tasks in a short time. This effectively reduces the computational resources required for model training and improves model training efficiency.

[0024] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This diagram illustrates a system architecture provided by an embodiment of the present disclosure.

[0027] Figure 2 This diagram illustrates a system deployment according to an embodiment of the present disclosure.

[0028] Figure 3This illustration shows an application scenario diagram of an application recommendation scenario provided by an embodiment of this disclosure.

[0029] Figure 4 This diagram illustrates an application scenario of a bioinformatics annotation provided by an embodiment of the present disclosure.

[0030] Figure 5 This diagram illustrates a flowchart of a large language model training method provided in an embodiment of this disclosure.

[0031] Figure 6 This diagram illustrates a hierarchical classification label provided in an embodiment of the present disclosure.

[0032] Figure 7 This diagram illustrates a classification prompt template provided in an embodiment of the present disclosure.

[0033] Figure 8 This diagram illustrates a sample classification prompt provided in an embodiment of the present disclosure.

[0034] Figure 9 This diagram illustrates another sample classification prompt provided in an embodiment of the present disclosure.

[0035] Figure 10 This diagram illustrates a flowchart of another large language model training method provided in an embodiment of this disclosure.

[0036] Figure 11 A flowchart illustrating a large language model training method provided in an embodiment of this disclosure is shown.

[0037] Figure 12 This diagram illustrates a model architecture diagram of a large language model provided in an embodiment of this disclosure.

[0038] Figure 13 This diagram illustrates the structure of a hierarchical classification layer provided in an embodiment of the present disclosure.

[0039] Figure 14 This illustration shows a label diagram of a hierarchical multi-label classification application provided in an embodiment of the present disclosure.

[0040] Figure 15 This diagram illustrates a classification method provided in an embodiment of the present disclosure.

[0041] Figure 16 This is a block diagram of a large language model training device provided in an embodiment of this disclosure.

[0042] Figure 17 This is a block diagram of a sorting device provided in an embodiment of the present disclosure.

[0043] Figure 18This is a block diagram of a computer device provided in an embodiment of the present disclosure.

[0044] Figure 19 This is a block diagram of a computer-readable storage medium provided in an embodiment of this disclosure. Detailed Implementation

[0045] To enable those skilled in the art to better understand the solutions disclosed herein, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. The implementation methods described with reference to the accompanying drawings are exemplary and are only used to explain this disclosure, and should not be construed as limiting this disclosure. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0046] In some processes described in the specification, claims, and the foregoing drawings, multiple steps are included in a specific order. However, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not themselves represent any execution order. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order, sequence, or quantity.

[0047] It is worth noting that in the specific embodiments of this disclosure, data related to target object attributes corresponding to the target classification object is involved. When the above embodiments of this disclosure are applied to specific products or technologies, permission or consent from the target object is required, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. For example, when an embodiment of this disclosure needs to obtain target object attributes, separate permission or consent for the target object attributes can be obtained through pop-up windows or redirection to a confirmation page. After obtaining separate permission or consent, the necessary related data for enabling the embodiments of this disclosure to operate normally can then be obtained.

[0048] The Large Language Model (LLM) disclosed herein is a language model composed of an artificial neural network with many parameters (typically billions or more weights), trained on a large amount of unlabeled text using self-supervised or semi-supervised learning. LLMs are general-purpose models that perform well on a wide range of tasks, rather than being trained for a specific task. Pre-trained on massive amounts of text data, LLMs excel in understanding grammatical structures, semantic relationships, and reasoning about complex problems. LLMs include Generative Pre-trained Transformer (GPT) models, such as GPT-3 and GPT-4, which are widely used in tasks such as translation, dialogue generation, classification and clustering, and question answering.

[0049] Hierarchical multi-label classification can be seen as a special case of multi-label classification, where different labels are not independent but hierarchically divided, forming a tree-like structure between categories. If an object belongs to a sub-label, it naturally belongs to the corresponding parent label. Furthermore, each object can belong to multiple classes, thus each object can be associated with multiple different paths in the class hierarchy (i.e., the tree structure). Hierarchical multi-label classification is widely used in text classification, bioinformatics, and recommender systems, for example, assigning articles to hierarchical topic categories based on content or classifying gene function annotations hierarchically.

[0050] Currently, hierarchical multi-label classification based on text information can assign instances to multiple categories, which may also have a hierarchical structure, by utilizing the textual information of the instances to be classified. This classification task has wide applications in many fields, including news classification: classifying news articles according to topics or categories, such as economics, sports, etc., while considering more specific subcategories; product review analysis: analyzing product reviews on e-commerce websites, tagging product characteristics, user experience, purchase tendencies, etc., involved in the reviews; in addition, it also includes legal document classification and application software (APP) classification, etc.

[0051] For example, software classification based on descriptive text is a task that categorizes software according to textual data such as descriptions of its functions. This classification method can help organize and manage a large number of software resources, making it easier for users to find software that meets their needs. For instance, software can be classified into different categories such as office software, entertainment software, development tools, and security software, and each category can further include multi-level subcategories. For example, entertainment software can include game software and social software, and game software can include puzzle games and strategy games.

[0052] Due to the widespread application of large language models in natural language processing, related technologies have begun to utilize them for text classification tasks. For example, a collected text dataset is input into a large language model, which generates text vector representations through pre-training. These text vector representations are then used in subsequent text classification models to complete the text classification. Alternatively, the text to be classified is directly input into the large language model, and instructions are given to the model to perform text classification. However, these classification methods have shown poor performance. In particular, they exhibit limitations in hierarchical multi-label classification, which requires semantic understanding of complex hierarchical structures, and the accuracy of the classification results is often unsatisfactory.

[0053] To address the aforementioned problems, embodiments of this disclosure propose a large language model training method and related methods. The system implementation environment and related application scenarios of the methods involved in this disclosure are described below.

[0054] Please see Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The large language model training method and related methods provided in this disclosure can be applied to system 100. The data acquisition device 110 is used to acquire a large number of training samples, each training sample including corresponding sample object attributes and label hierarchical classification labels. The data acquisition device 110 can store the training samples in database 120 for subsequent model training. The training device 130 can train the large language model based on the training samples in database 120 to obtain the trained target large language model, i.e., target model 101. The model training process can include supervised fine-tuning and reinforcement learning based on human feedback. Target model 101 can be a deep neural network composed of one or more neural networks. For example, the target large language model 101 can be a pre-trained model such as GPT, which is not limited here.

[0055] The execution device 140 can deploy the target model 101 trained by the training device 130. During the computation and other related processing performed by the processing module 141 of the execution device 140, the execution device 140 can call data, programs, etc. in the data storage system 142 for corresponding computational processing, and store the processing results and instructions obtained from the computational processing into the data storage system 142. Data transmission between the execution device 140 and the user device 150 can be performed through an input / output (I / O) interface.

[0056] For example, when the execution device 140 receives the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object sent by the user device 150, it calls the processing module 141 to construct a target classification prompt based on the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object, and inputs the target classification prompt into the target model 101. The target model 101 generates the target level classification tag corresponding to the target classification object and returns the target level classification tag to the user device 150.

[0057] The aforementioned training device 130, execution device 140, and user device 150 can be computer devices such as servers or terminals. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), blockchain, and big data and artificial intelligence platforms. The terminal can be a laptop, tablet, desktop computer, etc.

[0058] It should be noted that, Figure 1 This is merely a schematic diagram of a system architecture provided by an embodiment of this disclosure. The system architecture described in this disclosure is for the purpose of more clearly illustrating the technical solutions of this disclosure and does not constitute a limitation on the technical solutions provided by this disclosure. For example, Figure 1 The execution device 140 can be used to deploy the target model 101 trained by the training device 130. In other cases, the user device 150 can also directly deploy the target model 101 trained by the training device 130 and perform hierarchical multi-label classification processing locally.

[0059] Please see Figure 2 , Figure 2 A schematic diagram of a system deployment is shown. Exemplarily, the large language model training method and related methods provided in this disclosure can also be deployed on, for example, […]. Figure 2 The system shown is a C / S (Client / Server) architecture system. This system may include servers, gateways, the internet, and computer equipment.

[0060] A server is a computer system that provides certain services to the server-side. Compared to ordinary computer equipment, servers have higher requirements in terms of stability, security, and performance. Figure 2As shown, server 210 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines).

[0061] A gateway is also known as an internetwork connector or protocol converter. For example... Figure 2 As shown, gateway 220 achieves network interconnection at the transport layer via Internet 230, and is a computer system or device that acts as a translator. Between two systems using different communication protocols, data formats, or languages, or even completely different architectures, a gateway is a translator. At the same time, a gateway can also provide filtering and security functions.

[0062] Computer devices can send requests to servers, requesting access to or use resources, data, or services on the server. For example... Figure 2 As shown, the computer device can include various forms such as a tablet computer 240a, a desktop computer 240b, a laptop computer 240c, and a personal digital assistant (not identified). Furthermore, it can be a single computer device or a collection of multiple computer devices. The computer device is capable of communicating with the Internet 230 via wired or wireless means. Figure 2 As shown, tablet computer 240a can communicate with the Internet 230 via wireless router 250. Similarly, messages sent by desktop computer 240b to server 210 can be transmitted to the corresponding server 210 via gateway 220. Messages sent by server 210 to computer devices can also be transmitted to the corresponding laptop computer 240c via gateway 220.

[0063] In some embodiments, the classification method provided in this disclosure can be implemented partly on a server and partly on a computer device. For example, the computer device sends the target object attributes corresponding to the acquired target classification object to the server. The server can construct a target classification prompt based on the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object. Further, the server can use a trained target large language model to perform hierarchical multi-label classification processing on the target classification object according to the target classification knowledge text and target object attributes in the target classification prompt, so as to select the corresponding target hierarchical classification tag for the target classification object from the target classification tag set, and return the target hierarchical classification tag to the computer device.

[0064] It should be noted that, Figure 2This is merely a schematic diagram of a system deployment provided by an embodiment of this disclosure. The system deployment scheme described in this embodiment is only for the purpose of more clearly illustrating the technical solutions of this disclosure and does not constitute a limitation on the technical solutions provided by this disclosure. For example, "computer device" can refer to one of multiple devices; this embodiment only uses one as an example. Figure 2 The three computer devices shown (tablet computer 240a, desktop computer 240b, and laptop computer 240c) are used as examples. Those skilled in the art will recognize that, as system deployment schemes evolve, the technical solutions provided in the embodiments of this invention are equally applicable to similar technical problems.

[0065] The large language model training method and related methods disclosed herein can be applied in various scenarios, including... Figure 3 The application recommendation scenarios shown and Figure 4 The example shown is a bioinformatics annotation scenario.

[0066] (1) Application recommendation scenarios.

[0067] App recommendation is a common technique used by app stores to recommend applications that users may be interested in. App recommendation requires systematic research and evaluation of applications to understand their characteristics, functions, and user experience. App categorization is a crucial step in the analysis and statistics process, especially hierarchical multi-tag categorization to improve accuracy. By performing hierarchical multi-tag categorization of applications in the initial stage of app recommendation, the app tags derived from the categorization can be matched with user preferences to complete the app recommendation process.

[0068] The classification method provided in this disclosure can be used to develop app store apps. For example... Figure 3 As shown, the app store can include a server 311 deployed on a cloud server 310 and a client 320 deployed on a mobile phone 330. The server 311 is primarily responsible for performing complex calculations and logical processing, processing data requested by the client 320, such as returning the application installation package for application download requests sent by the client 320. The client 320 primarily provides the interactive interface. When a user opens the app store interface 321 on the client 320, the app store interface 321 can display multiple applications recommended for download, such as the applications AAAA, BBBB, and CCCC displayed in the "Today's Recommendations" section of the app store interface 321 shown in Figure 3.

[0069] Specifically, server 311 obtains software data such as the software name and description of the applications used by the user daily through client 320. Server 311 can then use the trained target large language model to perform hierarchical multi-label classification of each application based on the software data, thereby obtaining the APP tags corresponding to each application. The target large language model is trained using the large language model training method provided in this embodiment. Further, the APP tags corresponding to each application are matched with the obtained user preferences. For example, users tend to prefer fitness-related apps, especially those with diet management functions. Based on the user's preferences, the APP tags predicted by the trained target large language model are matched with the user's interest tags.

[0070] For example, if the app AAAA's APP tags are {<Health>, <Health-Fitness>, <Diet>, <Diet-Diet Management>}, then AAAA will be considered a recommended application. Furthermore, based on other user preferences, other recommended applications such as BBBB and CCCC can be generated. Further, the server 311 can send brief descriptions of the recommended applications AAAA, BBBB, and CCCC to the client 320, which then displays these applications to the user in the app store interface 321.

[0071] (2) Bioinformatics annotation scenario.

[0072] In bioinformatics, hierarchical multi-label classification is widely used for annotating gene or protein functions. Gene or protein function annotation is a crucial bridge connecting genomics, proteomics, and biological significance, and is a key step in life science research, with significant value for both basic research and clinical translation. For example, proteins with known functions and their annotations are obtained from existing databases (such as UniProt or GOA) as training data. These proteins with known functions correspond to information about their structure, physicochemical properties, or other proteins. Figure 4 As shown, the large language model training method based on embodiments of this disclosure can train a target large language model for annotating protein sequences, and develop a protein sequence annotation system based on this target large language model for annotating protein sequences. Researchers can edit or paste the protein sequence to be annotated in the editing bar 421 of the client interface 420, or click button 422 to upload protein sequences to be annotated in batches. After completing the sequence input, researchers can click button 424 to start annotation.

[0073] Furthermore, client 420 generates an annotation request and sends it to the cloud server. The server performs sequence classification on the protein sequence to be annotated, obtaining classification results. For example, based on the amino acid sequence of the protein P12345 to be annotated, the classification results are {<biological process>, <biological process - metabolic process>, <biological process - signal transduction>, <molecular function>, <molecular function - catalytic activity>, <molecular function - ATP binding>, <cell composition - cell nucleus>}. The cloud server then returns the classification results as the annotation results for the corresponding amino acid sequence of protein P12345 to client 420, which displays the annotation results in the display bar 423 of the interface. Protein functions can typically be hierarchically classified into three main categories: molecular function, biological process, and cell composition. These categories are organized in a tree structure, with higher-level labels representing generalized functions and lower-level labels representing specific functions. Therefore, hierarchical multi-label classification can be used for protein sequence annotation.

[0074] It should be noted that, Figure 3 and Figure 4 These are merely illustrative diagrams illustrating two application scenarios provided by embodiments of this disclosure. The application scenarios described in these embodiments are only for clearer explanation of the technical solutions provided by these embodiments and do not constitute a limitation on the technical solutions provided by these embodiments. The large language model training method and related methods provided by these embodiments can also be applied to other application scenarios requiring hierarchical multi-label classification. As those skilled in the art will understand, with the evolution of application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0075] According to one embodiment of this disclosure, a method for training a large language model is provided. This method can be executed by a computer device (e.g., a desktop computer, laptop computer, or other terminal device with certain processing capabilities), which has at least storage, computation, and communication functions. Figure 5 A flowchart illustrating the large language model training method provided in this embodiment is shown. Figure 5 As shown, the training method for this large language model can specifically include:

[0076] Step 110: Obtain the classification knowledge text and sample classification label set.

[0077] Considering that the general training process of neural network models often requires multiple parameter tunings and has a long development cycle, related technologies have begun to utilize large language models for classification tasks. However, although large language models have powerful language understanding capabilities, they are essentially statistical generative models rather than specialized models designed specifically for text classification tasks. In particular, directly applying large language models to hierarchical multi-label classification tasks can easily lead to inaccurate classification. Therefore, this disclosure proposes a method for fine-tuning large language models using classification knowledge text to improve their classification accuracy in hierarchical multi-label classification tasks.

[0078] This is because in applications using deep neural network models such as BERT for classification, the training method mainly relies on changing the model structure or manually constructing classification features to achieve the preset classification effect. However, this training method is not only time-consuming but also requires a large number of training samples. Therefore, this embodiment of the present disclosure fine-tunes the large language model based on prompts, embedding classification knowledge text into sample classification prompts. This provides the large language model with clear classification knowledge for hierarchical multi-label classification during model training, enabling the target large language model after training to quickly learn the classification logic used to perform hierarchical multi-label classification tasks.

[0079] In this embodiment, the classification knowledge text represents text describing hierarchical multi-label classification logic. The large language model can quickly master the ability to perform hierarchical multi-label classification by learning from the classification knowledge text with a small number of training samples. The classification logic in the classification knowledge text can be set according to actual application needs. For example, in the medical field, classification knowledge text containing medical terminology can be set; in the software development field, classification knowledge text involving computer science terminology can be set. To enable the large language model to accurately perform hierarchical multi-label classification tasks, the classification knowledge text can include at least terminology standardization text, label explanation text, and classification standard text.

[0080] Terminology standardization text is used to instruct large language models to standardize the diverse information acquired in classification tasks. For example, because the feature attributes corresponding to training samples are described using English abbreviations, large language models cannot accurately grasp the specific meaning of these attributes, thus hindering accurate classification. For instance, terminology standardization text can at least include the mapping relationship between Chinese and English information. For example, in the gaming domain, the English abbreviations RPG for role-playing games and FPS for first-person shooter games. Through terminology standardization text, large language models can learn how to accurately understand the feature attributes corresponding to training samples, thereby improving classification accuracy.

[0081] The label explanation text is used to instruct the large language model to parse the classification range of the sample hierarchical classification labels in the sample classification label set. This is because in hierarchical multi-label classification tasks, the classification dimension between sample hierarchical classification labels includes both category and hierarchy. The large language model cannot reliably distinguish the fine-grained differences between sample hierarchical classification labels corresponding to different categories and levels, thus affecting the accuracy of the classification results of hierarchical multi-label classification tasks. For example, the label explanation text can include the classification range corresponding to each sample hierarchical classification label in the sample classification label set. For instance, the classification range corresponding to <Finance-Lottery> includes: welfare lottery and sports lottery; the classification range corresponding to <Finance-Lottery-Welfare Lottery> includes: welfare lottery, double color ball, seven-color lottery, happy 8, scratch-off lottery, etc.; the classification range corresponding to <Games-Chess and Card Games> includes: chess, mahjong, and Dou Dizhu, etc.

[0082] Adding explanatory text to labels clarifies the classification scope of hierarchical labels for samples, reducing misclassifications caused by unclear semantic understanding of labels in large language models. This effectively lowers the misjudgment rate of labels by large language models. Furthermore, the explanatory text guides large language models to select deeper subclass labels within the hierarchical classification system, significantly improving classification accuracy. In addition, the explanatory text transforms the complex label system into explicit knowledge, reducing the difficulty for large language models to infer from scratch and thus lowering the learning threshold.

[0083] Classification criteria text instructs large language models to select matching labels for objects to be classified from the set of labels based on the classification criteria or rules described in the text. Considering that in scenarios where labels are manually annotated, annotators have a basis for their annotations, without providing classification criteria text for large language models, the lack of clear classification criteria may lead to classification errors or inconsistencies, especially in hierarchical multi-label classification where complex hierarchical and category relationships exist between labels, making the large language model more prone to classification ambiguity. Furthermore, since large language models rely on the relevant attribute information of the objects to be classified for label understanding and selection reasoning, in complex hierarchical multi-label classification scenarios, the lack of classification criteria makes it difficult for the large language model to correctly infer the corresponding labels for the objects to be classified, thus affecting classification performance.

[0084] For example, the classification criteria text can include classification rules for various classification scenarios. For instance, if the attribute information of the object to be classified mentions "you will play as xxx," which means role-playing game (RPG), then it should be tagged with <Game-Role-Playing>. If the game's name contains the phrase "IO," or the game description indicates that the game belongs to the IO category, then it can be tagged with <Game-IO>. If a game is tagged with <'Game-Legend'>, <'Game-Fantasy'>, etc., it should also be tagged with <'Game-RPG'>, <'Game-ARPG'>, or <'Game-MMORPG'>.

[0085] In this way, during the model training process, the large language model is provided with clear classification knowledge for hierarchical multi-label classification. This enables the large language model to quickly master the complex classification logic that combines "hierarchical classification" and "multi-classification" in the hierarchical multi-label classification task, so as to select appropriate hierarchical classification labels for training samples in the future and improve the accuracy of hierarchical multi-label classification.

[0086] A sample classification label set refers to a collection used to store the hierarchical labels of all categories of samples. In classification tasks, a label is an identifier used to identify and distinguish different categories; it is typically used to describe the categories of data. The hierarchical classification labels in the sample classification label set are the labels used to divide training samples into different levels and categories. Each hierarchical classification label represents a specific category and level, where the level indicates the hierarchical relationship between labels, reflecting the subordinate or inclusive relationship between labels.

[0087] Please see Figure 6 The diagram shown illustrates the hierarchical classification labels. Figure 6 The hierarchical classification labels consist of three levels: the first level, the second level, and the third level, each corresponding to different categories. The category at the level above a category is its parent category, and the category at the level below a category is its subcategory. For example, category 1 is the parent category of categories 11 and 12, meaning the hierarchical classification labels for category 1 include the hierarchical classification labels for categories 11 and 12, respectively. Category 111 is a subcategory of category 11, meaning the hierarchical classification labels for category 11 belong to the hierarchical classification labels for category 11.

[0088] For example, the hierarchical classification label <Finance> corresponding to category 1 includes the hierarchical classification labels <Finance-Banking> and <Finance-Bookkeeping> corresponding to categories 11 and 12, respectively. The hierarchical classification label <Finance-Banking-Rural Bank> corresponding to category 11 belongs to the hierarchical classification label <Finance-Banking> corresponding to category 11.

[0089] As one implementation method, when the large language model training method described above is executed on a server, the server can obtain pre-stored classification knowledge text and sample classification label sets from a local database or cloud database through a data call interface. The classification knowledge text and sample classification label sets can be preset and stored in a local database or cloud database associated with the server by the model trainer.

[0090] Step 120: Construct sample classification prompts based on classification knowledge text, sample classification label set, and sample object attributes corresponding to training samples.

[0091] This disclosure proposes to fine-tune a large language model using classification knowledge text. In order to enable the large language model to clearly define the hierarchical multi-label classification task, classification knowledge text can be used to construct sample classification prompts as input to the large language model, thereby enabling the large language model to focus on generating outputs related to the hierarchical multi-label classification task based on the sample classification prompts.

[0092] In this embodiment, the training samples are the sample objects to be classified. These sample objects include, but are not limited to, news content, application software, and pharmaceuticals—object instances that can be described in words. In different classification tasks, the object instances can be diverse, depending on the actual task and application scenario, and are not limited here. For example, for preference classification tasks in a recommendation system, the object instance could be a user's product review; for terminal application security detection scenarios, the object instance could be the application software installed on the terminal.

[0093] Each training sample is associated with a corresponding sample object attribute and a hierarchical classification label. The hierarchical classification label refers to the hierarchical classification label assigned by humans when performing hierarchical multi-label classification on the training sample. The hierarchical classification label serves as the ground truth used during the training of the large language model, guiding the model to learn the target output. The large language model adjusts its parameters to optimize the performance of hierarchical multi-label classification by comparing its prediction results with the hierarchical classification labels (calculating the loss). Sample object attributes are textual descriptions of the training samples; they describe the characteristic attributes of the training samples and represent a way to express the semantic understanding of the training samples.

[0094] For example, training samples could be application software to be classified. Correspondingly, sample object attributes could include: App Name: a name reflecting the application software's function or theme; App Description: a textual description of the application software's functions, features, and advantages; Keywords: descriptive words or phrases used for searching and discovering the application software; Update Log: update content and improvement notes released when the application software is updated, which may include information such as fixed code errors or defects, new features, and performance improvements, helping users understand the latest changes to the application software, etc. As another example, training samples could also be drugs to be classified. Correspondingly, sample object attributes could be the drug's instruction manual, providing detailed information about the drug's use, including drug name and specifications: the generic and brand names of the drug, as well as information such as the drug's specifications and dosage form; ingredients and effects: a description of the drug's main components and mechanism of action, and the drug's effects on the body; indications: a description of the diseases or symptoms the drug is suitable for treating, etc.

[0095] In some embodiments, a classification prompt template for hierarchical multi-label classification is obtained, and then the classification knowledge text, sample classification label set, and sample object attributes corresponding to the training samples are filled into the classification prompt template to obtain a sample classification prompt. Specifically, this may include:

[0096] (1) Obtain classification task instructions and classification processing steps.

[0097] A classification task instruction is a command used to instruct a large language model to perform a hierarchical multi-label classification task. The classification task instruction explicitly describes the expected execution goal and output method to the large language model. In this embodiment, the classification task instruction needs to be clear, concise, and explicit. Clarity means that the classification task instruction should clearly state the task requirements of the hierarchical multi-label classification task, avoiding ambiguity. Conciseness means that the classification task instruction should be as brief as possible but sufficiently detailed, without containing redundant information. Explicitness means that the classification task instruction should clearly define the goal of the large language model in performing the hierarchical multi-label classification task.

[0098] The classification processing step refers to the specific steps used to instruct the large language model to complete the hierarchical multi-label classification task. Optionally, the classification processing step may include a label classification processing step, a label correctness determination step, and a label hierarchy constraint step. Specifically, the label classification processing step instructs the large language model to select hierarchical classification labels for the training samples from the sample classification label set. The label correctness determination step determines whether the selected hierarchical classification labels belong to the sample classification label set. The label hierarchy constraint step matches the selected hierarchical classification labels belonging to subclasses with the corresponding hierarchical classification labels of their parent classes. Optionally, the classification processing step may also include a label output step, instructing the large language model to output the predicted labels according to an output format.

[0099] As one implementation method, the classification processing steps can be manually set and stored by professional technicians. The manual setting only needs to include at least the aforementioned label classification processing steps, label correctness determination steps, and label hierarchy constraint steps; the specific content description format is not limited. Classification task instructions can also be manually set and stored by professional technicians. The manual setting only needs to meet the aforementioned clarity, conciseness, and explicitness; the specific content description format is not limited. For example, the classification task instructions can instruct the large language model to act as an expert, selecting corresponding prediction hierarchy classification labels from the sample classification label set for the training samples.

[0100] For example, a classification task instruction might read, "Suppose you are an NLP expert working on a hierarchical multi-label classification problem. Please label and classify the tags for the app based on the input [app_name], [package_name], and [app_desc]. The labels are stored in [label_list], and the label format can be <first-level label - second-level label - third-level label>, where second-level labels are sub-labels of first-level labels, and third-level labels are sub-labels of second-level labels. Second-level and third-level labels may not exist." In the example above, [app_name], [package_name], and [app_desc] are the attributes of the sample objects corresponding to the training samples, and [label_list] is the set of sample classification labels.

[0101] (2) Generate a template according to the classification task instructions and classification processing steps to obtain a classification prompt template.

[0102] A category hint template is a predefined text framework used to generate structured hints. By dynamically inserting variables or parameters, standardized hints suitable for hierarchical, multi-label classification tasks can be quickly generated. Using category hint templates improves the reusability, flexibility, and consistency of the hints.

[0103] As one implementation method, when obtaining classification task instructions and classification processing steps, the classification task instructions and classification processing steps are used to generate a classification prompt template through a combination of fixed structure and dynamic variables. Figure 7 This shows a schematic diagram of a category suggestion template. For example... Figure 7 As shown, the fixed structure of this classification prompt template includes classification task instructions, classification processing steps, classification knowledge text to be filled, and sample object attributes (app_name, package_name, and app_desc) and sample classification label set (label_list) to be filled. The sample object attributes and sample classification label set are set as dynamic variables in the form of {}.

[0104] (3) Fill the classification knowledge text, sample classification label set and sample object attributes corresponding to the training samples into the classification prompt template to obtain the sample classification prompt.

[0105] One implementation method involves assigning specific sample object attributes and sample classification label sets to corresponding dynamic variables to complete the construction of sample classification prompts. Please refer to [link / reference]. Figure 8 This is a schematic diagram of a sample classification prompt. For example... Figure 8 As shown, the dynamic variables corresponding to the classification knowledge text, sample object attributes, and sample classification label set have been assigned specific values.

[0106] It should be noted that, Figure 8 The sample classification prompts shown are merely examples. To more clearly illustrate the technical solutions of the embodiments of this application, and considering space limitations, only partial information is provided. The specific content and fixed structure of the sample classification prompts are set according to the needs of actual application scenarios and are not limited here. For example, Figure 9 Another sample classification prompt is shown, in which the classification knowledge text is displayed in the form of reasoning in the classification processing steps (steps 6-9), while the sample object attributes and sample classification label set are set in the preamble of the sample classification prompt.

[0107] Step 130: Input the sample classification prompts into the large language model. The large language model performs hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the sample classification prompts, so as to select the corresponding prediction hierarchical classification labels for the training samples from the sample classification label set.

[0108] In some embodiments, when a large language model receives a sample classification prompt, it analyzes the classification task instructions in the prompt to obtain the objective of the hierarchical multi-label classification task. Then, by executing specific classification processing steps, it utilizes classification knowledge text and sample object attributes to perform hierarchical multi-label classification processing on the training samples. Finally, it selects the corresponding predicted hierarchical classification label for the training samples from the sample classification label set. The specific classification processing steps of the large language model may include:

[0109] (1) Perform hierarchical multi-label classification based on the classification knowledge text and sample object attributes in the sample classification prompts. Select at least one hierarchical classification label to be tested from the sample classification label set for the training samples.

[0110] As one implementation method, the classification knowledge text may include at least terminology standardization text, label explanation text, and classification standard text. The large language model can learn hierarchical multi-label classification knowledge from the terminology standardization text, label explanation text, and classification standard text, perform hierarchical multi-label classification processing on training samples according to the sample object attributes, and then select the corresponding predicted hierarchical classification label for the training samples from the sample classification label set. Specifically, this may include:

[0111] (1.1) Based on the terminology standardization text, perform terminology standardization on the attributes of the sample objects to obtain standardized object attributes;

[0112] (1.2) Based on the label explanation text, the classification range of each sample classification label in the sample classification label set is determined to obtain the label classification range corresponding to each sample classification label;

[0113] (1.3) Using the classification standard text, based on the standardized object attributes and label classification range, select at least one corresponding test level classification label from the sample classification label set for the training sample.

[0114] As an example, taking a training sample of a game application, the large language model, based on the mapping relationship between Chinese and English information in the standardized text, can standardize "RPG game" in the game description of the application to "role-playing game," thus accurately understanding that the application is a role-playing game. Furthermore, after determining the classification range of the sample hierarchical classification label <game-role-playing game> in the sample classification label set based on the label explanation text, the large language model learns that the classification range corresponding to the sample hierarchical classification label <game-role-playing game> includes role-playing games, action role-playing games, etc. Further, based on the classification rules in the classification standard text, the large language model can determine that the game application belongs to the role-playing game category and select the sample hierarchical classification label <game-role-playing game> from the sample classification label set as the test hierarchical classification label corresponding to the game application.

[0115] (2) The test hierarchical classification labels that belong to the sample classification label set are determined as the correct hierarchical classification labels corresponding to the training samples.

[0116] Considering the generation mechanism of large language models and their limitations in understanding hierarchical multi-label classification, insufficient understanding of this specific classification task may cause the model's output of test-level classification labels to exceed the range of the sample classification label set. This is because the output of large language models is based on probability distributions rather than strict rule matching. When generating test-level classification labels, the model predicts the most likely output based on classification knowledge text and sample object attributes. However, this prediction is not entirely limited to the sample classification label set because the sample classification label set is only passed to the model through sample classification prompts, rather than using constraint mechanisms (such as regular expressions). The model may not always correctly remember or follow this label set, causing information in the prompts to be weakened or ignored during the generation process.

[0117] In addition, the "hallucination problem" of large language models may also be one of the reasons why the output exceeds the sample classification label set. When generating output, large language models focus more on the relevance and fluency of language patterns than on strict facts or constraints. For example, when a large language model attempts to select from the sample classification label set, it may mistakenly consider certain words not in the set as relevant or reasonable choices, thus generating these "fictitious" labels that do not belong to the sample classification label set. To address this, embodiments of this disclosure propose that after the large language model outputs the test-level classification labels corresponding to the training samples, it should reflect on whether its output is reasonable, that is, verify whether the test-level classification labels belong to the sample classification label set. This prompts the large language model to reflect and improve the reliability of its output.

[0118] As one implementation method, a verification prompt can be constructed based on the classification label of the level to be tested. The task instruction of this verification prompt is to instruct the large language model to perform a correctness check on the classification label of the level to be tested generated in the above steps, determining whether the classification label exists in the sample classification label set. If it does not exist, then a new hierarchical multi-label classification process is performed based on the classification knowledge text and sample object attributes in the sample classification prompt, selecting a new classification label for the training samples from the sample classification label set. If it exists, then there is no need to reselect the new classification label. Optionally, the task instruction of the verification prompt can instruct the large language model to determine whether the classification label of the level to be tested belongs to the given sample classification label set. The specific structure and content of the verification prompt are not limited here.

[0119] (3) For each correct hierarchical classification label, perform parent class label labeling to obtain the predicted hierarchical classification label containing the parent class label for the training sample.

[0120] Considering the unique hierarchical relationship between classification labels in hierarchical multi-label classification scenarios, to ensure the consistency of hierarchical logic in the classification results—that is, if a child label is selected while the parent label is ignored, the actual output will conflict with the hierarchical structure, violate the rules of the hierarchical classification system, and produce semantic errors or misunderstandings—this embodiment of the disclosure, after obtaining the correct hierarchical classification labels, also allows the large language model to constrain the hierarchical structure.

[0121] As one implementation method, the large language model can detect whether each correct hierarchical classification label is a subclass label. If a correct hierarchical classification label is a subclass label, then the parent class label corresponding to that correct hierarchical classification label will also be obtained from the sample classification label set. For example, if the label <Education-Quality Education-Primary and Secondary Education> is selected, the parent class labels corresponding to that label will also be selected simultaneously: <Education-Quality Education> and <Education>.

[0122] By using parent class labeling, on the one hand, the logical consistency of the label hierarchy in hierarchical multi-label classification scenarios is maintained; on the other hand, the output of the large language model has a complete hierarchical structure path, which can constrain the selection range of subclass labels. For example, if <finance-bank> is selected, it must meet the constraint of <finance>. In this way, the possibility of the large language model generating incorrect labels will be limited to a specific range, effectively reducing the probability of outputting labels that are irrelevant to the hierarchy, thereby improving the accuracy of hierarchical multi-label classification.

[0123] Considering that hierarchical multi-label classification tags not only have explicit relationships (i.e., parent-child hierarchical relationships) but also implicit relationships (i.e., multi-category relationships), for example, although <Game-Legend> and <Game-RPG> do not have a direct hierarchical relationship, these two tags are often jointly labeled for the same game based on the game's theme and gameplay. Therefore, this disclosure proposes a method for obtaining associated hierarchical classification tags based on such implicit relationships to improve the comprehensiveness and accuracy of hierarchical multi-label classification.

[0124] In some embodiments, by setting prompts, the large language model can detect that the predicted hierarchical classification label of the training sample has an associated hierarchical classification label. When it is detected that the predicted hierarchical classification label has an associated hierarchical classification label, both the associated hierarchical classification label and the predicted hierarchical classification label are output; when it is detected that the predicted hierarchical classification label does not have an associated hierarchical classification label, the predicted hierarchical classification label is output.

[0125] For example, when the predicted hierarchical classification labels are detected as <game-legend> and <game-fantasy>, the associated hierarchical classification labels <game-RPG> or <game-ARPG> can be obtained, and then <game-legend>, <game-fantasy> and <game-RPG> can be output together.

[0126] As an example, the output format of the classification labels can be specified. For instance, the classification processing steps in the classification prompt can be detailed: output the labels selected for the training samples as strings, with multiple labels concatenated using \t, and the labels have no order. And, note! Only return the final result label string; do not return the inference process to avoid parsing failure. Returning anything other than the result label string will result in a penalty. This ensures a consistent and accurate output format.

[0127] Step 140: Determine the target loss value based on the difference between the predicted hierarchical classification label corresponding to the training sample and the labeled hierarchical classification label corresponding to the training sample.

[0128] In deep learning, the loss function measures the gap or error between the model's prediction and the actual target, guiding the update of model parameters to minimize the difference between the predicted and true values. By optimizing the loss function, the model can continuously learn and improve its performance, making its predictions more accurate.

[0129] In the training process of a large language model, the optimization objective of this disclosure is to minimize the difference between the predicted hierarchical classification labels and the labeled hierarchical classification labels of the training samples. Therefore, a target loss value can be determined based on the difference between the predicted and labeled hierarchical classification labels of the training samples. The large language model is then trained based on this target loss value.

[0130] Considering the special nature of the labels in this embodiment, namely multi-level classification labels, it is necessary to compare the classification labels of each training sample at each level to determine whether they have the same label at the corresponding level. Therefore, this multi-classification task can be replaced by multiple binary classification tasks with hierarchical classification labels as the classification standard, so that the target loss value can be calculated using the cross-entropy loss function.

[0131] Step 150: Train the large language model using the target loss value to obtain the trained target large language model.

[0132] As one implementation method, supervised fine-tuning (SFT) is performed on a large language model using a target loss value. During the supervised fine-tuning process, it is determined whether the target loss value meets the training termination condition. The training termination condition can be that the target loss value is less than a preset value, the target loss value no longer changes, or the number of supervised fine-tuning iterations reaches a preset number. When the target loss value does not meet the training termination condition, the model parameters of the large language model are adjusted until the target loss value meets the training termination condition. When the target loss value meets the training termination condition, the supervised fine-tuning of the large language model is stopped, resulting in the supervised fine-tuned target large language model.

[0133] As another implementation method, supervised fine-tuning of the large language model is performed using the target loss value to obtain a fine-tuned large language model. To further improve the hierarchical multi-label classification performance of the model, the fine-tuned large language model can be further enhanced to obtain a enhanced target large language model. For example, prompts are constructed based on the sample attribute information corresponding to the erroneous samples to obtain classification judgment prompts corresponding to the erroneous samples. These prompts are then input into the fine-tuned large language model for hierarchical multi-label classification processing to obtain the error hierarchical classification labels corresponding to the erroneous samples. Further, the error hierarchical classification labels are scored according to a label scoring model to obtain the classification scores corresponding to the error hierarchical classification labels. The fine-tuned large language model is then enhanced using the classification scores to obtain the enhanced target large language model.

[0134] This embodiment can acquire pre-set classification knowledge text and sample classification label sets, and construct sample classification prompts based on the classification knowledge text, sample classification label sets, and sample object attributes corresponding to the training samples. These prompts are then input into a large language model, which performs hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the prompts, selecting corresponding predicted hierarchical classification labels from the sample classification label set. Since the classification knowledge text describes the classification knowledge, i.e., the classification logic, for hierarchical multi-label classification, the large language model can select suitable predicted hierarchical classification labels from the sample classification label set based on the learned classification logic and the analysis of the sample object attributes of the training samples. Furthermore, based on the difference between the predicted hierarchical classification labels and the labeled hierarchical classification labels corresponding to the training samples, a target loss value is determined. This target loss value is then used to train the large language model, resulting in a trained target large language model.

[0135] By fine-tuning a large language model based on prompts, classification knowledge text is embedded into sample classification prompts. This provides the large language model with clear classification knowledge for hierarchical multi-label classification during model training, enabling the trained target large language model to quickly grasp the complex classification logic combining "hierarchical classification" and "multi-classification" in hierarchical multi-label classification tasks. This effectively improves the accuracy of hierarchical multi-label classification. Furthermore, this embodiment's fine-tuning model training using sample classification prompts, leveraging predefined classification knowledge text, allows the large language model to gain a deep understanding of hierarchical multi-label classification tasks in a short time, effectively reducing the computational resources required for model training and improving training efficiency. Since the classification knowledge text in the sample classification prompts can be customized according to actual application needs, this fine-tuning method based on sample classification prompts allows the large language model to more flexibly handle hierarchical multi-label classification tasks with varying requirements, thereby improving the generalization ability of the trained target large language model on different hierarchical multi-label classification tasks.

[0136] Regarding the implementation method in the above embodiment where step 150 involves supervised fine-tuning of the large language model followed by reinforcement training, another embodiment of this disclosure provides a different method for training a large language model. This method is executed by a computer device (e.g., a desktop computer, laptop computer, or other terminal device with certain processing capabilities), which must at least have storage, computation, and communication functions. Please refer to... Figure 10 and Figure 11 , Figure 10 This embodiment shows a flowchart illustrating the large language model training method. Figure 11A flowchart of the large language model training method provided in this embodiment is shown. The following will combine... Figure 10 and Figure 11 The specific steps of training this large language model are explained below:

[0137] like Figure 11 The flowchart shown illustrates the training method for a large language model. Based on the requirements of different levels of multi-label classification tasks, corresponding raw data is collected. This raw data includes a large number of object attributes. By randomly sampling the raw data, a certain scale of sample data containing various classification scenarios can be obtained. Then, the sample data is manually labeled, that is, each sample data is manually labeled with the corresponding sample level classification label. Thus, the labeled sample data can be used as training samples. Furthermore, for each training sample, a sample classification prompt can be constructed based on the sample object attributes, classification knowledge text, and sample classification label set, serving as standard data for subsequent model training and testing.

[0138] For example, splitting standard data yields a classification training set for model training and a classification test set for model testing. Then, based on the classification training set, the pre-trained large language model is subjected to supervised fine-tuning to obtain a fine-tuned large language model. This fine-tuned large language model is then subjected to reinforcement training to obtain the trained target large language model. Finally, the performance of the trained target large language model is tested using the classification test set to ensure that the trained target large language model has accurate hierarchical multi-label classification performance.

[0139] Step 210: Obtain the classification knowledge text and sample classification label set.

[0140] Step 220: Based on the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples, construct the sample classification prompt.

[0141] Step 230: Input the sample classification prompts into the large language model. The large language model performs hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the sample classification prompts, so as to select the corresponding prediction hierarchical classification labels for the training samples from the sample classification label set.

[0142] Step 240: Determine the target loss value based on the difference between the predicted hierarchical classification label corresponding to the training sample and the labeled hierarchical classification label corresponding to the training sample.

[0143] As one implementation method, the execution process of steps 210 to 240 in this embodiment can refer to steps 110 to 140 in the above embodiment, and will not be repeated here.

[0144] Step 250: Perform supervised fine-tuning on the large language model using the target loss value to obtain the fine-tuned large language model after supervised fine-tuning.

[0145] In some embodiments, supervised fine-tuning of the large language model is performed using a target loss value, and during the supervised fine-tuning process, it is determined whether the target loss value meets the training termination condition. If the target loss value does not meet the training termination condition, all model parameters of the large language model are adjusted, i.e., full fine-tuning, until the target loss value meets the training termination condition. When the target loss value meets the training termination condition, supervised fine-tuning of the large language model is stopped, resulting in the supervised fine-tuned target large language model.

[0146] Considering that updating all model parameters of a large language model requires a huge amount of computing resources and storage space, the embodiments of this disclosure can also use a parameter-efficient fine-tuning method to train the large language model in order to minimize the number of model parameters that need to be adjusted.

[0147] In other embodiments, embodiments of this disclosure freeze the feature extraction layer of the pre-trained large language model and add a hierarchical classification layer after the feature extraction layer to complete the hierarchical multi-label classification task. Thus, during supervised fine-tuning, the first model parameters corresponding to the feature extraction layer of the large language model are fixed, and the second model parameters corresponding to the hierarchical classification layer of the large language model are supervisedly fine-tuned using the target loss value, resulting in a trained, supervised fine-tuned large language model.

[0148] Please see Figure 12 , Figure 12 This illustrates a model architecture diagram for a large language model. (For example...) Figure 12 As shown, the large language model to be fine-tuned includes its own (i.e., the large language model's) feature extraction layer and a hierarchical classification layer. The hierarchical classification layer is used for hierarchical multi-label classification based on the output of the feature extraction layer. The feature extraction layer of the large language model is usually responsible for capturing general semantic information to generate feature representations. It is independent of the specific hierarchical multi-label classification task and is suitable for direct reuse. For example, after the sample classification prompt is input into the feature extraction layer of the large language model, the feature extraction layer understands the sample classification prompt, obtains the feature information required for hierarchical multi-label classification, and passes this feature information to the hierarchical classification layer to complete the hierarchical multi-label classification task. The feature extraction layer can be the embedding layer or a low-level Transformer (such as the first few layers of BERT or GPT) of the large language model. The hierarchical classification layer refers to a classification head (such as a classifier) ​​specifically designed to handle hierarchical multi-label classification tasks.

[0149] In one implementation, the hierarchical classification layer may include a preset number of sub-hierarchical classification layers, where the preset number is the number of first-level classification labels included in the sample classification label set, and each sub-hierarchical classification layer corresponds to one first-level classification label. Each sub-hierarchical classification layer in the hierarchical classification layer is configured for hierarchical classification of a classification category in a hierarchical multi-label classification task. For example, Figure 13 This diagram illustrates the structure of a hierarchical classification layer. Figure 14 This diagram illustrates a hierarchical, multi-label classification system. For example... Figure 13 and Figure 14 As shown, when using hierarchical classification layers to perform hierarchical multi-category labeling on applications, sub-hierarchical classification layer A can be used for hierarchical classification under the first-level category "Games", sub-hierarchical classification layer B can be used for hierarchical classification under the first-level category "Education", and sub-hierarchical classification layer C can be used for hierarchical classification under the first-level category "Social".

[0150] As an example, based on the first-level category of the training samples, parameters are updated only for the sub-level classification layers corresponding to the first-level category to improve the model's fine-tuning efficiency. Specifically, based on the first-level classification label corresponding to the predicted hierarchical classification label, a target sub-level classification layer is determined from a preset number of sub-level classification layers. Then, the other layer model parameters of the other sub-level classification layers besides the target sub-level classification layer are fixed within the preset number of sub-level classification layers. Supervised fine-tuning of the target layer model parameters of the large language model corresponding to the target sub-level classification layer is performed using the target loss value, resulting in a trained, supervised fine-tuned large language model. Here, the other layer model parameters and the target layer model parameters belong to the second model parameters of the hierarchical classification layer. For example, if the first-level classification label corresponding to the predicted hierarchical classification label of an application is <game>, then... Figure 14 In this context, sub-level classification layer A is the target sub-level classification layer. Therefore, the parameters of sub-level classification layers B and C are fixed, and the parameters of sub-level classification layer A, which is the target sub-level classification layer, are updated only.

[0151] In supervised fine-tuning of large language models, by updating parameters only for the target sub-level classification layer corresponding to the first-level classification label of the predicted hierarchical classification label of the training samples, and without simultaneously updating the parameters of multiple sub-level classification layers corresponding to the first-level categories, the memory usage during supervised fine-tuning can be significantly reduced. This is especially beneficial for large language models with a large number of parameters, effectively reducing computational and storage overhead. Furthermore, by independently updating the parameters of the sub-level classification layers corresponding to different first-level categories—that is, fine-tuning—interference between "multi-classification" tasks can be effectively avoided. This isolation ensures that each hierarchical classification under a first-level category can independently learn classification knowledge based on its unique hierarchical structure, avoiding mutual influence between "multi-classification" tasks. It also prevents a classification under one first-level category from "forgetting" the classification knowledge of other first-level categories during training (i.e., catastrophic forgetting). Thus, the accuracy of the entire hierarchical multi-label classification is effectively improved.

[0152] Considering that the fine-tuned large language model may still have some mispredictions or biases, especially in its inaccurate handling of certain types of training samples or classification scenarios, and that the model may unintentionally learn some biases or inconsistent behaviors during the fine-tuning process, this disclosure proposes to further enhance the fine-tuned large language model through reinforcement training to optimize its performance in handling hierarchical multi-label classification tasks. The specific optimization process is as follows.

[0153] Step 260: Perform hierarchical multi-label classification on the training samples based on the fine-tuned large language model, and filter out the erroneous samples based on the classification results.

[0154] After supervised fine-tuning, the training samples are reclassified hierarchically using the fine-tuned large language model. This allows us to identify which training samples were incorrectly labeled by the fine-tuned large language model, thus inferring which type of training sample or classification scenario the fine-tuned large language model is not accurate enough in handling.

[0155] As one implementation method, referring to steps 110 to 140 in the above embodiments, the predicted hierarchical classification label corresponding to each training sample can be obtained again. Furthermore, by comparing the predicted hierarchical classification label and the labeled hierarchical classification label corresponding to the training sample, training samples whose predicted hierarchical classification label and the labeled hierarchical classification label do not match can be identified as erroneous samples.

[0156] Step 270: Reinforce the fine-tuned large language model using error samples to obtain the reinforced target large language model.

[0157] In some embodiments, prompts are constructed based on the sample attribute information corresponding to the erroneous samples to obtain classification judgment prompts for the erroneous samples. These prompts are then input into a fine-tuned large language model for hierarchical multi-label classification processing to obtain the error-level classification labels corresponding to the erroneous samples. Further, the error-level classification labels are scored according to a label scoring model to obtain the classification scores corresponding to the error-level classification labels. These classification scores are then used to reinforce the fine-tuned large language model, resulting in a reinforced target large language model.

[0158] The label scoring model refers to a model used to score the error-level classification labels output by a fine-tuned large language model. This model is trained using human feedback. For example, the scoring rules are manually set: 100 points are assigned to correct labels, and the scores for incorrect labels follow these rules: 100 points is the maximum score; for each output error-level classification label, 20 points are deducted for incorrect labeling, and 10 points are deducted for over- or under-labeling, with a minimum score of 0. The manually scored error-level classification labels are collected as training data, and supervised learning methods are used to teach the label scoring model to evaluate the scores of the output labels.

[0159] As an example, the classification scores of the error-level classification labels output by the label scoring model can be used to reinforce the fine-tuned large language model, resulting in a reinforced target large language model. For instance, based on the Proximal Policy Optimization (PPO) algorithm, a reinforcement loss value is calculated to optimize the model parameters according to the classification scores. For example, the reinforcement loss value is determined based on the difference between the classification score and a preset cumulative score, and the fine-tuned large language model is reinforced using this loss value until the reinforcement training termination condition is met, resulting in the reinforced target large language model. Here, the preset cumulative score refers to the state value estimated by the value function in the Proximal Policy Optimization algorithm, used to measure the overall superiority of the current state. It should be noted that reinforcement training of the fine-tuned large language model can also use methods other than the Proximal Policy Optimization algorithm, such as policy gradient-based methods, direct reward regression methods, or Actor-Critic methods, etc., which are not limited here.

[0160] By performing reinforcement training on a fine-tuned large language model, optimization can be specifically targeted at error-prone cases, thereby reducing errors in hierarchical multi-label classification and improving the robustness of the trained target large language model on hierarchical multi-label classification tasks. Through reinforcement learning based on human feedback, the fine-tuned large language model gradually improves its performance in subsequent interactions based on reward signals obtained from human feedback, i.e., classification scores. This self-improvement process allows the final trained target large language model to continuously improve in real-world hierarchical multi-label classification scenarios, especially when facing novel or complex classification scenarios.

[0161] In this embodiment of the disclosure, to verify the performance of the classification method, performance testing can be performed using a classification test set, that is, performance testing of the trained large language model on a hierarchical multi-label classification task. The testing method is a comparative test, that is, performance testing is performed on the same classification test set, corresponding to existing technologies and the method of this disclosure respectively:

[0162] Test metric: Micro-F1 score.

[0163] The F1 score is the harmonic mean of precision and recall, calculated as: F1 score = 2 × (precision × recall) / (precision + recall). Precision refers to the proportion of true positive samples predicted as positive. Recall refers to the proportion of true positive samples predicted as positive by the classification method. The Micro-F1 score is an evaluation metric for multi-class / multi-label problems. It is calculated by first determining the overall precision and recall for all classes, and then using the F1 score formula to calculate the Micro-F1 score.

[0164] Test subjects: BERT-based classification models (existing technology) and target large language models trained in this publication.

[0165] Test Implementation: On the same classification test set, hierarchical multi-label classification tasks were performed using both a BERT-based classification model and a trained target large language model. Based on the classification results, various test metrics were calculated, and the test results are shown in the table below:

[0166]

[0167] As can be seen from the table above, compared with existing technologies, the target large language model trained in this publication achieves better classification results in hierarchical multi-label classification tasks. Specifically, the Micro-F1 score for the first-level classification is improved by 0.8%, the Micro-F1 score for the second-level classification is improved by 0.9%, and the Micro-F1 score for the third-level classification is improved by 5.4%.

[0168] Another embodiment of this disclosure provides a classification method that can be executed by a computer device (e.g., a desktop computer, laptop computer, or other terminal device with certain processing capabilities), which at least has storage, computing, and communication functions. This classification method can be executed after step 150 or step 270 of the above embodiments, respectively. Please refer to... Figure 15 , Figure 15 A flowchart illustrating the classification method provided in this embodiment is shown. The following will combine... Figure 15 The specific steps of this classification method are explained below:

[0169] Step 310: Obtain the target classification knowledge text and the target classification label set.

[0170] During the model inference phase, to ensure the trained target large language model maintains the same classification performance as during training, classification knowledge text, also known as target classification knowledge text, needs to be added to the input of the trained target large language model. This target classification knowledge text and target classification label set are determined by the specific hierarchical multi-label classification scenario. For example, the target classification knowledge text and target classification label set corresponding to different hierarchical multi-label classification scenarios can be pre-stored in a database. Then, in a specific classification scenario, the appropriate target classification knowledge text and target classification label set are obtained according to the actual needs of the classification scenario.

[0171] Step 320: Based on the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object, construct the target classification prompt.

[0172] As one implementation method, a classification prompt template for hierarchical multi-tag classification is obtained. Then, by filling the classification prompt template with the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object, a target classification prompt message is obtained. For details, please refer to step 120 in the above embodiment; it will not be elaborated upon here.

[0173] Step 330: Input the target classification prompt into the target big language model. The target big language model performs hierarchical multi-label classification processing on the target classification object based on the target classification knowledge text and target object attributes in the target classification prompt, so as to select the corresponding target hierarchical classification label for the target classification object from the target classification label set.

[0174] As one implementation method, the target classification prompt is input into the trained target large language model. Then, the trained target large language model can perform terminology standardization processing on the target object attributes based on the terminology standardization text in the target classification knowledge text, obtaining standardized object attributes. Further, based on the label explanation text in the target classification knowledge text, the classification range of each sample classification label in the target classification label set is determined, obtaining the label classification range corresponding to each sample classification label. Then, using the classification standard text in the target classification knowledge text, according to the standardized object attributes and label classification range, the corresponding target level classification label is selected from the target classification label set for the target classification object. For details, refer to step 130 in the above embodiment; it will not be elaborated here.

[0175] Please see Figure 16 The diagram illustrates a structural block diagram of a training apparatus 400 for a large language model provided in an embodiment of this disclosure. The apparatus may include: a sample acquisition module 410, a prompting construction module 420, a sample classification module 430, a loss determination module 440, and a model training module 450.

[0176] The sample acquisition module 410 is used to acquire classification knowledge text and sample classification label set;

[0177] The prompt construction module 420 is used to construct sample classification prompts based on the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples.

[0178] The sample classification module 430 is used to input the sample classification prompt into the large language model, and the large language model performs hierarchical multi-label classification processing on the training samples according to the classification knowledge text and sample object attributes in the sample classification prompt, so as to select the corresponding prediction hierarchical classification label for the training samples from the sample classification label set;

[0179] The loss determination module 440 is used to determine the target loss value based on the difference between the predicted hierarchical classification label corresponding to the training sample and the labeled hierarchical classification label corresponding to the training sample.

[0180] The model training module 450 is used to train the large language model using the target loss value to obtain the trained target large language model.

[0181] In some embodiments, the sample classification module 430 may include a classification processing submodule, a label determination submodule, and a label marking submodule.

[0182] The classification processing submodule is used to perform hierarchical multi-label classification processing based on the classification knowledge text in the sample classification prompt and the sample object attributes, and to select at least one test hierarchical classification label for the training sample from the sample classification label set;

[0183] The label determination submodule is used to determine the test hierarchical classification label belonging to the sample classification label set as the correct hierarchical classification label corresponding to the training sample;

[0184] The label marking submodule is used to perform parent class label marking processing on each of the correct hierarchical classification labels to obtain the predicted hierarchical classification labels containing the parent class labels corresponding to the training samples.

[0185] In some embodiments, the classification knowledge text includes at least terminology standardization text, label explanation text, and classification standard text; the classification processing submodule may be specifically used to: perform terminology standardization processing on the sample object attributes according to the terminology standardization text to obtain standardized object attributes; perform classification range determination processing on each sample classification label in the sample classification label set based on the label explanation text to obtain the label classification range corresponding to each sample classification label; and use the classification standard text to select at least one corresponding test-level classification label from the sample classification label set for the training sample according to the standardized object attributes and label classification range.

[0186] In some embodiments, the sample classification module 430 may further include a common output submodule and a direct output submodule.

[0187] A common output submodule is used to jointly output the associated hierarchical classification label and the predicted hierarchical classification label when the predicted hierarchical classification label is detected to have an associated hierarchical classification label.

[0188] The direct output submodule is used to output the predicted hierarchical classification label when it is detected that the predicted hierarchical classification label does not have an associated hierarchical classification label.

[0189] In some embodiments, the prompt construction module 420 may be specifically used to: obtain classification task instructions and classification processing steps; perform template generation processing according to the classification task instructions and the classification processing steps to obtain a classification prompt template; and fill the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples into the classification prompt template to obtain a sample classification prompt.

[0190] In some embodiments, the model training module 450 may include a supervised fine-tuning submodule, a sample selection submodule, and a reinforcement training submodule.

[0191] The supervised fine-tuning submodule is used to perform supervised fine-tuning on the large language model using the target loss value, so as to obtain the fine-tuned large language model after supervised fine-tuning.

[0192] The sample filtering submodule is used to perform hierarchical multi-label classification processing on the training samples based on the fine-tuned large language model, and to filter out the error samples according to the classification processing results. The error samples are used to represent the training samples corresponding to the hierarchical classification labels that do not match the labeled hierarchical classification labels in the classification processing results.

[0193] The reinforcement training submodule is used to reinforce the fine-tuned large language model using the error samples to obtain the reinforced target large language model.

[0194] In some embodiments, the large language model includes a feature extraction layer and a hierarchical classification layer, wherein the hierarchical classification layer is used to perform hierarchical multi-label classification based on the output of the feature extraction layer; the supervised fine-tuning submodule may further include a parameter fixing unit and a parameter fine-tuning unit.

[0195] A parameter fixing unit is used to fix the first model parameters of the feature extraction layer corresponding to the large language model.

[0196] The parameter fine-tuning unit is used to perform supervised fine-tuning of the second model parameters of the corresponding hierarchical classification layer of the large language model using the target loss value, so as to obtain the trained supervised fine-tuned large language model.

[0197] In some embodiments, the hierarchical classification layer includes a preset number of sub-hierarchical classification layers, the preset number being the number of labels of the first-level classification labels included in the sample classification label set, and each sub-hierarchical classification layer corresponding to a first-level classification label;

[0198] The parameter fine-tuning unit can be specifically used to: determine the target sub-level classification layer from the preset number of sub-level classification layers based on the first-level classification label corresponding to the predicted level classification label; fix the other layer model parameters of the other sub-level classification layers besides the target sub-level classification layer in the preset number of sub-level classification layers; perform supervised fine-tuning of the target layer model parameters of the target sub-level classification layer corresponding to the large language model using the target loss value, to obtain the trained supervised fine-tuned large language model; wherein, the other layer model parameters and the target layer model parameters belong to the second model parameters corresponding to the level classification layer.

[0199] In some embodiments, the reinforcement training submodule may include: a prompting construction unit, an error classification unit, a scoring processing unit, and a reinforcement training unit.

[0200] The prompt construction unit is used to construct a prompt based on the sample attribute information corresponding to the erroneous sample, and obtain the classification judgment prompt corresponding to the erroneous sample.

[0201] The error classification unit is used to input the classification judgment prompt into the fine-tuned large language model for hierarchical multi-label classification processing to obtain the error hierarchical classification label corresponding to the error sample.

[0202] The scoring processing unit is used to score the error level classification label according to the label scoring model to obtain the classification score corresponding to the error level classification label;

[0203] The reinforcement training unit is used to reinforce the fine-tuned large language model through the classification score to obtain the target large language model after reinforcement training.

[0204] This embodiment can acquire pre-set classification knowledge text and sample classification label sets, and construct sample classification prompts based on the classification knowledge text, sample classification label sets, and the sample object attributes corresponding to the training samples. These sample classification prompts are then input into a large language model. Since the classification knowledge text describes the classification knowledge of hierarchical multi-label classification, i.e., classification logic, the large language model can select appropriate predicted hierarchical classification labels for the training samples from the sample classification label set based on the learned classification logic and the analysis of the sample object attributes of the training samples. Furthermore, based on the difference between the predicted hierarchical classification labels corresponding to the training samples and the labeled hierarchical classification labels corresponding to the training samples, a target loss value is determined. The large language model is then trained using this target loss value to obtain the trained target large language model. Furthermore, by fine-tuning the large language model based on prompts, classification knowledge text is embedded into the sample classification prompts. This provides the large language model with clear classification knowledge for hierarchical multi-label classification during model training, enabling the trained target large language model to quickly master the complex classification logic combining "hierarchical classification" and "multi-classification" in hierarchical multi-label classification tasks. This effectively improves the accuracy of hierarchical multi-label classification. Since the classification knowledge text in the sample classification prompts can be customized according to actual application needs, this fine-tuning method based on sample classification prompts allows the large language model to more flexibly cope with hierarchical multi-label classification tasks with varying requirements, improving the generalization ability of the trained target large language model on different hierarchical multi-label classification tasks.

[0205] In some embodiments, the reinforcement training unit may be specifically used to: determine a reinforcement loss value based on the difference between the classification score and the preset cumulative score; perform reinforcement training on the fine-tuned large language model using the reinforcement loss value until the reinforcement training termination condition is met, and obtain the target large language model after reinforcement training.

[0206] Please see Figure 17 The diagram illustrates a structural block diagram of a classification device 500 provided in an embodiment of this disclosure. The device may include: an object acquisition module 510, a prompt generation module 520, and an object classification module 530.

[0207] The object acquisition module 510 is used to acquire the target classification knowledge text and the target classification label set;

[0208] The prompt generation module 520 is used to construct a target classification prompt based on the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object;

[0209] The object classification module 530 is used to input the target classification prompt into the target large language model, and the target large language model performs hierarchical multi-label classification processing on the target classification object according to the target classification knowledge text and target object attributes in the target classification prompt, so as to select the corresponding target hierarchical classification label for the target classification object from the target classification label set;

[0210] The target large language model is trained according to the large language model training method provided in the above embodiments.

[0211] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the described devices, modules, etc., can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0212] In the several embodiments provided in this disclosure, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0213] Furthermore, the functional modules in the various embodiments of this disclosure can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0214] like Figure 18As shown in the figures, this disclosure also provides a computer device 600, which can be in various forms such as a desktop computer or a laptop computer. The computer device 600 includes a processor 610, a memory 620, a power supply 630, and an input unit 640. The memory 620 stores a computer program, which, when called by the processor 610, can execute the various method steps provided in the above embodiments. Those skilled in the art will understand that the structure of the computer device shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0215] The processor 610 may include one or more processing cores. The processor 610 connects to various parts of the entire battery management system using various interfaces and lines. It executes instructions, programs, instruction sets, or program sets stored in the memory 620, calls data stored in the memory 620, performs various functions and processes data within the battery management system, and performs various functions and processes data within the computer device, thereby providing overall control of the computer device. Optionally, the processor 610 may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 610 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 610 and may be implemented separately using a communication chip.

[0216] The memory 620 may include random access memory (RAM) or read-only memory (ROM). The memory 620 can be used to store instructions, programs, instruction sets, or program assemblies. The memory 620 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created during the use of the computer device (such as phonebook and audio / video data). Accordingly, the memory 620 may also include a memory controller to provide the processor 610 with access to the memory 620.

[0217] The power supply 630 can be logically connected to the processor 610 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 630 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0218] The input unit 640 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to object setting and function control.

[0219] Although not shown, the computer device 600 may also include an output unit, a display unit, etc., which will not be described in detail here. Specifically, in the embodiments of this disclosure, the processor 610 of the computer device loads the executable files corresponding to the processes of one or more computer programs into the memory 620 according to the following instructions, and the processor 610 runs the data stored in the memory 620, such as telephone books and audio / video data, thereby implementing the various method steps provided in the foregoing embodiments.

[0220] like Figure 19 As shown, this disclosure also provides a computer-readable storage medium 700, which stores a computer program 710. The computer program 710 can be invoked by a processor of a computer device to execute various method steps provided in this disclosure.

[0221] Computer-readable storage media can be electronic storage devices such as flash memory, electrically erasable programmable read-only memory (EEPROM), hard disks, or ROM. Optionally, the computer-readable storage medium can also be a non-volatile computer-readable storage medium. The computer-readable storage medium 700 has storage space for a computer program that performs any of the method steps in the above embodiments. These computer programs can be read from or written to one or more computer program products. The computer programs can be compressed in an appropriate form.

[0222] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the various method steps provided in the above embodiments.

[0223] The terms "comprising" and "including," and any variations thereof, in this disclosure and the foregoing drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus. "At least one" means one or more, and "more than" means two or more. "Greater than," "less than," "exceeding," etc., are understood to exclude the number itself. "Above," "below," "within," etc., are understood to include the number itself. The terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined objective, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) may be used to implement one or more modules or units. Furthermore, each module or unit may be part of a larger module or unit that includes the functionality of that module or unit.

[0224] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0225] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0226] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a program product. This computer program product is stored in a storage medium and includes several computer programs to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of this disclosure. It should also be understood that the various implementation methods provided by the embodiments of this disclosure can be arbitrarily combined to achieve different technical effects.

[0227] The above are merely preferred embodiments of this disclosure and are not intended to limit this disclosure in any way. Although this disclosure has been disclosed above with reference to preferred embodiments, it is not intended to limit this disclosure. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this disclosure. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this disclosure without departing from the content of the technical solution of this disclosure shall still fall within the scope of the technical solution of this disclosure.

Claims

1. A method for training a large language model, the method comprising: The method includes: Obtain classification knowledge text and sample classification label set; Based on the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples, a sample classification prompt is constructed. The sample classification prompt is input into the large language model, which performs hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the sample classification prompt, so as to select the corresponding prediction hierarchical classification label for the training samples from the sample classification label set. The target loss value is determined based on the difference between the predicted hierarchical classification label corresponding to the training sample and the labeled hierarchical classification label corresponding to the training sample. The target large language model is trained using the target loss value to obtain the trained target large language model.

2. The method of claim 1, wherein, The step of performing hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the sample classification prompt, to select the corresponding predicted hierarchical classification label for the training samples from the sample classification label set, includes: Based on the classification knowledge text and sample object attributes in the sample classification prompt, hierarchical multi-label classification processing is performed, and at least one hierarchical classification label to be tested is selected for the training sample from the sample classification label set. The test hierarchical classification labels belonging to the sample classification label set are determined as the correct hierarchical classification labels corresponding to the training samples; For each correct hierarchical classification label, a parent class label is marked to obtain the predicted hierarchical classification label containing the parent class label for the training sample.

3. The method of claim 2, wherein, The classification knowledge text includes at least terminology standardization text, label explanation text, and classification standard text; The step of performing hierarchical multi-label classification processing based on the classification knowledge text in the sample classification prompt and the sample object attributes, and selecting at least one test hierarchical classification label for the training sample from the sample classification label set, includes: Based on the terminology standardization text, the attribute of the sample object is subjected to terminology standardization processing to obtain standardized object attributes; Based on the label explanation text, the classification range determination process is performed on each sample classification label in the sample classification label set to obtain the label classification range corresponding to each sample classification label; Using the classification standard text, and based on the standardized object attributes and label classification range, at least one corresponding test-level classification label is selected from the sample classification label set for the training sample.

4. The method of claim 2, wherein, The method further includes: When it is detected that the predicted hierarchical classification label has an associated hierarchical classification label, the associated hierarchical classification label and the predicted hierarchical classification label are output together; When it is detected that the predicted hierarchical classification label does not have an associated hierarchical classification label, the predicted hierarchical classification label is output.

5. The method of claim 1, wherein, The step of constructing sample classification prompts based on the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples includes: Obtain classification task instructions and classification processing steps; A classification prompt template is generated based on the classification task instructions and the classification processing steps. The classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples are filled into the classification prompt template to obtain the sample classification prompt.

6. The method according to any one of claims 1 to 5, characterized in that, The step of training the large language model using the target loss value to obtain the trained target large language model includes: The large language model is then subjected to supervised fine-tuning using the target loss value to obtain a fine-tuned large language model after supervised fine-tuning. The training samples are subjected to hierarchical multi-label classification based on the fine-tuned large language model, and error samples are obtained by filtering the classification results. The error samples are used to represent the training samples corresponding to the hierarchical classification labels that do not match the labeled hierarchical classification labels in the classification results. The fine-tuned large language model is reinforced by the error samples to obtain the reinforced target large language model.

7. The method of claim 6, wherein, The large language model includes a feature extraction layer and a hierarchical classification layer. The hierarchical classification layer is used to perform hierarchical multi-label classification based on the output of the feature extraction layer. The step of performing supervised fine-tuning of the large language model using the target loss value to obtain a supervised fine-tuned large language model includes: The first model parameters of the feature extraction layer corresponding to the large language model are fixed; The second model parameters of the corresponding classification layer of the large language model are subjected to supervised fine-tuning using the target loss value to obtain the trained supervised fine-tuned large language model.

8. The method of claim 7, wherein, The hierarchical classification layer includes a preset number of sub-hierarchical classification layers, where the preset number is the number of labels in the first-level classification label set included in the sample classification label set, and each sub-hierarchical classification layer corresponds to one first-level classification label. The step of performing supervised fine-tuning of the second model parameters of the corresponding classification layer of the large language model using the target loss value to obtain the trained supervised fine-tuned large language model includes: Based on the first-level classification label corresponding to the predicted level classification label, the target sub-level classification layer is determined from the preset number of sub-level classification layers; The model parameters of other sub-level classification layers other than the target sub-level classification layer are fixed in the preset number of sub-level classification layers; The target layer model parameters of the target sub-level classification layer of the large language model are supervisedly fine-tuned using the target loss value to obtain the trained supervised fine-tuned large language model. Among them, the other layer model parameters and the target layer model parameters belong to the second model parameters corresponding to the hierarchical classification layer.

9. The method of claim 6, wherein, The step of strengthening the fine-tuned large language model through the error samples to obtain the strengthened target large language model includes: Based on the sample attribute information corresponding to the erroneous sample, a prompt message is constructed to obtain the classification judgment prompt message corresponding to the erroneous sample; The classification judgment prompt is input into the fine-tuned large language model for hierarchical multi-label classification processing to obtain the error hierarchical classification label corresponding to the error sample; The error level classification labels are scored according to the label scoring model to obtain the classification score corresponding to the error level classification labels; The fine-tuned large language model is reinforced by the classification score to obtain the reinforced target large language model.

10. The method of claim 9, wherein, The step of enhancing the fine-tuned large language model through the classification score to obtain the enhanced target large language model includes: The reinforcement loss value is determined based on the difference between the classification score and the preset cumulative score; The fine-tuned large language model is reinforced using the aforementioned reinforcement loss value until the reinforcement training termination condition is met, thus obtaining the target large language model after reinforcement training.

11. A classification method characterized by, The method includes: Obtain the target classification knowledge text and the target classification label set; Based on the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object, a target classification prompt is constructed. The target classification prompt is input into the target big language model. The target big language model performs hierarchical multi-label classification processing on the target classification object based on the target classification knowledge text and target object attributes in the target classification prompt, so as to select the corresponding target hierarchical classification label for the target classification object from the target classification label set. The target large language model is trained according to the large language model training method described in any one of weight requirements 1 to 10.

12. A training device for a large language model, characterized in that, The device includes: The sample acquisition module is used to acquire classification knowledge text and sample classification label sets; The prompt construction module is used to construct sample classification prompts based on the classification knowledge text, the sample classification label set, and the sample object attributes corresponding to the training samples. The sample classification module is used to input the sample classification prompts into the large language model, and the large language model performs hierarchical multi-label classification processing on the training samples based on the classification knowledge text and sample object attributes in the sample classification prompts, so as to select the corresponding prediction hierarchical classification label for the training samples from the sample classification label set; The loss determination module is used to determine the target loss value based on the difference between the predicted hierarchical classification label corresponding to the training sample and the labeled hierarchical classification label corresponding to the training sample. The model training module is used to train the large language model using the target loss value to obtain the trained target large language model.

13. A sorting device, characterized in that The device includes: The object acquisition module is used to acquire the target classification knowledge text and the target classification label set; The prompt generation module is used to construct a target classification prompt based on the target classification knowledge text, the target classification tag set, and the target object attributes corresponding to the target classification object; The object classification module is used to input the target classification prompt into the target large language model. The target large language model performs hierarchical multi-label classification processing on the target classification object based on the target classification knowledge text and target object attributes in the target classification prompt, so as to select the corresponding target hierarchical classification label for the target classification object from the target classification label set. The target large language model is trained according to the large language model training method described in any one of weight requirements 1 to 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the large language model training method as described in any one of claims 1 to 10, or the classification method as described in claim 11.

15. A computer device, comprising: include: Memory; A processor, wherein a computer program is stored in the memory, and when executed by the processor, the computer program implements the large language model training method as described in any one of claims 1 to 10, or the classification method as described in claim 11.

16. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the large language model training method as described in any one of claims 1 to 10, or the classification method as described in claim 11.

Images