Fine-tuning instruction construction method and device, equipment and medium
By performing multi-level labeling on negative text samples and modifying the initial fine-tuning instructions for constructing a large model, the bias in the pre-trained model's understanding of negative text samples was resolved, thus improving the performance of text classification tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAT COMP NETWORK & INFORMATION SECURITY MANAGEMENT CENT
- Filing Date
- 2024-05-21
- Publication Date
- 2026-07-31
AI Technical Summary
In existing technologies, pre-trained language models sometimes fail to understand technical terms or specific nouns when processing negative text samples, leading to comprehension biases and affecting training effectiveness and performance.
By dividing the labels of negative text samples into multiple levels of target classification labels, initial fine-tuning instruction data is constructed using a pre-set large model. Modifications are then made when the target model cannot follow the instructions, forming target fine-tuning instruction data to improve the model's understanding and adaptability to the instructions.
It improves the availability of fine-tuning instructions, enabling text classification tasks to better fit the knowledge of pre-trained models and enhance the performance of models on specific tasks.
Smart Images
Figure CN118734928B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for constructing fine-tuning instructions. Background Technology
[0002] In the field of Natural Language Processing (NLP), the strategy of pre-training followed by fine-tuning has become an extremely popular training paradigm in recent years. The pre-training stage is essentially an unsupervised learning method that uses a large amount of text data to enable the model to absorb and learn a rich knowledge system.
[0003] Subsequently, through supervised learning fine-tuning techniques, the model can be optimized for specific downstream tasks. The core of this approach lies in leveraging a language model that has already mastered broad semantic representations during the pre-training phase to continue training for specific tasks, rather than retraining a new model from scratch.
[0004] Since the pre-training phase lays a solid knowledge foundation for the model, this method can effectively transfer the knowledge gained in pre-training to downstream tasks, thereby significantly improving task performance. Furthermore, carefully designed fine-tuning instructions can better match the characteristics of the pre-training data, pointing the model to an efficient path towards the task objective.
[0005] Therefore, the design of fine-tuning instructions is particularly crucial in this process, as it directly relates to whether the model can fully utilize the knowledge learned in the pre-training stage and whether it can maximize performance on specific tasks. Summary of the Invention
[0006] To address the aforementioned technical problems, this disclosure provides a method, apparatus, device, and medium for constructing fine-tuning instructions.
[0007] According to one aspect of this disclosure, a method for constructing fine-tuning instructions is provided, comprising: The labels of negative text samples are divided into multiple levels of target classification labels; Based on the target classification label, an initial fine-tuning instruction data is constructed for the negative text sample using a preset large model; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text sample: the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; If the target model to be fine-tuned cannot follow the initial fine-tuning instruction data, the initial fine-tuning instruction data will be modified to the target fine-tuning instruction data.
[0008] According to another aspect of this disclosure, an apparatus for constructing fine-tuning instructions is provided, comprising: The classification module is used to divide the labels of negative text samples into multiple levels of target classification labels; The instruction construction module is used to construct initial fine-tuning instruction data for the negative text sample based on the target classification label and using a preset large model; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text sample: the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; The instruction modification module is used to modify the initial fine-tuning instruction data to the target fine-tuning instruction data when the target model to be fine-tuned cannot follow the initial fine-tuning instruction data.
[0009] According to another aspect of this disclosure, an electronic device is provided, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the above method.
[0010] According to another aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the above-described method.
[0011] The technical solution provided in this disclosure has the following advantages compared with the prior art: The technical solution provided in this disclosure includes: dividing the labels of negative text samples into multiple levels of target classification labels; constructing initial fine-tuning instruction data for negative text samples using a pre-set large model based on the target classification labels; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text samples: keywords associated with the negative text samples, classification categories of the negative text samples, and reason analysis for the classification of the negative text samples; if the target model to be fine-tuned cannot follow the initial fine-tuning instruction data, the initial fine-tuning instruction data is modified into target fine-tuning instruction data. This technical solution targets negative text samples searched from knowledge bases or the internet, first dividing them into target classification labels, and then constructing initial fine-tuning instruction data based on these labels. Regarding the problem of the model not understanding the instructions, this embodiment can check whether the target model can follow the initial fine-tuning instruction data, and if it cannot, modify the initial fine-tuning instruction data into target fine-tuning instruction data. The resulting target fine-tuning instructions enable the text classification task to better fit the knowledge of the pre-trained target model, improving the usability of the fine-tuning instructions. Attached Figure Description
[0012] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0013] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0014] Figure 1 This is a flowchart of the method for constructing the fine-tuning instructions described in the embodiments of this disclosure; Figure 2 This is a structural block diagram of the apparatus for constructing fine-tuning instructions according to an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of the electronic device described in an embodiment of this disclosure. Detailed Implementation
[0015] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0016] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0017] Fine-tuning instructions are crucial for model optimization, as they directly affect whether the model can fully utilize the knowledge learned during the pre-training phase and whether it can maximize performance on specific tasks.
[0018] In this disclosure, some methods have been developed to leverage large language models for various tasks, and there are also multiple methods for constructing large model training datasets.
[0019] For example, in existing practices for building fine-tuning instruction sets for large-scale language models for cloud computing applications, the conventional process includes scraping data from the internet, manually annotating it, using a large model for annotation, and finally validating the annotation results using a scoring model. This process typically utilizes knowledge base data, online collected data, and data generated by the large model to construct the fine-tuning instruction dataset. However, practical experience has shown that some technical terms or specific nouns may never have appeared in the model's training data, causing the model to be unable to understand their meaning. This can lead to model comprehension bias, thereby affecting the effectiveness and performance of model training.
[0020] Furthermore, in other text classification methods based on large language models, a fine-tuning instruction dataset for the large model is constructed by combining the text to be classified and prompt word templates, and the label category of the candidate set is determined by calculating the similarity between texts. However, in practical applications in specific domains, further in-depth exploration and refined operation are needed to ensure that the pre-trained model can better adapt to specific scenarios, thereby outputting results that better meet user expectations. Specifically, the use of appropriate fine-tuning instructions (prompts) to invoke the large model to obtain data, as mentioned in the proposed solution, is actually a crucial step. To achieve this goal, the fine-tuning instructions need to be carefully designed and constructed, which is a process involving creativity and technical expertise. Constructing fine-tuning instructions is not simply about posing a question, but about deeply understanding the essence of the problem and how to guide the model to generate the data required by the user through precise language expression.
[0021] In this context, embodiments of this disclosure provide a method, apparatus, device, and medium for constructing fine-tuning instructions. This technology mainly relates to a method for constructing fine-tuning instructions for text classification tasks, which involves classifying information on the network into relevant categories and providing information such as keywords and classification reasons. The fine-tuning instructions constructed in this disclosure enable text classification tasks to better fit pre-trained models.
[0022] Figure 1 This flowchart illustrates a method for constructing a fine-tuning instruction according to an embodiment of the present disclosure. This method can be executed by a fine-tuning instruction construction device configured on a terminal, which can be implemented in software and / or hardware. Figure 1 As shown, the method for constructing fine-tuning instructions may include the following steps.
[0023] Step S102: Divide the labels of negative text samples into multiple levels of target classification labels.
[0024] This embodiment describes the process of constructing classification labels, which can be achieved through two parts: setting classification labels and verifying classification labels.
[0025] Regarding the setting of classification labels, this embodiment includes: dividing the categories of labels for negative text samples according to different levels to obtain initial classification labels at multiple levels.
[0026] Negative text samples include data retrieved from knowledge bases and data crawled from specific web sources. Each negative text sample corresponds to its own tag, such as content of category A1, and the severity and level of detail of different tags may vary. Therefore, this embodiment classifies the tags of negative text samples according to different levels. As an example, the different levels can include the following three levels from high to low: a first-level category including multiple categories such as A1 and A2 content; a second-level category including multiple categories such as B1 and B2 content; and a third-level category including multiple categories such as C1 and C2 content. Through the above level division, initial classification tags with levels can be determined for each negative text sample.
[0027] Considering that lower-level categories can be mapped to higher-level categories, such as third-level categories being mapped to first-level and second-level categories, more detailed classification labels can be constructed based on the third-level categories and some second-level categories that do not have third-level categories. This will result in more detailed, diverse, accurate, and comprehensive classification labels.
[0028] For the classification label verification, this embodiment includes: inputting the initial classification label into the target model to be fine-tuned, and determining whether the output semantic information of the target model is the correct semantic information corresponding to the initial classification label.
[0029] Typically, when performing classification tasks using a large model, the model must first be able to understand the meaning of the classification labels. Only when the large model understands the meaning of the labels can it acquire the pre-trained knowledge behind them. Based on this, this embodiment needs to feed the initial classification labels to the target model to be fine-tuned, and verify whether the target model knows their meaning. That is, the initial classification labels are input into the target model to be fine-tuned, and the target model identifies and outputs the semantic information of the initial classification labels. It is then determined whether the output semantic information of the target model is the correct semantic information corresponding to the initial classification labels.
[0030] The initial classification label that is not the correct semantic information is determined as the first label to be processed.
[0031] It is understandable that, given a large number of initial classification labels, the target model may not accurately understand the meaning of a small number of initial classification labels (such as "related to region A"). For these few initial classification labels, the target model's output semantic information is not correct. Therefore, this embodiment determines the initial classification labels whose output semantic information is not correct as the first labels to be processed, and these first labels need to be changed to labels that are easy for the target model to understand and that conform to the task.
[0032] Furthermore, the name of the first label can be modified, for example, changing "related to region A" to "related to the politics of region A". Alternatively, fine-tuning instruction data for the first label can be constructed to obtain a second label that the target model can understand semantically; wherein, the fine-tuning instruction data for the first label is in question-answer pair format, used to train the target model to output the correct semantic information of the first label. For example, even after modifying the name of the first label, the target model still cannot correctly understand the semantics of the modified label, so fine-tuning instruction data for the first label can be constructed. This fine-tuning instruction data is in question-answer pair format, and the target model is trained using this fine-tuning instruction data so that the target model can correctly understand the meaning of the first label.
[0033] The second label and the initial classification label (where the output semantic information is correct) are determined as the target classification label; wherein, the target classification label is a label that is easily understood by the target model. In this embodiment, the initial classification label (where the output semantic information is correct) is a label that the target model can understand the semantics of originally, while the second label is a label that the target model can understand the semantics of after modification or fine-tuning instruction data. Therefore, the second label and the initial classification label (where the output semantic information is correct) are determined as the final target classification label.
[0034] Through the above embodiments, it is possible to construct target classification labels with multiple levels that are easy for the target model to understand for a large number of negative text samples.
[0035] Step S104: Based on the target classification label, construct initial fine-tuning instruction data for the negative text samples using a pre-set large model; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text samples: the keywords associated with the negative text samples, the classification category of the negative text samples, and the reason analysis for the classification of the negative text samples; wherein, the classification category of the negative text samples is determined from the target classification label.
[0036] In practical applications, manual annotation is labor-intensive and tedious because datasets require the construction of keywords and classification reasons. Annotators often have limited knowledge and may annotate keywords and reasons that the large model does not understand, thus limiting annotation diversity. To address these issues, this embodiment uses a large model to generate fine-tuning instruction data, allowing the model to better understand the generated fine-tuning instructions.
[0037] In this embodiment, a preset large model is used to annotate the negative text samples with initial instruction information, and the initial instruction information is constructed into initial fine-tuning instruction data; wherein, the initial instruction information includes, for example, role information, background information, requirement statement information and output format.
[0038] Role information is used to identify the reviewer of inappropriate content, such as an expert with extensive experience in reviewing inappropriate content.
[0039] Background information is used to determine whether the text contains inappropriate information. Specifically, it is responsible for reviewing articles submitted by users to identify whether they contain inappropriate content.
[0040] This embodiment, by constructing role information and background information, can review whether a text contains inappropriate content or not. It is equivalent to constructing a review mindset chain for harmful and harmless information, making the initial fine-tuning instruction data containing role information and background information more practical and logical.
[0041] The requirement statement information is used to indicate the keywords associated with the negative text samples, the classification categories of the negative text samples, and the reasons for the classification of the negative text samples. Specific requirements for the requirement statement information are as follows: If inappropriate content is found in the article, please identify one or more closely related keywords and provide a detailed analysis and explanation. You also need to select the most suitable category from the provided classification directory, which will be used to record the target classification label. If no inappropriate content is found in the article, you should still list the keywords and provide a reason analysis to explain the basis for the judgment that no inappropriate content was found. The classification category can be left blank.
[0042] Output format, used to represent the field format of the initial fine-tuning instruction data, for example: \n\n```json\n{\n\"Keywords\": [\"xxx\", \"xxx\"],\n\"Cause Analysis\": \"xxx\",\n\"Category\": [\"xxx\"]\n}\n```.
[0043] After constructing initial fine-tuning instruction data for negative text samples using a preset large model, this embodiment may further include: adjusting at least one of the role information, background information, requirement statement information, and output format contained in the initial fine-tuning instruction data.
[0044] Specifically, character information and background information need to be more diverse. Taking character information as an example, multiple identical semantic descriptions can be generated based on a large model, and these descriptions can be concatenated to obtain a variety of character information. Similarly, background information can be processed in the same way.
[0045] The requirements statement needs to fully explain the tasks that the large model needs to perform, while avoiding language that could confuse the large model. For example, in the same article, if the subject used in the preceding content is "article", then the subject in the following content should still be "article", instead of being replaced by other terms such as "content", "composition", or "work".
[0046] For the classification catalog, all categories need to be randomly combined to increase the diversity of model instructions.
[0047] In terms of output format, it is necessary to understand in advance whether the large model can stably output the desired format, and the output fields should avoid language that may confuse the model. At the same time, it is also necessary to prepare for subsequent tasks and output in an easy-to-parse format.
[0048] The above embodiments construct initial instruction information by adding initial instruction information to the initial fine-tuning instruction data, that is, by adding multiple information such as role information, background information, and keywords and reasons in the requirement statement information. This allows the target model to reason step by step to obtain the conclusion based on the initial fine-tuning instruction data, making the target model more accurate and the reasoning more logical.
[0049] Furthermore, in some embodiments, large models with pre-learned knowledge are used to annotate text data. Experiments have shown that manual annotation has significant errors, is time-consuming and labor-intensive, and suffers from low annotation diversity, poor logical consistency, and non-standardized thought chain formatting. Therefore, large models such as Qwen72b, GPT4, and YaYi30B are chosen for annotation. It should be noted that specific domain classification tasks require models tailored to specific scenarios. Therefore, it is necessary to select an appropriate large model based on the usage scenario. Additionally, the initial fine-tuning instructions for large model annotations can be manually checked to avoid data bias.
[0050] Step S106: If the target model to be fine-tuned cannot follow the initial fine-tuning instruction data, modify the initial fine-tuning instruction data to the target fine-tuning instruction data.
[0051] After constructing the initial fine-tuning instruction data according to the above embodiments, it is necessary to experiment with the initial fine-tuning instruction data in the target model to be fine-tuned in order to check whether the target model can follow the initial fine-tuning instruction data well. Based on this, this embodiment may include the following steps (1)-(4): (1) Input the initial fine-tuning instruction data into the target model to be fine-tuned to check whether the target model follows the initial fine-tuning instruction data.
[0052] (2) If the target model cannot follow the initial fine-tuning instruction data, modify the text of the initial fine-tuning instruction data.
[0053] In one specific embodiment, the initial fine-tuning instruction data is split into multiple initial instruction information; the initial instruction information is modified into target instruction information with the same semantics but different descriptive text; and the multiple target instruction information are constructed into the modified fine-tuning instruction data.
[0054] The initial fine-tuning instruction data is broken down, and each instruction is checked step by step for problems. The initial instruction information is modified to target instruction information with the same semantics but different descriptive text. The following modification example can be used as a reference: Construct a thought process chain for reviewing harmful and harmless information. Add requirements for role construction, output keywords, reasons, and step-by-step reasoning to arrive at conclusions from the target model, so that the target model can arrive at results more accurately and logically.
[0055] By prodding the engineering test to find the target model, one can better learn the fine-tuning instructions for knowledge. By breaking down the instructions and examining the results, one can determine the quality of the instruction's construction.
[0056] Enhance the target model's conceptual understanding by cross-validating the target model with manually labeled results to identify classification labels that are difficult for the target model to understand. Simultaneously, rewrite or expand the explanations of these difficult-to-understand labels.
[0057] (3) Input the modified fine-tuning instruction data back into the target model. That is, go back to step (1) and check again whether the target model follows the modified fine-tuning instruction data.
[0058] (4) When the target model follows the modified fine-tuning instruction data, the modified fine-tuning instruction data is determined to be the target fine-tuning instruction data; the target fine-tuning instruction data is data that is targeted at the target model and fits the characteristics of the target model's pre-training data.
[0059] The above steps (1)-(3) are executed once or multiple times until the target model can follow the modified fine-tuning instruction data. The above repeated process is then stopped, and the current modified fine-tuning instruction data is determined as the final target fine-tuning instruction data.
[0060] The target fine-tuning instruction data obtained from the above embodiments is highly targeted to the target model to be fine-tuned, and is more in line with the characteristics of the target model's pre-training data. It points out an efficient path to the task objective for the target model, enabling the text classification task to better fit the knowledge of the pre-trained target model.
[0061] After obtaining the target fine-tuning instruction data for fine-tuning the target model according to the above embodiments, this embodiment may further include: using a fine-tuning dataset composed of multiple target fine-tuning instruction data to train the target model for N epochs.
[0062] It is understandable that each negative text sample corresponds to its own target fine-tuning instruction data. Therefore, a large amount of target fine-tuning instruction data can be combined into a fine-tuning dataset, which can then be used to fine-tune the target model. For example, the fine-tuning dataset can be used to perform full-parameter fine-tuning on the YaYi13b model for two epochs. One practical training result shows that accuracy and recall can both reach over 80% among m (e.g., m = 80) target classification labels, with good output of information such as reasons and keywords. It is evident that the target fine-tuning instruction data constructed through this embodiment can effectively improve the fine-tuning effect of the target model and can be practically applied in classification tasks with numerous labels.
[0063] In summary, the method for constructing fine-tuning instructions provided in this disclosure includes: dividing the labels of negative text samples into multiple levels of target classification labels; constructing initial fine-tuning instruction data for negative text samples using a pre-set large model based on the target classification labels; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text samples: keywords associated with the negative text samples, classification categories of the negative text samples, and reason analysis for the classification of the negative text samples; if the target model to be fine-tuned cannot follow the initial fine-tuning instruction data, the initial fine-tuning instruction data is modified into target fine-tuning instruction data. This technical solution addresses the issue of negative text samples searched from knowledge bases or the internet, first dividing them into target classification labels, and then constructing initial fine-tuning instruction data based on these labels. Regarding the problem of the model not understanding the instructions, this embodiment can check whether the target model can follow the initial fine-tuning instruction data, and if it cannot, modify the initial fine-tuning instruction data into target fine-tuning instruction data. The resulting target fine-tuning instructions enable the text classification task to better fit the knowledge of the pre-trained target model, improving the usability of the fine-tuning instructions.
[0064] Furthermore, the aforementioned method for constructing fine-tuning instructions based on the segmentation of target classification labels can better train large language models, improve the model's instruction compliance ability, and better fit the knowledge of pre-trained models. By segmenting target classification labels, categories that the model does not understand can be identified. By modifying the target classification labels and constructing fine-tuning instruction data for categories that the model does not understand, the problem of the model not understanding classification categories can be effectively solved.
[0065] Furthermore, this disclosure constructs a thought chain labeling system for harmful and harmless information, and adds role construction and thought chain requirements to allow the model to reason step by step to reach a conclusion.
[0066] This disclosure enhances the model's conceptual understanding capabilities by identifying classification labels that are difficult for the model to understand through cross-validation between the model and manually labeled results, and then rewriting or expanding the interpretation of these labels. Furthermore, the proposed prompting engineering testing method improves the usability of fine-tuning instruction data, thereby enabling better model fitting.
[0067] Of course, depending on the actual task requirements, the fine-tuning instructions and output content of different tasks can be constructed using the scheme disclosed herein, and there are no restrictions on this.
[0068] Figure 2 This is a structural block diagram of a fine-tuning instruction construction apparatus provided in an embodiment of the present disclosure. This apparatus is used to implement the fine-tuning instruction construction method provided in the above embodiments. Figure 2 The apparatus for constructing the fine-tuning instruction includes the following modules: The classification module 210 is used to classify the labels of negative text samples into multiple levels of target classification labels; The instruction construction module 220 is used to construct initial fine-tuning instruction data for the negative text sample based on the target classification label and using a preset large model; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text sample: the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; The instruction modification module 230 is used to modify the initial fine-tuning instruction data into target fine-tuning instruction data when it is found that the target model to be fine-tuned cannot follow the initial fine-tuning instruction data.
[0069] In one embodiment, the classification module 210 is specifically used for: The labels of negative text samples are divided into categories according to different levels to obtain initial classification labels at multiple levels; The initial classification label is input into the target model to be fine-tuned, and it is determined whether the output semantic information of the target model is the correct semantic information corresponding to the initial classification label. The initial classification label whose output semantic information is not the correct semantic information is determined as the first label to be processed; The name of the first label is modified, or fine-tuning instruction data for the first label is constructed to obtain a second label that enables the target model to understand the semantics; wherein, the fine-tuning instruction data for the first label is in the format of a question-answer pair, and is used to train the target model to output the correct semantic information of the first label; The second label and the initial classification label, which is the correct semantic information in the output, are determined as the target classification label; wherein, the target classification label is a label that is easy for the target model to understand.
[0070] In one embodiment, the instruction construction module 220 is specifically used for: Based on the target classification label, a preset large model is used to annotate the negative text samples with initial instruction information, and the initial instruction information is used to construct initial fine-tuning instruction data; wherein, the initial instruction information includes: Role information, used to identify the person who reviews inappropriate information; Background information is used to review whether the text contains inappropriate information; The requirement statement information is used to indicate the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; Output format, a field format used to represent the output of the initial fine-tuning instruction data; By adding the initial instruction information to the initial fine-tuning instruction data, the reasoning of the target model after fine-tuning based on the initial fine-tuning instruction data can achieve more accurate and logical results.
[0071] In one embodiment, the device further includes an adjustment module for: At least one of the following included in the initial fine-tuning instruction data: the role information, the background information, the requirement statement information, and the output format, shall be adjusted.
[0072] In one embodiment, the instruction modification module 230 is specifically used for: The initial fine-tuning instruction data is input into the target model to be fine-tuned in order to check whether the target model follows the initial fine-tuning instruction data; If the target model cannot follow the initial fine-tuning instruction data, the initial fine-tuning instruction data is modified in text. The modified fine-tuning instructions are then input back into the target model. The modified fine-tuning instruction data is determined to be the target fine-tuning instruction data when the target model follows the modified fine-tuning instruction data; the target fine-tuning instruction data is data that is specific to the target model and conforms to the characteristics of the target model's pre-training data.
[0073] In one embodiment, the instruction modification module 230 is specifically used for: The initial fine-tuning instruction data is split into multiple initial instruction information; The initial instruction information is modified into target instruction information with the same semantics but different descriptive text; The multiple target instruction information is constructed into modified fine-tuning instruction data.
[0074] In one embodiment, the device further includes a training module for: The target model is trained for N epochs using a fine-tuning dataset consisting of multiple target fine-tuning instruction data.
[0075] The device provided in this embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.
[0076] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Figure 3 As shown, the electronic device 300 includes one or more processors 301 and memory 302.
[0077] The processor 301 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 300 to perform desired functions.
[0078] The memory 302 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 301 may execute the program instructions to implement the fine-tuning instruction construction method of the embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.
[0079] In one example, the electronic device 300 may also include an input device 303 and an output device 304, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).
[0080] In addition, the input device 303 may also include, for example, a keyboard, a mouse, etc.
[0081] The output device 304 can output various information to the outside, including determined distance information, direction information, etc. The output device 304 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.
[0082] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device 300 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 300 may include any other suitable components depending on the specific application.
[0083] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program for executing the above-described method for constructing fine-tuning instructions.
[0084] The present disclosure provides a computer program product for constructing fine-tuning instructions, including a method, apparatus, electronic device, and medium. The computer program product includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.
[0085] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0086] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for constructing fine-tuning instructions, characterized in that, include: The labels of negative text samples are divided into multiple levels of target classification labels; Based on the target classification label, an initial fine-tuning instruction data is constructed for the negative text sample using a preset large model; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text sample: the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; If the target model to be fine-tuned cannot follow the initial fine-tuning instruction data, the initial fine-tuning instruction data will be modified to the target fine-tuning instruction data. The method of classifying negative text samples into multiple levels of target classification labels includes: The labels of negative text samples are divided into categories according to different levels to obtain initial classification labels at multiple levels; The initial classification label is input into the target model to be fine-tuned, and it is determined whether the output semantic information of the target model is the correct semantic information corresponding to the initial classification label. The initial classification label whose output semantic information is not the correct semantic information is determined as the first label to be processed; The name of the first label is modified, or fine-tuning instruction data for the first label is constructed to obtain a second label that enables the target model to understand the semantics; wherein, the fine-tuning instruction data for the first label is in the format of a question-answer pair, and is used to train the target model to output the correct semantic information of the first label; The second label and the initial classification label, which is the correct semantic information in the output, are determined as the target classification label; wherein, the target classification label is a label that is easy for the target model to understand; The step of constructing initial fine-tuning instruction data for the negative text samples based on the target classification label using a preset large model includes: Based on the target classification label, a preset large model is used to annotate the negative text samples with initial instruction information, and the initial instruction information is used to construct initial fine-tuning instruction data; wherein, the initial instruction information includes: Role information, used to identify the person who reviews inappropriate information; Background information is used to review whether the text contains inappropriate information; The requirement statement information is used to indicate the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; Output format, a field format used to represent the output of the initial fine-tuning instruction data; The step of modifying the initial fine-tuning instruction data to the target fine-tuning instruction data when the target model to be fine-tuned cannot follow the initial fine-tuning instruction data includes: The initial fine-tuning instruction data is input into the target model to be fine-tuned in order to check whether the target model follows the initial fine-tuning instruction data; If the target model cannot follow the initial fine-tuning instruction data, the initial fine-tuning instruction data is modified in text. The modified fine-tuning instructions are then input back into the target model. The modified fine-tuning instruction data is determined to be the target fine-tuning instruction data when the target model follows the modified fine-tuning instruction data; the target fine-tuning instruction data is data that is specific to the target model and conforms to the characteristics of the target model's pre-training data.
2. The method of claim 1, wherein, After constructing initial fine-tuning instruction data for the negative text samples using a preset large model, the method further includes: At least one of the following included in the initial fine-tuning instruction data: the role information, the background information, the requirement statement information, and the output format, shall be adjusted.
3. The method of claim 1, wherein, The text modification of the initial fine-tuning instruction data includes: The initial fine-tuning instruction data is split into multiple initial instruction information; The initial instruction information is modified into target instruction information with the same semantics but different descriptive text; The multiple target instruction information is constructed into modified fine-tuning instruction data.
4. The method of claim 1, wherein, The method further includes: The target model is trained for N epochs using a fine-tuning dataset consisting of multiple target fine-tuning instruction data.
5. A construction apparatus of fine adjustment instructions, characterized by, include: The classification module is used to divide the labels of negative text samples into multiple levels of target classification labels; The instruction construction module is used to construct initial fine-tuning instruction data for the negative text sample based on the target classification label and using a preset large model; the initial fine-tuning instruction data is used to represent at least one of the following information of the negative text sample: the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; The instruction modification module is used to modify the initial fine-tuning instruction data into the target fine-tuning instruction data when the target model to be fine-tuned cannot follow the initial fine-tuning instruction data. The classification module is also used for: The labels of negative text samples are divided into categories according to different levels to obtain initial classification labels at multiple levels; The initial classification label is input into the target model to be fine-tuned, and it is determined whether the output semantic information of the target model is the correct semantic information corresponding to the initial classification label. The initial classification label whose output semantic information is not the correct semantic information is determined as the first label to be processed; The name of the first label is modified, or fine-tuning instruction data for the first label is constructed to obtain a second label that enables the target model to understand the semantics; wherein, the fine-tuning instruction data for the first label is in the format of a question-answer pair, and is used to train the target model to output the correct semantic information of the first label; The second label and the initial classification label, which is the correct semantic information in the output, are determined as the target classification label; wherein, the target classification label is a label that is easy for the target model to understand; The instruction construction module is also used for: Based on the target classification label, a preset large model is used to annotate the negative text samples with initial instruction information, and the initial instruction information is used to construct initial fine-tuning instruction data; wherein, the initial instruction information includes: Role information, used to identify the person who reviews inappropriate information; Background information is used to review whether the text contains inappropriate information; The requirement statement information is used to indicate the keywords associated with the negative text sample, the classification category of the negative text sample, and the reason analysis for the classification of the negative text sample; Output format, a field format used to represent the output of the initial fine-tuning instruction data; The instruction modification module is also used for: The initial fine-tuning instruction data is input into the target model to be fine-tuned in order to check whether the target model follows the initial fine-tuning instruction data; If the target model cannot follow the initial fine-tuning instruction data, the initial fine-tuning instruction data is modified in text. The modified fine-tuning instructions are then input back into the target model. The modified fine-tuning instruction data is determined to be the target fine-tuning instruction data when the target model follows the modified fine-tuning instruction data; the target fine-tuning instruction data is data that is specific to the target model and conforms to the characteristics of the target model's pre-training data.
6. An electronic device, comprising: The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1-4.