Data governance method based on large language model SFT fine tuning technology

Abstract

The invention provides a data governance method based on a large language model SFT fine tuning technology, and the method comprises the following steps: 1, carrying out data collection and preprocessing, and constructing a multi-modal data set for training; step 2, data annotation and treatment task definition: determining a data treatment target and annotating data in the multi-modal data set; step 3, selecting a basic large model and initializing the basic large model; step 4, based on the data marked in the step 2, performing SFT supervision fine tuning on the model initialized in the step 3; 5, verifying a data governance target, if a preset requirement is met, executing the step 6, otherwise, returning to the step 3; and step 6, performing formal deployment on the model after SFT supervision fine tuning for data management. The method is high in universality, can be applied to a scene in which a large model has field specificity and a certain generalization ability needs to be kept, and is particularly applied to the field in which information security and confidentiality are high.

Description

Technical Field

[0001] The present invention relates to a data management method, in particular to a data management method based on a large language model SFT fine-tuning technology. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] The open-source language model DeepSeek-R1, released in 2025, rocked the global tech world with a technological "earthquake." DeepSeek's powerful performance, exceptionally low cost, and wide application scenarios have ignited the AI big model revolution and ushered in a new wave of digital intelligence. As an advanced AI tool, DeepSeek boasts powerful language understanding and generation capabilities, making it applicable to fields like education, healthcare, manufacturing, and finance. It can effectively assist people in completing tasks more efficiently and improving work outcomes. However, general-purpose big models like DeepSeek, which target private domain knowledge, cannot address the needs of real-world business scenarios.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of the existing technology and provide a data governance method based on the large language model SFT fine-tuning technology.

[0006] In order to solve the above technical problems, the present invention discloses a data governance method based on the large language model SFT fine-tuning technology, comprising the following steps:

[0007] Step 1: Data collection and preprocessing to build a multimodal dataset for training;

[0008] Step 2: Data annotation and governance task definition, determine data governance goals and annotate data in multimodal datasets;

[0009] Step 3: Select the basic large model and initialize it;

[0010] Step 4: Based on the data labeled in step 2, perform SFT supervised fine-tuning on the model initialized in step 3;

[0011] Step 5: Verify the data governance objectives. If they meet the preset requirements, proceed to step 6; otherwise, return to step 3.

[0012] Step 6: Deploy the SFT supervised fine-tuned model for data governance.

[0013] Furthermore, the data collection and preprocessing described in step 1 include:

[0014] Step 1-1: Collect the data to be managed, including structured and unstructured data;

[0015] Step 1-2, segment the data set for storage;

[0016] Steps 1-3: vectorize the stored data to form a multimodal dataset.

[0017] Furthermore, the data annotation described in step 2 includes:

[0018] The data in the multimodal dataset constructed in step 1 are annotated one by one, and labels and conditional parameters θ are added.

[0019] Furthermore, the data governance objectives described in step 2 include:

[0020] Given a question Q consisting of x sequences, the output c of the large language model is 1～N The probability of answer A composed of a sequence is maximized.

[0021] Furthermore, the data governance objectives described in step 2 are expressed as follows:

[0022]

[0023] Among them, i is a random variable, N is the maximum number of sequences, c i is the answer consisting of i sequences, c is the generated label of answer A, P θ (c i |x,c ＜i ) means answer c i The probability of generating label c under the large language model output answer A and conditional parameter θ, c ＜i Indicates that when generating answers c1 to c N In the process, the subsequence of answers that have been generated for positions i before.

[0024] Furthermore, the step 2 of labeling the data in the multimodal dataset includes adding labels and rules, i.e., conditional parameters θ, to the data in the multimodal dataset to be trained.

[0025] Furthermore, the selection and initialization of the basic large model described in step 3 include:

[0026] Step 3-1: Select a basic large model based on the data governance scenario;

[0027] Step 3-2: Set the pre-training weights of the basic large model according to the data governance goals determined in step 2;

[0028] Step 3-3: Based on the data governance objectives determined in step 2, selectively freeze the parameters in the basic model that do not need to be adjusted.

[0029] Furthermore, the SFT supervised fine-tuning of the model initialized in step 3 described in step 4 includes:

[0030] Step 4-1: Convert the data annotations in step 2 into constraints, as follows:

[0031] Create constraints or constraint groups for the data annotated in step 2;

[0032] The total value C of a constraint or constraint group is calculated using the following function: total Indicates that the method is as follows:

[0033] C total =C task +λ·C rule

[0034] Among them, C task is a constraint or a constraint group task, C rule is the constraint violation penalty term, λ is the weight coefficient;

[0035] Step 4-2: Input the data labeled in step 2 into the model initialized in step 3. During the fine-tuning calculation process, observe the loss function L θ To calculate the optimization goal, that is, minimizing the loss.

[0036] Furthermore, the loss function L described in step 4-2 θ It is expressed as follows:

[0037]

[0038] Among them, θ represents the conditional parameter, M is the computational label classification task, c is a constraint or constraint group, C is the total value of the constraint or constraint group, i is a random variable, x i is the i-th question in the labeled data, p θ (c|x i ; θ) represents x i and the probability of generating label c under the condition of conditional parameters θ.

[0039] Furthermore, the verification of the data governance objectives described in step 5 includes:

[0040] Model performance verification and model output target verification; among them,

[0041] The model performance verification is to use the precision and recall rate indicators to evaluate the model after the SFT supervision fine-tuning in step 4;

[0042] The model output target verification is to monitor the output of the model after SFT supervision and fine-tuning in step 4 in real time to determine whether the output meets the data governance target determined in step 2.

[0043] Beneficial effects:

[0044] 1. The present invention can reduce errors and noise in training data through data collection and preprocessing, greatly improving the accuracy and generalization ability of large model fine-tuning.

[0045] 2. The present invention can ensure the accuracy and consistency of labels by annotating training data, especially in tasks involving professional knowledge in fields such as medicine and law, and can avoid deviations in model output results caused by labeling errors.

[0046] 3. The present invention uses the loss function in the SFT supervised fine-tuning method to continuously predict the calculation of the predicted category probability of the large model for the sample data to achieve the best output result.

[0047] 4. The present invention can not only improve the accuracy and efficiency of data governance, but also bring significant beneficial effects in terms of data security and private knowledge protection. Therefore, it can be widely used in scenarios where large models have domain specificity while maintaining a certain generalization capability, especially in areas with high information security and confidentiality. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0049] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0050] Figure 2 Sorting structure diagram for the sequence.

[0051] Figure 3 Schematic diagram of loss record during the SFT fine-tuning process of the Tongyi Qianwen large model in the embodiment. DETAILED DESCRIPTION

[0052] The overall concept of this invention is as follows: Model fine-tuning includes two strategies: full parameter fine-tuning and partial parameter fine-tuning. There are also many debugging methods, each with its own unique advantages and applicable scenarios. This invention focuses on partial parameter fine-tuning for data governance. Partial parameter fine-tuning selectively updates only certain weights in the model. This is particularly important when preserving most pre-trained knowledge. This approach reduces the risk of overfitting while preserving the model's generalization capabilities, improves training efficiency, and thus enhances its performance on data governance tasks.

[0053] The present invention proposes a data governance method based on the large language model SFT fine-tuning technology, comprising the following steps:

[0054] Step 1: Data collection and preprocessing, vectorizing and storing the collected multimodal data sets, specifically including:

[0055] Step 1-1, data collection, mainly collects structured and unstructured data;

[0056] Steps 1-2, data storage segmentation, are used to segment and store the data set, providing high bandwidth and high I / O read and write guarantees for subsequent efficient model training;

[0057] Steps 1-3, data vectorization, enable the dataset to be efficiently trained and analyzed by the model, capturing data features and relationships.

[0058] Step 2: Data labeling and governance task definition. Select the data to be labeled and trained based on the data governance objectives, and then define the data governance tasks, including:

[0059] Step 2-1: Formulate governance objectives and determine governance goals, such as privacy protection, data classification, etc. Its main purpose is to make the output c generated by the model under the premise of a given x sequence of questions (Q). 1～N The probability of the answer (Answer, represented by A) composed of a sequence is maximized, and its sequence structure is as follows Figure 2 As shown, its maximum value is:

[0060]

[0061] Among them, i is a random variable, N is the maximum number of sequences, c i is the answer composed of i sequences, c is the generated label of the answer A composed of the output sequence, P θ (c i |x,c ＜i ) means c i The probability of generating label c under the output answer A and conditional parameter θ, c ＜i Indicates that in generating the sequence c1 to c NIn the process, for the subsequence that has been generated before i positions;

[0062] Step 2-2: Label the training data. By adding labels and rules, i.e., conditional parameters θ, to the data to be trained, high-quality learnable information can be provided for the large model.

[0063] Step 3: Model selection and initialization. Select a general large model based on the characteristics of the dataset and freeze some parameters based on the weights for model initialization. Specifically, the following steps are performed:

[0064] Step 3-1: Select a general big model. Choose an appropriate big model based on the data governance scenario, mainly considering factors such as performance, accuracy, and security.

[0065] In one embodiment, DeepSeek R1-14B with logical reasoning can be selected as a general large model;

[0066] Step 3-2, load the pre-training weights. Set the pre-training weights according to the goal of step 2-1 to shorten the training time under limited resources.

[0067] Step 3-3: Freeze some parameters. Selectively freeze the parameters that do not need to be adjusted according to the goal of step 2-1. This way, you can only adjust the unfrozen parameters, improving fine-tuning efficiency.

[0068] Step 4, SFT (Supervised Fine-Tuning), select the STF supervised fine-tuning method to optimize the loss function and fine-tune the model in combination with relevant fine-tuning constraints, specifically including:

[0069] Step 4-1: Convert annotations into constraints. First, we need to formulate constraints or constraint groups for the annotated training data in step 2-2 (for example, to verify the grammatical correctness and semantic fidelity of the Chinese to English translation, the constraints may include the sentence structure conforming to the grammatical rules of the target language and the use of vocabulary accurately reflecting the semantics of the original text). Then, we use the function L total =L task +λ·L rule Convert the training data into a mathematical expression, where L task is the mission loss, L rule is the rule violation penalty term, λ is the weight coefficient;

[0070] Step 4-2, loss rate calculation, based on step 2-3 and step 3, the model is fine-tuned using the SFT (Supervised Fine-Tuning) method. Specifically, the model parameters are adjusted for the labeled data in step 2-2 to achieve the generation probability prediction of task-related labels. This invention adopts The loss function is calculated, where θ represents the conditional parameter, M is the task of calculating the label classification, c is the generated label, C is the total number of label categories, i is a random variable, and x i is the i-th question Q in the labeled data, p θ (c|x i ; θ) represents x i The probability of generating label c under the condition of and conditional parameter θ;

[0071] Step 5: Verify data governance goals. Deploy the fine-tuned model in a test environment for testing. If it does not meet the original goals, return to step 3. If it meets the goals, monitor the output. If it does not meet the goals, relabel the data. If it meets the goals, proceed to step 6.

[0072] The data governance goal verification specifically includes:

[0073] Step 5-1: Test the model after fine-tuning. Use indicators such as precision and recall to test and evaluate the performance of the fine-tuned model. If the performance does not meet the target set in step 2-1, return to step 3-1 to reselect a suitable large model, reload pre-trained weights, freeze some parameters, etc. If it meets the target, enter the real-time monitoring model output step;

[0074] Step 5-2: Real-time monitoring of model output. During the test, the output of the fine-tuned model needs to be monitored in real time. If the output does not meet the target set in step 2-1, it is necessary to return to step 2-2 and re-standardize the training data and re-perform SFT supervised fine-tuning. If it meets the target, the model is fixed.

[0075] Step 6: Deploy the fine-tuned model to the production environment to provide a reasonable approach for future data governance. This includes:

[0076] It mainly uses third-party front-end tools to deploy tested fixed models into the data governance system and test the performance of the model in the actual environment, such as using open webui combined with the localized model provided by ollama for release.

[0077] Example:

[0078] One embodiment of the present invention is a data governance method based on the large language model SFT fine-tuning technology, which is applied in the scenario of interpreting the unit price of personnel for software development and operation and maintenance in the transportation industry in the "2024 China Software Industry Benchmark Data.pdf" file, such as Figure 1 As shown in Figure 2, the SFT fine-tuning process for filtering large models includes the following steps:

[0079] Step 1: Data collection and preprocessing: Upload the 2024 China Software Industry Benchmark Data.pdf to the local document library, annotate the data by segment, and then vectorize and store the document data.

[0080] Step 2: Data annotation and governance task definition. Classify and label the text content "traffic" information and encode it using:

[0081]

[0082] Calculate the maximum target probability;

[0083] Step 3: Model selection and initialization: Based on the characteristics of the dataset, select the Tongyi Qianwen large model. Use the Dify open-source large model development tool to load the model parameters. Select "Development," "Operation and Maintenance," "Rate," and other parameters for model initialization.

[0084] Step 4: SFT supervised fine-tuning. Select the STF supervised fine-tuning method to optimize the loss function and fine-tune the model in combination with relevant fine-tuning constraints. Use:

[0085]

[0086] Calculate the loss rate and continuously adjust the relevant parameters to minimize the loss rate. θ represents the model parameters, M is the calculation label classification task, c is the generated label, C is the total number of label categories, i is a random variable, and x i is the i-th question Q in the labeled data, p θ (c|x i ; θ) represents x i The probability of generating label c under the conditions of and model parameters θ. The core implementation code is shown in Table 1:

[0087] Table 1 Core implementation code table

[0088]

[0089] The loss record of this fine-tuning process is as follows Figure 3 shown.

[0090] Step 5: Verify data governance goals. Deploy the fine-tuned model in a test environment for testing. If it does not meet the original goals, return to step 3. If it meets the goals, monitor the output. If it does not meet the goals, relabel the data. If it meets the goals, proceed to step 6. The verification code is shown in Table 2.

[0091] Table 2 Verification code table

[0092]

[0093]

[0094] Step 6: Fixed model deployment. Use the Dify tool to deploy the fine-tuned model into the policy push system to test the model's effectiveness in a real environment.

[0095] In actual use, when the prompt "Developer costs in the transportation industry" is entered, the system will quickly retrieve the document's section on the transportation industry and provide the corresponding calculation method and reference unit prices for personnel in different regions. If the prompt "Developer costs in Nanjing area" is given again, the system will also quickly output the calculation method and reference unit prices for developers in Nanjing area.

[0096] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium is capable of storing a computer program that, when executed by the data processing unit, can run the invention content of a data governance method based on a large language model SFT fine-tuning technology provided by the present invention and some or all of the steps in each embodiment. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0097] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a computer program, i.e., a software product. The computer program software product can be stored in a storage medium and includes a number of instructions for enabling a device including a data processing unit (which can be a personal computer, server, single-chip microcomputer, MCU, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0098] The present invention provides a data governance method based on the SFT fine-tuning technology of a large language model. There are many methods and approaches to implement this technical solution. The above is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A data governance method based on large language model SFT fine-tuning technology, characterized by: The following steps are involved: Step 1: Data collection and preprocessing to build a multimodal dataset for training; Step 2: Data annotation and governance task definition, determine data governance goals and annotate data in multimodal datasets; Step 3: Select the basic large model and initialize it; Step 4: Based on the data labeled in step 2, perform SFT supervised fine-tuning on the model initialized in step 3; Step 5: Verify the data governance objectives. If they meet the preset requirements, proceed to step 6; otherwise, return to step 3. Step 6: Deploy the SFT supervised fine-tuned model for data governance.

2. The data management method based on the large language model SFT fine-tuning technology according to claim 1 is characterized in that: The data collection and preprocessing described in step 1 include: Step 1-1: Collect the data to be managed, including structured and unstructured data; Step 1-2, segment the data set for storage; Steps 1-3: vectorize the stored data to form a multimodal dataset.

3. The data management method based on the large language model SFT fine-tuning technology according to claim 2 is characterized in that: Data annotation as described in step 2, including: The data in the multimodal dataset constructed in step 1 are annotated one by one, and labels and conditional parameters θ are added.

4. The data management method based on the large language model SFT fine-tuning technology according to claim 3 is characterized in that: Determine data governance objectives as described in Step 2, including: Given a question Q consisting of x sequences, the output c of the large language model is 1～N The probability of answer A composed of a sequence is maximized.

5. The data management method based on the large language model SFT fine-tuning technology according to claim 4 is characterized in that: The data governance objectives described in step 2 are expressed as follows: Among them, i is a random variable, N is the maximum number of sequences, c i is the answer consisting of i sequences, c is the generated label of answer A, P θ (c i |x,c ＜i ) means answer c i The probability of generating label c under the large language model output answer A and conditional parameter θ, c ＜i Indicates that when generating answers c1 to c N In the process, the subsequence of answers that have been generated for positions i before.

6. The data management method based on the large language model SFT fine-tuning technology according to claim 5 is characterized in that: The step 2 of labeling the data in the multimodal dataset includes adding labels and rules, i.e., conditional parameters θ, to the data in the multimodal dataset to be trained.

7. The data management method based on the large language model SFT fine-tuning technology according to claim 6 is characterized in that: Select the basic large model and initialize it as described in step 3, including: Step 3-1: Select a basic large model based on the data governance scenario; Step 3-2: Set the pre-training weights of the basic large model according to the data governance goals determined in step 2; Step 3-3: Based on the data governance objectives determined in step 2, selectively freeze the parameters in the basic model that do not need to be adjusted.

8. The data management method based on the large language model SFT fine-tuning technology according to claim 7 is characterized in that: The SFT supervised fine-tuning of the model initialized in step 3 described in step 4 includes: Step 4-1: Convert the data annotations in step 2 into constraints, as follows: Create constraints or constraint groups for the data annotated in step 2; The total value C of a constraint or constraint group is calculated using the following function: total Indicates that the method is as follows: C total =C task +λ·C rule Among them, C task For a constraint or constraint group task, C rule is the constraint violation penalty term, λ is the weight coefficient; Step 4-2: Input the data labeled in step 2 into the model initialized in step 3. During the fine-tuning calculation process, observe the loss function L θ To calculate the optimization goal, that is, minimizing the loss.

9. The data management method based on the large language model SFT fine-tuning technology according to claim 8 is characterized in that: The loss function L described in step 4-2 θ It is expressed as follows: Among them, θ represents the conditional parameter, M is the computational label classification task, c is a constraint or constraint group, C is the total value of the constraint or constraint group, i is a random variable, x i is the i-th question in the labeled data, p θ (c|x i ; θ) represents x i The probability of generating label c under the condition of and conditional parameters θ.

10. The data management method based on the large language model SFT fine-tuning technology according to claim 9 is characterized in that: Validation of data governance objectives as described in Step 5, including: Model performance verification and model output target verification; among them, The model performance verification is to use the precision and recall rate indicators to evaluate the model after the SFT supervision fine-tuning in step 4; The model output target verification is to monitor the output of the model after SFT supervision and fine-tuning in step 4 in real time to determine whether the output meets the data governance target determined in step 2.

Images