Industrial large model-oriented fine tuning data generation method, computer program and terminal

Through the combination of text recognition model and large language model, text data in professional fields is extracted and corrected, and errors and garbled codes are solved in the data transformation from books to large models, and high-quality fine-tuned data generation is achieved, which improves the model's Accuracy and efficiency.

CN119004100APending Publication Date: 2024-11-22UNIV OF ELECTRONICS SCI & TECH OF CHINA

Patent Information

Application Number
CN202411023180.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing fine-tuning data generation methods have problems with identification errors and formula garbled codes in the process of converting books from professional fields to large models, which affects data quality and fine-tuning effects, and obtaining high-quality fine-tuning data has challenges in vertical fields.

Method used

Text recognition model is used to extract text data from professional data sources, pre-process it through statistical feature text filtering function, and grammatical correction and confusion calculation are used to use large language models to segment text data to generate high-quality fine-tuning data.

Benefits of technology

It effectively avoids the illusion of big models, improves data quality and nature, reduces the annotation workload, improves the recognition accuracy and efficiency, and ensures the high quality and standardization of fine-tuning data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119004100A_ABST
    Figure CN119004100A_ABST
Patent Text Reader

Abstract

The invention discloses a fine tuning data generation method for an industry large model, a computer program and a terminal, and belongs to the field of artificial intelligence, and the method comprises the following steps: extracting text data in a professional data source picture; preprocessing the text data; identifying and correcting the grammar of the text data; calculating the confusion degree of the text data to eliminate sentences with low scores; and segmenting the text data, inputting the segmented text data into the large language model, and generating fine tuning data in combination with prompts. A text recognition model is adopted to recognize text data in a professional data source picture, and the problem of large model illusion is avoided; the grammar and spelling errors of the text data are checked and corrected by using the large language model enhanced through reinforcement learning, so that the data generation quality can be further improved; the text data are segmented and then input into the large language model, the method is more suitable for large-scale text data processing, semantic boundaries in the text can be effectively recognized while high efficiency is kept, and high-quality instruction fine adjustment data are generated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method, computer program and terminal for generating fine-tuning data for a large industry model. Background Art

[0002] In the field of artificial intelligence, large models have become a key force driving technological progress. Large models usually refer to neural networks with many parameters and complex structures. They have learned a wealth of general knowledge by pre-training on massive data. In order to apply large models with strong general capabilities to professional fields, fine-tuning has become an important means to improve the performance of models in specific fields or tasks. The fine-tuning process involves additional training of pre-trained models on datasets in specific fields to adjust model parameters to make them more suitable for specific application scenarios. This method can not only significantly improve the accuracy and efficiency of the model, but also reduce the resources and time required to train models from scratch for specific tasks. Fine-tuning data is the core of the fine-tuning process, which directly determines the performance of large models on specific tasks. High-quality fine-tuning data can provide accurate examples for the model and help the model learn task-related features and laws. The diversity and representativeness of fine-tuning data are crucial to improving the generalization ability of the model. It ensures that the model does not overfit on a specific type of data, but can adapt to a variety of different input situations. This approach can not only significantly improve the accuracy and efficiency of the model, but also reduce the resources and time required to train the model from scratch for a specific task. Therefore, for the construction of large models in vertical fields, data is of paramount importance. However, there are still many challenges in obtaining high-quality fine-tuning data in vertical fields:

[0003] (1) Existing fine-tuning data generation methods rely on distillation of GPT. However, the data obtained from GPT is inaccurate, which will cause the problem of large model hallucination. Accurate data in professional fields often exist in professional books.

[0004] (2) In the process of converting fine-tuning data from professional books to large models, many recognition errors and garbled formulas will occur, which seriously affect the data quality and fine-tuning effect. A complete set of technical processes for data cleaning and data conversion is urgently needed. Summary of the invention

[0005] The purpose of the present invention is to overcome the problems of the prior art and to provide a method, a computer program and a terminal for generating fine-tuning data for a large industry model.

[0006] The purpose of the present invention is to achieve the following technical solution: a method for generating fine-tuning data for an industry large model, the method comprising the following steps:

[0007] Use text recognition models to extract text data from professional data source images;

[0008] Preprocess the extracted text data based on statistical feature text filtering function;

[0009] Use a large language model to identify and correct the grammar of text data;

[0010] Use a large language model to calculate the perplexity of the text data, score the sentences based on the perplexity, and remove sentences with scores below the first threshold;

[0011] The text data is segmented into text segments whose content similarity is greater than a second threshold, the segmented text is input into the large language model, and the fine-tuning data is generated in combination with the prompts.

[0012] In one example, a text recognition model is used to extract text data from professional data source images, including:

[0013] Label the private data set, generate text detection boxes and recognition labels, and use the formula editor to edit the formula to obtain the manually labeled private data set;

[0014] Use manually annotated private datasets to fine-tune the text recognition model pre-trained on the public dataset, calculate the loss function, and optimize the loss function based on the stochastic gradient descent method to complete the training of the text recognition model;

[0015] Use the trained text recognition model to extract text data from images from professional data sources.

[0016] In one example, the extracted text data is preprocessed according to the statistical feature text filtering function, including:

[0017] Conduct statistical analysis on the frequency of use, distribution location, and ratio of punctuation marks to text in the standardized text data;

[0018] Cluster analysis algorithm is used to perform pattern recognition on statistical analysis data, and a text feature model that can reflect standard text features is constructed based on the cluster analysis results;

[0019] The text feature model is used to analyze the text data, and regular expressions are used to match the garbled symbols in the text data, and clean up operations are performed, including replacing, deleting or correcting symbols that do not meet the specifications.

[0020] In one example, a large language model based on reinforcement learning with AI feedback is used to identify and correct the grammar of text data, including:

[0021] A variety of prompt engineering methods are used to guide the grammatical error correction model of the large language model to score the grammatical correctness of text data and generate preference labels to indicate the degree to which the candidate items conform to grammatical rules;

[0022] The reward model is trained according to the preference labels, and each candidate generated by the large language model is evaluated using the trained reward model. A reward value is assigned to each candidate, and a reinforcement learning algorithm is used to calculate the gradient value of the expected value of the reward value. The parameters of the large language model are updated based on the gradient value, thereby optimizing the large language model.

[0023] Use the optimized large language model to identify and correct the grammar of text data.

[0024] In one example, multiple prompt engineering methods include one or more of preface guidance, thought chain reasoning, and context learning; preface guidance provides task descriptions and scoring criteria for the large language model; thought chain reasoning first uses the large language model to explain the grammatical rationality of each candidate, and then generates a preference distribution based on the explanation; context learning enables the large language model to learn the application of grammatical rules by providing examples of completing manual error correction, rationality explanation and label annotation.

[0025] In one example, a large language model is used to calculate the perplexity of text data, including:

[0026] A large language model is used to convert text data into vector form. The large language model calculates the perplexity of the sentence based on the sentence features calculated by the text feature model, and scores the sentence based on the perplexity, semantics, and punctuation distribution.

[0027] In one example, segmenting the text data into text segments whose content similarity is greater than a second threshold includes:

[0028] Convert text data into vectors;

[0029] Generate a hash signature for the vectorized text fragment using the minimum hash algorithm;

[0030] Locally sensitive hashing is used to cluster text fragments with hash signatures whose similarity is greater than a second threshold to form candidate segmentation points, thereby achieving text segmentation.

[0031] It should be further explained that the technical features corresponding to the various examples of the above method can be combined or replaced with each other to form a new technical solution.

[0032] The present invention also includes a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method for generating fine-tuning data for an industry large model formed by any one of the above examples or a combination of multiple examples.

[0033] The present invention also includes a terminal, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and when the processor runs the computer instructions, the steps of the method for generating fine-tuning data for industry large models formed by any one or more of the above examples are executed.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. In one example, the present invention uses a text recognition model to recognize text data in professional data source images to avoid the problem of large model hallucination; pre-processes the extracted text data based on the statistical feature text filtering function, which can eliminate low-quality data that does not meet the standards (such as garbled characters, error information, etc.); uses a large language model enhanced by reinforcement learning to check and correct grammatical and spelling errors in text data, which can further improve the quality of data generation; introduces a perplexity index to further eliminate sentences with unnatural structures or words, ensuring the high quality and naturalness of text data; and divides the text data and inputs it into a large language model, which is more suitable for large-scale text data processing, and can effectively identify semantic boundaries in the text while maintaining high efficiency, and generate high-quality (format specification, semantic integrity, accuracy, etc.) instruction fine-tuning data.

[0036] 2. In one example, a text recognition model pre-trained on a public dataset is fine-tuned using a private dataset. Only the private dataset needs to be labeled, which greatly reduces the labeling workload for fine-tuning the large model and can further improve the recognition accuracy and efficiency of the text recognition model.

[0037] 3. In one example, based on the statistical feature text filtering function, and in-depth analysis of the statistical features of the text content such as punctuation distribution, symbol-to-word ratio, etc., the regular expression method is used to clean up garbled symbols, which can ensure the standardization and readability of the text data.

[0038] 4. In one example, reinforcement learning and the Chinese grammar correction of a large language model are combined to fully utilize the powerful grammar correction capability of the large language model and avoid the problem of excessive correction of the large language model. At the same time, the error correction scheme of the large language model is evaluated using a reward model based on the AI ​​feedback method, and the feedback is fed back to the large language model for reinforcement learning. The entire process does not require human intervention, which improves the efficiency of grammar recognition and correction of text data.

[0039] 5. In one example, a method is proposed to use a large language model to calculate the text perplexity and then perform text filtering. By combining the calculation of perplexity with the analysis and understanding of the text by the large language model, meaningless text information can be filtered more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings. The accompanying drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to represent the same or similar parts. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0041] Figure 1 A flow chart of a method provided for an example of the present invention. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the description of the present invention, it should be noted that the directions or positional relationships indicated by "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. are directions or positional relationships based on the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the use of ordinal numbers (e.g., "first and second", "first to fourth", etc.) is to distinguish objects, and is not limited to this order, and cannot be understood as indicating or implying relative importance.

[0044] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0045] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0046] In one example, if Figure 1 As shown, a method for generating fine-tuning data for a large industry model includes the following steps:

[0047] S1: Use text recognition model to extract text data from professional data source images;

[0048] S2: preprocessing the extracted text data based on statistical feature text filtering function;

[0049] S3: Use a large language model to identify and correct the grammar of text data;

[0050] S4: Use the large language model to calculate the perplexity of the text data, score the sentences according to the perplexity, and remove sentences with scores lower than the first threshold;

[0051] S5: Segment the text data into text segments whose content similarity is greater than a second threshold, input the segmented text into the large language model, generate fine-tuning data in combination with the prompt, and realize the conversion of text data into instruction data.

[0052] Among them, in step S1, the professional data source is professional books, journals, etc., which can be stored in the form of files (such as PDF) or pictures. Optionally, the text recognition model can be an OCR (Optical Character Recognition) model, which mainly performs text recognition based on rules and statistical models, such as hidden Markov models (HMM) and support vector machines (SVM). The present invention aims to convert vertical professional books into large language model fine-tuning data, and through text processing and knowledge extraction technology, convert the key information and professional knowledge in the books into a data format that can be used for fine-tuning, thereby greatly reducing the data preparation cost required to customize a large language model for a specific mechanical engineering task, while ensuring that the model can obtain the necessary domain expertise. Preferably, this example uses the Paddle OCR text recognition model, which supports multi-language text recognition, including Chinese and English, and can process tilted and vertical text, and can adapt to industry fields containing a large number of formulas and charts, such as the machine tool field.

[0053] Furthermore, the original professional data source images may have problems such as blurred images, and the OCR model in this example inevitably mixes in garbled characters and erroneous information. Therefore, step S2 uses statistical feature text filtering function to conduct in-depth analysis of the extracted text data to eliminate low-quality data that does not meet the standards.

[0054] Furthermore, the preprocessed text filters out low-quality text data, but there are still many grammatical errors (or spelling errors) in the text. Therefore, step S3 identifies and corrects the grammar based on reinforcement learning and the grammar correction model of the large language model, providing reliability guarantee for the generation of fine-tuning data.

[0055] In step S4, the first threshold can be set manually based on historical experience, and sentences that do not meet the requirements are eliminated based on the first threshold; by introducing perplexity-based indicator filtering technology, the perplexity of text data is calculated using a pre-trained large language model, which is used as a basis for evaluating the naturalness of the text, and sentences with unnatural structures or words are eliminated to ensure the high quality and naturalness of the text data.

[0056] In step S5, the second threshold can be set manually based on historical experience, and a text segment with high similarity is selected based on the second threshold. The data format of instruction fine-tuning usually includes two parts: input data and target data. Among them, the input data (Input) is the data that the large language model needs to process. For text data, the input data can be a series of text samples, each sample may include an instruction or question and related context information. The sample is usually expressed in the form of a text sequence, which can be a simple sentence, or a description, or a question and its answer pair. The target data (Output) is the result that the large language model needs to predict or generate. For instruction fine-tuning, the target data may be a complete answer, a summary, a translation result, or the output of completing a specific task. The target data corresponds to the input data and clearly indicates the expected output that the model should produce when given the input. The present invention utilizes the text understanding ability of the large language model to generate instruction fine-tuning data in the form of question-answer text pairs through the method of prompt engineering.

[0057] In one example, a text recognition model is used to extract text data from professional data source images, including:

[0058] S11: annotate the private data set, generate text detection boxes and recognition labels, and edit the formula using a formula editor to obtain a manually annotated private data set;

[0059] S12: Fine-tune the text recognition model pre-trained on the public dataset using the manually annotated private dataset, calculate the loss function, and optimize the loss function based on the stochastic gradient descent method to complete the training of the text recognition model;

[0060] S13: Use the trained text recognition model to extract text data from professional data source images.

[0061] In step S11, the purpose of manually annotating a private data set is to make the model more suitable for specific tasks in a specific field. In this example, the PPOCRLabel tool is used to annotate the private data set, generate text detection boxes and recognition labels, and then the corresponding formulas are edited by the latex tool. There are 500 manually annotated data sets in this example. Finally, the data is divided into a training set and a test set according to a 10 to 1 random extraction method. The public data set uses a real data set for mathematical formula recognition in open source Chinese documents. The data set contains a total of 200 document pages, including 1166 independent formulas and 3022 embedded formulas, which are selected from CEB documents from 24 digital sources. When using a public data set to pre-train a text recognition model, the public data set can also be divided into a training set and a test set according to a 10 to 1 random extraction method.

[0062] In step S12, the text recognition model pre-trained on the large dataset (shared dataset) is further fine-tuned by constructing a specific dataset (private dataset), and the CTC (Connectionist Temporal Classification) loss of the text recognition model is calculated. The CTC loss allows the model to map the input sequence (such as the feature sequence extracted from the image) to the output sequence (such as the recognized text sequence), and the SGD (stochastic gradient descent) algorithm is used to optimize the loss function, so as to improve the OCR recognition ability of the formula.

[0063] More specifically, during the training process, a virtual environment is first created and a specific version of PaddlePaddle-gpu and PaddleOCR is installed, and public data sets and manually annotated private data sets are used for training. The training process is divided into two main parts: text detection and text recognition. For the text detection model, it is necessary to download a pre-trained detection model, such as MobileNetV3_large_x0_5, and modify the configuration file to meet the training requirements. Then execute the training command and visualize the training process through the VisualDL tool. After the training is completed, evaluate the model performance and export the model for subsequent use. For the text recognition model, similar to the recognition step of the text detection model, it is necessary to download and prepare a pre-trained text recognition model, such as the English recognition model of PP-OCRv3, modify the configuration file, perform training, and use VisualDL to visualize the training process. After the recognition model training is completed, it is also necessary to evaluate and export the model. After the training is completed, a similar method is used to train the text recognition model. Finally, step S13 is the inference process, in which text recognition can be performed in batches, and the model returns data in json format, which is converted into text data through a Python script.

[0064] In one example, the extracted text data is preprocessed according to the statistical feature text filtering function, including:

[0065] S21: Statistical analysis of the frequency of use, distribution position, and ratio of punctuation marks to text in the standardized text data;

[0066] S22: using a cluster analysis algorithm to perform pattern recognition on the statistical analysis data, and constructing a text feature model that can reflect the standard text features based on the cluster analysis results;

[0067] S23: Analyze the text data using the text feature model, match the garbled symbols in the text data using regular expressions, and perform cleaning operations, including replacing, deleting or correcting symbols that do not meet the specifications.

[0068] Specifically, in this example, a large amount of standard text data (text data without garbled characters and standardized formula expressions) is first collected and preprocessed, and a detailed statistical analysis is conducted on the frequency of use, distribution position, and ratio of symbols to words of punctuation marks in the standard text data to establish a standard model for punctuation distribution and symbol-to-word ratio. Then, a cluster analysis algorithm is used to perform in-depth pattern recognition on the statistical data to form a text feature model that can accurately reflect the characteristics of standard text. Finally, according to the text feature model, sentence by sentence analysis is performed to automatically identify and use regular expressions to match garbled symbols in the text, and necessary cleaning operations are performed, including replacing, deleting or correcting symbols that do not meet the standards to ensure the standardization and readability of the text data, which not only improves the processing efficiency of text data, but also enhances the accuracy of text analysis and information extraction, and is suitable for a wide range of language environments and text scenarios.

[0069] In one example, a large language model is used to identify and correct the grammar of text data, including:

[0070] S31: Use multiple prompt engineering methods to guide the grammatical error correction model of the large language model to score the grammatical correctness of the text data and generate preference labels to indicate the degree to which the candidate items (which can be sentences, paragraphs, words or characters, etc.) conform to the grammatical rules;

[0071] S32: Train a reward model according to the preference labels, use the trained reward model to evaluate each candidate generated by the large language model, assign a reward value to each candidate, and use a reinforcement learning algorithm to calculate the gradient value of the expected value of the reward value, and update the parameters of the large language model based on the gradient value, so as to optimize the large language model;

[0072] S33: Use the optimized large language model to identify and correct the grammar of text data.

[0073] Specifically, in Chinese natural language processing, Chinese grammatical error correction (CGEC) has long been regarded as a basic task. The CGEC task aims to correct all possible grammatical errors in the input sentence, which is challenging because it requires the model to have a comprehensive understanding of the complex semantics of the text. LLMs (Large Language Models) perform supervised fine-tuning on CGEC data, but the performance on the CGEC task still cannot achieve the expected effect. The main reason is that the relatively free generation paradigm makes the sentences generated by LLMs often unable to meet the minimum change principle pursued by CGEC. Therefore, the present invention proposes to use reinforcement learning methods to train and constrain large language models, improve the grammatical error correction ability of large language models, and reduce the occurrence of overcorrection. Specifically, this example uses a pre-trained large language model (such as GPT) to identify grammatical problems in the text and make corresponding correction suggestions. At the same time, the RLAIF (Reinforcement Learning from AI Feedback, reinforcement learning based on artificial intelligence feedback) method is adopted to guide the optimization of grammatical error correction strategies using feedback generated by large language models.

[0074] Specifically, Chinese text is input into a large language model, and the large language model is requested to score the grammatical correctness of the text. The large language model will generate a preference label based on its internal language model evaluation, indicating which candidate items are more in line with grammatical rules. In order to improve the accuracy of LLM in generating preference labels, the present invention adopts a variety of prompt engineering methods to better guide the large language model. The various prompt engineering methods include one or more of preface guidance, chain of thought reasoning, and context learning; preface guidance provides a detailed task description and scoring criteria for the large language model to help the large language model better understand the scoring task; chain of thought reasoning first uses a large language model to explain the grammatical rationality of each candidate item, and then generates a preference distribution based on the explanation; context learning enables the large language model to learn the application of grammatical rules by providing examples of completing manual error correction, rationality explanation, and label annotation.

[0075] Then, the preference labels are used to train a reward model (RM), which will learn how to evaluate the effect of grammatical correction based on the preferences of the large language model. During the training process, a reinforcement learning algorithm is used. This example specifically uses the REINFORCE algorithm to optimize the parameters of the large language model so that the grammatical corrections it generates are more in line with human preferences. The core of the REINFORCE algorithm is to update the parameters of the model through reward signals. In this scenario, the environment is composed of Chinese text, and the behavior is each grammatical correction proposed by the model. The model's strategy is represented by π θ, defined by the current model parameters θ, which determines the t Next take action A t The probability of the large language model to correct the incorrect sentence is the error correction solution taken by the large language model. In the grammatical correction task, each step of the model involves generating or selecting a grammatically correct sentence. The REINFORCE algorithm uses the reward signal from the reward model (RM) to guide the learning process of the model. The reward signal is determined based on the consistency of the corrected sentence generated by the large language model with the preferences of the human annotator. The update rule of the REINFORCE algorithm can be expressed as:

[0076]

[0077] Among them, t represents the time step; α represents the learning rate, which controls the amplitude of parameter update; represents the gradient of the model parameter θ; logπ θ (A t |X t ) is the model taking action A t The logarithm of the strategy probability when R(X t , A t ) means that in a given context X t Next take action A t Rewards received.

[0078] Preferably, in order to avoid the large language model producing unnatural or low-quality language output in the process of pursuing high rewards, the so-called "reward hacking" phenomenon, this example introduces a regularization term, such as Kullback-Leibler divergence, expressed as DKL, in the objective function of the REINFORCE algorithm. This helps to maintain the naturalness and diversity of the model output. Ultimately, the optimization objective of the large language model can be integrated into the following form:

[0079]

[0080] Where J(θ) represents the strategy π θ The objective function of represents the expected value; β is a hyperparameter used to balance the importance of the reward signal and the regularization term; π SFT (A|X) represents the initial supervised fine-tuning strategy obtained through supervised fine-tuning (SFT), which ensures that the model update does not deviate from the original high-quality output.

[0081] In this way, the introduction of the REINFORCE algorithm enables the Chinese grammar correction model of the present invention to learn how to generate more accurate, more natural, and more human-preferred grammar corrections in continuous iterations, thereby achieving continuous improvement and optimization of model performance. At the same time, the large language model not only participates in generating candidate items, but also participates in evaluating the quality of candidate items, and feeds these evaluations back to the model training, forming a closed-loop self-improvement process, that is, the large language model of reinforcement learning of the present application through AI feedback recognizes and corrects the grammar of text data, and the whole process does not require human interference.

[0082] In one example, a large language model is used to calculate the perplexity of text data, including:

[0083] A large language model is used to convert text data into vector form. The large language model calculates the perplexity of the sentence based on the sentence features calculated by the text feature model, and scores the sentence based on the perplexity, semantics, and punctuation distribution.

[0084] The traditional perplexity calculation method uses the Bert model to calculate the perplexity index, and then uses the average value as the text filtering standard. This method is prone to semantic misjudgment, and is slow in the case of large texts. Therefore, the present invention uses a large language model to calculate the text perplexity, and after vectorizing the text through the embedding layer of the large language model, the text perplexity is calculated, and then the sentence features are calculated in combination with the text feature model in step S22 and input into the large language model, so that the large language model can be comprehensively analyzed. The perplexity calculation involves masking each token (word unit, which can be a word, character, punctuation or other language element) in a given sentence, and then calculating the model to predict the negative log likelihood loss (Negative Log Likelihood, NLL) of these masked tokens. The calculation expression of the negative log likelihood loss is:

[0085]

[0086] Where N is the total number of words or sentences in the text; p(w i ) represents the probability of the model predicting the i-th word or sentence. After summing and averaging the NLL of all tokens, the perplexity (PPL) of the sentence is obtained by taking the exponent.

[0087]

[0088] Among them, P(w i |w 1 , ..., w i-1 ) indicates that given the previous context w 1 , ..., w i-1Next word or sentence w i After the perplexity is calculated, the large language model combines perplexity, semantics, and punctuation distribution to judge the sentence from three aspects: whether the semantics are smooth, whether the punctuation ratio is abnormal, and whether the perplexity is abnormal or an extreme point in the continuous context. The large language model scores the sentences based on the above three criteria and filters out low-scoring sentences, thereby eliminating those unnatural or low-quality sentences.

[0089] In one example, segmenting the text data into text segments whose content similarity is greater than a second threshold includes:

[0090] S51: Convert text data into vectors;

[0091] S52: Generate a hash signature for the vectorized text fragment using a MinHash algorithm;

[0092] S53: Use local sensitive hashing to cluster text segments with hash signatures whose similarity is greater than a second threshold to form candidate segmentation points, thereby achieving text segmentation.

[0093] Specifically, in order to generate instruction fine-tuning data in the form of question-answer text pairs, the large-scale text data that has been cleaned is first split into texts, the text content is converted into vectors through the Word2Vec model, and then the MinHash algorithm is used to generate hash signatures for the vectors. The hash signature can capture the similarity between text fragments, so that similar text fragments are mapped to the same "bucket" or "box" with high probability in the hash space. In actual operation, the entire text collection needs to be preliminarily processed first, key features are extracted, and each text fragment is converted into a vector form of fixed length. Then, a shorter hash signature is generated for each vector through the MinHash algorithm, which can approximate the similarity between the original vectors in the form of probability. Subsequently, these signatures are mapped into a hash table using the LSH function, and text fragments with the same hash value are clustered together to form candidate segmentation points. Finally, by further analyzing and verifying these candidate segmentation points, the final text segmentation boundary is determined, and the purpose of segmenting long text into paragraphs or units with similar content is achieved. This example method is particularly suitable for processing large-scale text data because it can effectively identify semantic boundaries in the text while maintaining high efficiency. The segmented text data is then fed into a large language model, which uses prompt engineering to summarize the text fragments and simulate questions. Finally, the question data pairs are generated and stored in a CSV file for fine-tuning.

[0094] The above examples are combined to obtain a preferred example of the present invention, in which the method comprises the following steps:

[0095] S10: annotate the private dataset, generate text detection boxes and recognition labels, and use the formula editor to edit the formula to obtain the manually annotated private dataset; use the manually annotated private dataset to fine-tune the text recognition model pre-trained on the public dataset, calculate the loss function, and optimize the loss function based on the stochastic gradient descent method to complete the training of the text recognition model; use the trained text recognition model to extract text data from professional data source images;

[0096] S20: Perform statistical analysis on the frequency of use, distribution position, and ratio of punctuation marks to text in the standard text data; use a cluster analysis algorithm to perform pattern recognition on the statistical analysis data, and build a text feature model that can reflect the standard text features based on the cluster analysis results; use the text feature model to analyze the text data, and use regular expressions to match the garbled symbols in the text data, and perform cleaning operations, including replacing, deleting or correcting symbols that do not meet the standards, to achieve preprocessing of the text data;

[0097] S30: Use a variety of prompt engineering methods to guide the grammatical error correction model of the large language model to score the grammatical correctness of the text data, generate preference labels, and use them to indicate the degree to which the candidate items conform to the grammatical rules; train the reward model according to the preference labels, use the trained reward model to evaluate each candidate item generated by the large language model, assign a reward value to each candidate item, and use the reinforcement learning algorithm to calculate the gradient value of the expected value of the reward value, and update the parameters of the large language model based on the gradient value, so as to optimize the large language model; use the optimized large language model to identify and correct the grammar of the text data;

[0098] S40: The text data is converted into vector form using a large language model. The large language model calculates the perplexity of the sentence in combination with the sentence features calculated by the text feature model, and scores the sentence in combination with the perplexity, semantics, and punctuation distribution.

[0099] S50: Convert text data into vectors; generate hash signatures for vectorized text segments using the minimum hash algorithm; use local sensitive hashing to cluster text segments with hash signatures whose similarity is greater than a second threshold to form candidate segmentation points, thereby achieving text segmentation; input the segmented text into a large language model, and generate fine-tuning data in combination with prompts.

[0100] Through the above method, the present invention combines multiple technologies such as OCR technology, deep learning and large language models to generate high-quality instruction fine-tuning data, thereby solving the problem of lack of instruction data in building large language models in vertical fields.

[0101] An example of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method for generating fine-tuning data for a large industry model formed by any one of the above examples or a combination of multiple examples. The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0102] An example of the present invention also provides a storage medium, which has the same inventive concept as a method for generating fine-tuning data for an industry large model formed by any one of the above examples or a combination of multiple examples, and stores computer instructions thereon, which, when executed, execute the steps of the method for generating fine-tuning data for an industry large model formed by any one of the above examples or a combination of multiple examples.

[0103] Based on this understanding, the technical solution of this embodiment, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0104] An example of the present invention further provides a terminal, which has the same inventive concept as any example or a combination of multiple examples corresponding to the above-mentioned method for generating fine-tuning data for a large industry model, including a memory and a processor, wherein the memory stores computer instructions that can be run on the processor, and the processor executes the steps of the above-mentioned method for generating fine-tuning data for a large industry model when running the computer instructions. The processor can be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0105] In one example, the terminal, i.e., the electronic device, is presented in the form of a general-purpose computing device, and the components of the electronic device may include but are not limited to: at least one processing unit (processor) mentioned above, at least one storage unit mentioned above, and a bus connecting different system components (including storage units and processing units).

[0106] The storage unit stores a program code, which can be executed by the processing unit, so that the processing unit performs the steps of various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification. For example, the processing unit can perform the above-mentioned method for generating fine-tuning data for a large industry model.

[0107] The storage unit may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache memory unit, and may further include a read-only memory unit (ROM).

[0108] The storage unit may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0109] The bus may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0110] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0111] Through the above description, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to this exemplary embodiment can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method of the exemplary embodiment of the present application.

[0112] The above specific implementation methods are detailed descriptions of the present invention. It cannot be determined that the specific implementation methods of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions and substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for generating fine-tuning data for industry large models, characterized in that: The following steps are involved: Use text recognition models to extract text data from professional data source images; Preprocess the extracted text data based on statistical feature text filtering function; Use a large language model to identify and correct the grammar of text data; Use a large language model to calculate the perplexity of the text data, score the sentences based on the perplexity, and remove sentences with scores below the first threshold; The text data is segmented into text segments whose content similarity is greater than a second threshold, the segmented text is input into the large language model, and the fine-tuning data is generated in combination with the prompts.

2. The method for generating fine-tuning data for industry large models according to claim 1 is characterized in that: Use text recognition models to extract text data from professional data source images, including: Label the private data set, generate text detection boxes and recognition labels, and use the formula editor to edit the formula to obtain the manually labeled private data set; Use manually annotated private datasets to fine-tune the text recognition model pre-trained on the public dataset, calculate the loss function, and optimize the loss function based on the stochastic gradient descent method to complete the training of the text recognition model; Use the trained text recognition model to extract text data from images from professional data sources.

3. The method for generating fine-tuning data for industry large models according to claim 1 is characterized in that: The extracted text data is preprocessed based on the statistical feature text filtering function, including: Conduct statistical analysis on the frequency of use, distribution location, and ratio of punctuation marks to text in the standardized text data; Cluster analysis algorithm is used to perform pattern recognition on statistical analysis data, and a text feature model that can reflect standard text features is constructed based on the cluster analysis results; The text feature model is used to analyze the text data, and regular expressions are used to match the garbled symbols in the text data, and clean up operations are performed, including replacing, deleting or correcting symbols that do not meet the specifications.

4. The method for generating fine-tuning data for industry large models according to claim 1 is characterized in that: Use a large language model to identify and correct the grammar of text data, including: A variety of prompt engineering methods are used to guide the grammatical error correction model of the large language model to score the grammatical correctness of text data and generate preference labels to indicate the degree to which the candidate items conform to grammatical rules; The reward model is trained according to the preference labels, and each candidate generated by the large language model is evaluated using the trained reward model. A reward value is assigned to each candidate, and a reinforcement learning algorithm is used to calculate the gradient value of the expected value of the reward value. The parameters of the large language model are updated based on the gradient value, thereby optimizing the large language model. Use the optimized large language model to identify and correct the grammar of text data.

5. The method for generating fine-tuning data for industry large models according to claim 4 is characterized in that: The various prompt engineering methods include one or more of preface guidance, thought chain reasoning, and context learning; preface guidance provides task descriptions and scoring criteria for the large language model; thought chain reasoning first uses the large language model to explain the grammatical rationality of each candidate, and then generates a preference distribution based on the explanation; context learning enables the large language model to learn the application of grammatical rules by providing examples of completing manual error correction, rationality explanation and label annotation.

6. The method for generating fine-tuning data for industry large models according to claim 1 is characterized in that: Use a large language model to calculate the perplexity of text data, including: A large language model is used to convert text data into vector form. The large language model calculates the perplexity of the sentence based on the sentence features calculated by the text feature model, and scores the sentence based on the perplexity, semantics, and punctuation distribution.

7. The method for generating fine-tuning data for industry large models according to claim 1 is characterized in that: Segmenting the text data into text segments whose content similarity is greater than a second threshold value includes: Convert text data into vectors; Generate a hash signature for the vectorized text fragment using the minimum hash algorithm; Locally sensitive hashing is used to cluster text fragments with hash signatures whose similarity is greater than a second threshold to form candidate segmentation points, thereby achieving text segmentation.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for generating fine-tuning data for industry large models as described in any one of claims 1 to 7 are implemented.

9. A terminal comprising a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, characterized in that: When the processor runs the computer instructions, it executes the steps of the method for generating fine-tuning data for industry large models as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Deep learning text character detection method based on image moment correction

    CN112580507A

  • Error correction method and device for Chinese text in power field, storage medium and computing equipment

    CN114118065A

  • Natural language artificial intelligence system and method for water and electricity operation and maintenance knowledge

    CN115563968A

  • Text processing method and device, text retrieval method and device, electronic equipment and storage medium

    CN116028618A

  • Industrial data knowledge extraction method and device, computer equipment and storage medium

    CN116956923A

Cited By

  • Large model training method, model, system and equipment based on scoring mechanism

    CN119357675A

  • Numerical control file analysis method and equipment for deep sea connector and medium

    CN119576875A

  • Port knowledge abstract generation method based on large model

    CN119808932A

  • Large language model knowledge correction method based on adaptive comparative learning

    CN119808952A

  • A large language model knowledge correction method based on adaptive contrast learning

    CN119808952B