A fully unsupervised large language model fine-tuning training platform

Abstract

This invention discloses a fully unsupervised large language model fine-tuning training platform, comprising a source dataset management module for processing uploaded text and generating data suitable for fine-tuning training; a model fine-tuning training module for selecting models for fine-tuning and evaluating the trained models; and a data inference generation module for inputting relevant data into the corresponding models based on task requirements and scoring and reviewing the data generated by the models. This invention significantly reduces the manpower and time costs in the data preparation stage for large model fine-tuning training. The pre-trained data used for inference of large models exhibits excellent actual performance, generating data that surpasses data manually annotated by ordinary business personnel, which greatly ensures the quality of model fine-tuning training.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a fully unsupervised large language model fine-tuning training platform. Background Technology

[0002] When general-purpose large language models are used in real-world vertical business scenarios, their general capabilities can sometimes become an obstacle. This is because while general-purpose large language models possess extremely high language capabilities and a certain level of reasoning ability, they have very little knowledge of the specific professional knowledge of the vertical domain, making it difficult to provide accurate and professional answers effectively. Real-world business scenarios require domain-specific large models with industry know-how, capable of providing professional knowledge results that are reliable, controllable, and secure in terms of data and privacy. There are three feasible paths to achieve the above functions: building a large model in-house, using prompts, and fine-tuning a large model. Building a large model in-house involves feeding the model with domain documents directly for pre-training in an unsupervised manner. This is suitable for situations where computing power, funding, time, and data are extremely abundant. For example, OpenAI's GPT project initially invested $800 million and trained for five years from GPT1 to GPT4. The chip requirement was over 30,000 NVIDIA A100 GPUs. Prompts organize domain knowledge into prompts, injecting knowledge into the model through input prompts. This is suitable for domains with very little professional knowledge. However, the length of prompts that a large model can accept has an upper limit (approximately 128KB), and the fine-tuning effect is very limited. Fine-tuning a large model involves injecting supervised question-and-answer pairs (QPs) into the model based on a general pre-trained model, targeting specific tasks or domains. This is suitable for scenarios with domain-knowledge-generated QPs, sufficient computing power, and financial support (one ten-thousandth of the cost of self-built systems). Fine-tuning involves generating QPs from domain knowledge and then injecting them into the large model. (Depending on the complexity of the knowledge, the amount of QP data can range from hundreds of thousands to millions). Using labeled data shortens training time. Therefore, fine-tuning a general large model for practical application is the most feasible approach. Fine-tuning large language models involves prompt word fine-tuning, supervised fine-tuning, instruction fine-tuning, low-rank fine-tuning, and optimizing the language model based on human feedback using reinforcement learning. A fine-tuned large model can then demonstrate the ability to solve specific tasks in a professional domain, offering hope for practical business applications. However, achieving fine-tuning requires three key elements:

[0003] 1. Hardware computing power: speeds up training time (does not affect model performance);

[0004] 2. Question-answer pair data: A key factor affecting the accuracy and reliability of the model. It requires hundreds of thousands to millions of pieces of supervised data.

[0005] 3. Base model: Currently, apart from self-built models, the base models used by various manufacturers are similar.

[0006] Computing power can be solved with funding; algorithmic foundations can be built using general models; however, the "question-answer pair data" remains a major challenge, consuming significant resources, manpower, and time. Taking OpenAI's fine-tuning of the GPT-3 model as an example, the fine-tuning phase requires hundreds to tens of millions of text samples with annotations (the exact number depends on the complexity of the task and the domain specificity). In a specific domain, the annotation time required to generate fine-tuning data from 100,000 text samples is: 5-6 experienced annotators, 6-8 weeks. For enterprise fine-tuning training, because business experts are involved, the time and cost increase exponentially. For example, in a domain-specific question-answering system, given existing industry documentation, compiling 200,000 question-answer training data requires: if reading 1500-word documents to generate 5 questions, then at most 10 documents can be read and 50 questions generated per hour. Completing 200,000 question-answer data samples would require 4000 hours, or 500 days. The required cost is: 5000 RMB per person-day (business expert) * 500 RMB (assuming an 8-hour workday with no breaks) = 2.5 million RMB. Furthermore, continuous data updates require ongoing investment from business experts, and the training cycle is extremely long, which cannot meet business needs. Therefore, the method for preparing the data needed for fine-tuning training needs to be changed from requiring 100% manual annotation to a review-based approach, combining both to significantly reduce time and manpower costs. Summary of the Invention

[0007] To address the shortcomings mentioned above, this invention provides a fully unsupervised large language model fine-tuning training platform.

[0008] To achieve the above objectives, this invention provides a fully unsupervised large language model fine-tuning training platform, comprising:

[0009] The source dataset management module is used to process uploaded text and generate data suitable for fine-tuning training.

[0010] The model fine-tuning training module is used to select the model for fine-tuning training and to test and evaluate the trained model.

[0011] The data reasoning and generation module is used to input the relevant data into the corresponding model based on task requirements, and to score and review the data generated by the model.

[0012] The source dataset management module includes a data management submodule and a data processing submodule. The data management submodule has functions for adding, deleting, modifying, and querying the uploaded text. The data processing submodule includes OCR parsing and text paragraph segmentation. The OCR parsing includes converting documents of different formats into text data. The text paragraph segmentation includes initially segmenting the text data to obtain multiple paragraphs; using a text similarity calculation algorithm to extract the topic of each paragraph; if the topics of the paragraphs are the same, they are not segmented; if the topics of the paragraphs are different, they are segmented according to the topics; and the segmented paragraphs are further segmented according to the model's understanding length.

[0013] Preferably, the text data is initially segmented to obtain multiple paragraphs, including:

[0014] Determine whether the text data contains a chapter format;

[0015] If it exists, the text data will be initially segmented according to chapters;

[0016] If the keyword does not exist, then the text data is segmented using keywords.

[0017] Preferably, keyword segmentation of the text data includes:

[0018] The text data is divided into fixed-length blocks to obtain text blocks of fixed length.

[0019] Keywords were extracted from each text block using the TF-IDF algorithm;

[0020] Calculate the overlap of keywords between each text block. If the overlap reaches a preset threshold, merge the text blocks into a paragraph.

[0021] Preferably, the method of using a text similarity calculation algorithm to extract the topic of each paragraph includes:

[0022] Multiple keywords were extracted from each paragraph using the TF-IDF algorithm;

[0023] The paragraphs and multiple keywords are converted into vectors using the bag-of-words model;

[0024] The cosine similarity between each paragraph vector and each keyword vector is calculated using a cosine similarity algorithm.

[0025] If the cosine similarity scores are all greater than a preset threshold, then the paragraphs have the same theme.

[0026] Otherwise, the paragraphs have different themes.

[0027] Preferably, segmentation according to the topic includes:

[0028] Obtain keywords whose cosine similarity is greater than a preset threshold, and extract the sentences containing the keywords;

[0029] The sentence and the text between the sentences are merged into one sub-paragraph, and the remainder is divided into another sub-paragraph;

[0030] Repeat the similarity calculation for each sub-paragraph until all sub-paragraphs have the same topic.

[0031] Preferably, the cosine similarity algorithm formula is:

[0032]

[0033] In the formula: A and B are two text vectors, A i and B i These are the i-th elements of vectors A and B, respectively, and n is the number of elements in the vectors.

[0034] Preferably, the formula for the bag-of-words model is:

[0035]

[0036] In the formula: BoW(w) i Let w be the value of the i-th element in vector w. i Let be the i-th word, and n be the total number of words in the paragraph; |w j =v i |If the j-th word in w equals v i If the value is 1, then the value is 1; otherwise, it is 0.

[0037] Preferably, the model fine-tuning training module includes a base large model management submodule, a fine-tuning large model management submodule, and a fine-tuning training submodule. The base large model management submodule includes an open-source large model, and the fine-tuning large model management submodule includes models obtained through fine-tuning training, including dedicated question-answering reasoning large models, dedicated summarizing reasoning large models, dedicated task planning reasoning large models, dedicated domain translation large models, and dedicated style writing reasoning large models. The fine-tuning training submodule is used to test and evaluate the trained models.

[0038] Preferably, the data reasoning generation module includes a reasoning task management submodule and a data review submodule. The reasoning task management submodule has functions for adding, deleting, modifying, and querying reasoning tasks, and the data review submodule is used to score and review the accuracy of the generated data.

[0039] Preferably, it also includes a fine-tuning model application module and a team collaboration management module.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] This invention significantly reduces the manpower and time costs in the data preparation stage for fine-tuning training of large models. The pre-trained data used for inference of large models has excellent actual performance, and the generated data surpasses the data manually compiled and labeled by ordinary business personnel. This greatly ensures the quality of model fine-tuning training. It completes the generation, review, fine-tuning training, deployment, and online application of large model training data in one stop. The entire process is highly efficient and automated, which can accelerate the application of large models in various vertical fields and help traditional enterprises accelerate their intelligent transformation. Attached Figure Description

[0042] Figure 1 This is a flowchart of the fully unsupervised large language model fine-tuning training platform of this invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Reference Figure 1 This invention provides a fully unsupervised large language model fine-tuning training platform, comprising:

[0045] The source dataset management module is used to process uploaded text and generate data suitable for fine-tuning training.

[0046] The model fine-tuning training module is used to select the model for fine-tuning training and to test and evaluate the trained model.

[0047] The data reasoning and generation module is used to input relevant data into the corresponding model based on task requirements, and to score and review the data generated by the model.

[0048] The source dataset management module includes a data management submodule and a data processing submodule. The data management submodule has functions for adding, deleting, modifying, and querying uploaded text. The data submodule includes OCR parsing and text segmentation. OCR parsing includes converting documents of different formats into text data, and text segmentation includes:

[0049] Obtain the text data to be segmented;

[0050] Specifically, the text data to be segmented is uploaded. In addition, the text data may also include a description of key information about the file, which can be used to assist in text segmentation.

[0051] The text data is initially segmented to obtain multiple paragraphs;

[0052] Specifically, the system first determines whether the text data has a clear structured format of chapters. If it does, the text data is initially segmented according to chapters, and the text is output in chapter units. If it does not, the text data is segmented by keywords.

[0053] Furthermore, algorithms are used to extract keywords from the entire text, or similarity probability matching is performed using provided key information. When relevant keywords appear or the matching probability reaches a threshold, the entire paragraph is extracted. The output consists of multiple paragraph units containing key information.

[0054] Furthermore, keyword segmentation of text data includes:

[0055] The text data is divided into fixed-length blocks to obtain text blocks of fixed length.

[0056] The TF-IDF algorithm is used to extract keywords from each text block. There are multiple keywords, which will be used to evaluate the relevance between text blocks.

[0057] Calculate the overlap of keywords between each text block. If the overlap reaches a preset threshold, merge the text blocks into a paragraph.

[0058] In this embodiment, the calculation formula of the TF-IDF algorithm is as follows:

[0059] TF (Term Frequency): Term frequency is the frequency with which a word appears in a document.

[0060] The calculation formula is:

[0061]

[0062] IDF (Inverse Document Frequency): Inverse document frequency is a measure of how much information a word provides.

[0063] The calculation formula is:

[0064]

[0065] TF-IDF value: is the product of these two quantities.

[0066] The calculation formula is:

[0067] TFIDF(t) = TF(t) × IDF(t).

[0068] If the keyword overlap reaches 30% or higher, the two text blocks are considered to have a high degree of thematic relevance.

[0069] The topic of each paragraph is extracted using a text similarity calculation algorithm. If the topics of paragraphs are the same, they are not segmented; if the topics of paragraphs are different, they are segmented according to their topics.

[0070] Specifically, the TF-IDF algorithm is used to extract multiple keywords from each paragraph;

[0071] Transform paragraphs and multiple keywords into vectors using the bag-of-words model;

[0072] The cosine similarity algorithm is used to calculate the cosine similarity between each paragraph vector and each keyword vector.

[0073] If the cosine similarity is greater than the preset threshold, then the paragraphs have the same theme;

[0074] Otherwise, the paragraphs have different themes;

[0075] Keywords with a cosine similarity greater than a preset threshold are identified, and the sentences containing these keywords are extracted.

[0076] Combine sentences and the text between sentences into one sub-paragraph, and divide the remainder into another sub-paragraph;

[0077] Repeat the similarity calculation for each sub-paragraph until all sub-paragraphs share the same topic.

[0078] In this embodiment, the cosine similarity algorithm formula is:

[0079]

[0080] In the formula: A and B are two text vectors, A i and B i These are the i-th elements of vectors A and B, respectively, and n is the number of elements in the vectors.

[0081] The formula for the bag-of-words model is:

[0082]

[0083] In the formula: BoW(w) i Let w be the value of the i-th element in vector w. i Let be the i-th word, and n be the total number of words in the paragraph; |w j =v i |If the j-th word in w equals v i If the value is 1, then the value is 1; otherwise, it is 0.

[0084] In this embodiment, a context-preserving length parameter of 300 characters is set to determine whether to preserve or separate the connection between a paragraph and the previous paragraph. The application of this parameter enhances the flexibility and semantic coherence of the text segmentation process, which is crucial for improving the subsequent language model processing performance.

[0085] Specifically, paragraphs on the same topic are kept intact, while paragraphs on different topics are split into smaller parts. The length parameter is maintained in combination with the given context. If the parameter is less than the paragraph length, the link between the paragraph and the previous paragraph is broken. If the parameter is greater than the paragraph length, the link between the paragraph and the previous paragraph is maintained.

[0086] The segmented paragraphs are then divided according to the model's understanding length.

[0087] Specifically, if the input consists of multiple paragraphs, it is segmented into individual paragraphs. If the input is a single paragraph, no additional segmentation is performed. If the paragraph exceeds the model's comprehension length, it is segmented into appropriate lengths from top to bottom. The first segment is directly input into the large model, and then these segmented texts are combined and input into the large model again according to the length that the large model can understand. Alternatively, while ensuring semantic coherence, individual paragraphs are segmented by periods. If a sentence exceeds the model's comprehension length, the sentence is also segmented according to natural breakpoints.

[0088] This invention combines keyword extraction, text similarity calculation, and context length parameters to bring about a novel text preprocessing approach. This method not only excels in maintaining the semantic coherence of text but also significantly improves the accuracy and efficiency of text processing through flexible and precise segmentation strategies. Especially when dealing with texts that lack clear structure or distinct chapter divisions, our system can intelligently analyze the text's theme and structure, achieving more detailed and efficient segmentation. Further processing of each segmented paragraph or sentence ensures that these text units can be effectively understood and processed by large language models. This method not only solves key problems in text preprocessing and paragraph segmentation in traditional techniques but also provides higher-quality input data for large language models. This is significant for improving the overall performance of NLP technology and significantly promotes the application and development of language models in a wider range of fields.

[0089] In this embodiment, OCR parsing is used to convert document formats such as PDF and JPG, which are inconvenient for large models to process, into text documents, a basic text format that is easier for large models to process. The data inference generation module includes two sub-modules: inference task management and generated data review. It uses various pre-trained data inference generation models, including: a dedicated question-answering inference model (automatically inferring large-scale, high-quality question-answer pairs based on text paragraphs in the source dataset; this question-answer pairs are the data format needed for fine-tuning the domain-specific question-answering model); a dedicated summary inference model (automatically inferring large-scale, high-quality summary data based on text paragraphs in the source dataset to train the summary generation model); a dedicated task planning inference model (automatically inferring large-scale, high-quality task planning data based on existing business system data, such as CSV files, to train the task planning model); a dedicated domain translation model (automatically inferring large-scale, high-quality original text and translation data based on Chinese and foreign language text in the source dataset to train the dedicated domain translation model); a dedicated style writing inference model (automatically inferring and transcribing large-scale, high-quality dedicated style text data based on specific styles in the source dataset to train the dedicated style writing model); and other pre-trained large models, covering as many business scenarios as possible. The inference task management module includes functions for adding, deleting, modifying, and querying inference tasks. It allows users to create inference tasks, select source datasets, configure task parameters, choose specific domains, and input personalized prompts. These elements are considered when generating question-and-answer pairs for the large model. The generated data review module includes accuracy and scoring checks. Human intervention prioritizes data accuracy; compared to purely manual annotation, this combined review and scoring approach is over 100 times more efficient. Scoring the generated data is beneficial for subsequent model fine-tuning and training, effectively aligning the model to high-scoring data. The fine-tuning training module comprises three sub-modules: the base large model management sub-module, the fine-tuning large model management sub-module, and the fine-tuning training sub-module. The fine-tuning training process is encapsulated as much as possible as a black box, allowing users to complete training with simple configuration. The base large model management sub-module contains commercially available open-source large models from both domestic and international sources. The fine-tuning large model management sub-module contains models obtained through user fine-tuning or multiple rounds of fine-tuning training. Users can choose any model from these two libraries for a new round of fine-tuning training, requiring the selection of a training dataset from the data inference generation module. The fine-tuning training submodule can use multiple evaluation sets to objectively evaluate the fine-tuning effect.

[0090] This platform also includes a fine-tuning model application module, which comprises two sub-modules: model deployment and model application. Model deployment supports both cloud and local deployment, providing users with flexible application options. When users choose to deploy on the platform in the cloud, they can interact with the model in the model application module. The interaction format is similar to ChatGPT, using input boxes for intelligent question-and-answer dialogue.

[0091] This platform also includes a team collaboration management module, comprising three sub-modules: team member management, collaborative review, and collaborative applications. This facilitates team users' use of the platform as a productivity tool. Team member management includes CRUD (Create, Read, Update, Delete) functions for team members, allowing users to set team member roles and permissions. Collaborative review primarily functions within the data inference generation module's data review sub-module, enabling multiple users to collaborate on reviews, each completing a portion, thus accelerating the review process and facilitating faster entry into the training phase. Collaborative applications allow team members to fine-tune the trained large model, deploying it in the cloud and making it available within the team. This allows all team members to share the same fine-tuned large model, making collaboration smoother.

[0092] In summary, the product platform based on the above functional modules has a closed-loop capability covering the entire process from training data generation, simple configuration and execution of fine-tuning training, deployment and application of fine-tuned large models, fine-tuning of large model training with incremental data training, application and iteration, which can effectively solve the problems mentioned above.

[0093] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A fully unsupervised large language model fine-tuning training platform, characterized in that, include: The source dataset management module is used to process uploaded text and generate data suitable for fine-tuning training. The model fine-tuning training module is used to select the model for fine-tuning training and to test and evaluate the trained model. The data reasoning and generation module is used to input the relevant data into the corresponding model based on task requirements, and to score and review the data generated by the model. The source dataset management module includes a data management submodule and a data processing submodule. The data management submodule has functions for adding, deleting, modifying, and querying the uploaded text. The data processing submodule includes OCR parsing and text segmentation. The OCR parsing includes converting documents of different formats into text data. The text segmentation includes initially segmenting the text data to obtain multiple paragraphs; using a text similarity calculation algorithm to extract the topic of each paragraph; if the topics of the paragraphs are the same, they are not segmented; if the topics of the paragraphs are different, they are segmented according to the topics; and the segmented paragraphs are further segmented according to the model's understanding length. The text data is initially segmented to obtain multiple paragraphs, including: Determine whether the text data contains a chapter format; If it exists, the text data will be initially segmented according to chapters; If not, then the text data is segmented by keywords; Keyword segmentation of the text data includes: The text data is divided into fixed-length blocks to obtain text blocks of fixed length. Keywords were extracted from each text block using the TF-IDF algorithm; Calculate the overlap of keywords between each text block. If the overlap reaches a preset threshold, merge the text blocks into a paragraph. The text similarity calculation algorithm is used to extract the topic of each paragraph, including: Multiple keywords were extracted from each paragraph using the TF-IDF algorithm; The paragraphs and multiple keywords are converted into vectors using the bag-of-words model; The cosine similarity algorithm is used to calculate the cosine similarity between each paragraph vector and each keyword vector. If the cosine similarity scores are all greater than a preset threshold, then the paragraphs have the same theme. Otherwise, the paragraphs have different themes; Segmentation according to the aforementioned topic includes: Obtain keywords whose cosine similarity is greater than a preset threshold, and extract the sentences containing the keywords; The sentence and the text between the sentences are merged into one sub-paragraph, and the remainder is divided into another sub-paragraph; Repeat the similarity calculation for each sub-paragraph until all sub-paragraphs have the same topic; The formula for the cosine similarity algorithm is: In the formula: A and B are two text vectors, and These are the i-th elements of vectors A and B, respectively, and n is the number of elements in the vectors; The model fine-tuning training module includes a base large model management submodule, a fine-tuning large model management submodule, and a fine-tuning training submodule. The base large model management submodule includes an open-source large model, and the fine-tuning large model management submodule includes models obtained through fine-tuning training, including dedicated question-answering reasoning large models, dedicated summarizing reasoning large models, dedicated task planning reasoning large models, dedicated domain translation large models, and dedicated style writing reasoning large models. The fine-tuning training submodule is used to test and evaluate the trained models. The data inference generation module generates a large model by using various pre-trained data inference methods, including: Dedicated question-answering reasoning large model: Automatically infers and generates large-scale, high-quality question-answer sentence pairs based on text paragraphs in the source dataset. The question-answer sentence pairs data is the data format required for fine-tuning and training the domain-specific question-answering large model; Dedicated large-scale summary inference model: Automatically infers and generates large-scale, high-quality summary data based on text paragraphs in the source dataset, which is used to train the large-scale summary generation model; Dedicated task planning reasoning model: Based on existing business system data, it automatically infers and generates large-scale, high-quality task planning data to train the task planning model. Domain-Specific Translation Model: Based on the source dataset, large-scale, high-quality original text and translation data are automatically generated through Chinese and foreign language reasoning to train a domain-specific translation model. Large-scale inference model for dedicated style writing: Based on the specific style in the source dataset, a large-scale, high-quality dedicated style text data is automatically transcribed and used to train a large-scale dedicated style writing model; The inference task management module is used to create inference tasks, select source datasets, configure task parameters, select specific domains, and input personalized prompts. These will all be considered when the large model generates question-answering pair inferences.

2. The fully unsupervised large language model fine-tuning training platform according to claim 1, characterized in that, The formula for the bag-of-words model is: In the formula: Let i be the value of the i-th element in vector w. Let be the i-th word, and n be the total number of words in the paragraph; If the j-th word in w equals If the value is 1, then the value is 1; otherwise, it is 0.

3. The fully unsupervised large language model fine-tuning training platform according to claim 1, characterized in that, The data reasoning generation module includes a reasoning task management submodule and a data review submodule. The reasoning task management submodule has functions for adding, deleting, modifying, and querying reasoning tasks, while the data review submodule is used to score and review the accuracy of the generated data.

4. The fully unsupervised large language model fine-tuning training platform according to claim 1, characterized in that, It also includes a fine-tuning model application module and a team collaboration management module.

Images