General-purpose model-based and transfer learning-based private knowledge content generation method

Abstract

The application relates to a private knowledge content generation method based on a general large model and transfer learning. The method comprises the following steps: acquiring a structured database, performing foreign key association between tables on structured relational data in the structured database to obtain table association data; processing the table association data to obtain a text data set and an instruction data set, respectively, performing lora fine-tuning on the general large model by using the text data set and the instruction data set to obtain a private knowledge large model; acquiring real-time private data, constructing a reference material vector library according to a plurality of segmented text vectors corresponding to the real-time private data, selecting a reference material vector closest to an input vector corresponding to user input from the reference material vector library to obtain corresponding reference material; and splicing the user input and the reference material according to an instruction template, inputting the spliced result into the private knowledge large model to obtain corresponding generated content. The method can improve the real-time performance and effectiveness of the generated content.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method for generating private knowledge content based on a general knowledge big model and transfer learning. Background Technology

[0002] Currently, pre-trained large-scale language models such as OpenAI's ChatGPT, Baidu's Wenxin Yiyan, Kunlun Wanwei's Tiangong 3.5, and Alibaba's Tongyi Qianwen all employ hundreds of billions of parameters and trillions of token characters, pre-trained using techniques such as instruction fine-tuning and reinforcement learning, demonstrating excellent content generation capabilities. In vertical domains, many excellent transfer models have also emerged, such as the DoctorGLM Chinese medical consultation model trained on ChatGLM-6B and Chinese medical dialogue data, the LexiLaw Chinese legal large-scale model trained on Chinese-LLaMA and Chinese legal dialogue question-and-answer data, and the XuanYuan Chinese financial large-scale model trained on BLOOM and publicly available Chinese financial data.

[0003] However, large-scale pre-trained language models such as ChatGPT and Wenxin Yiyan, as well as large-scale vertical private knowledge models such as DoctorGLM and LexiLaw, are all trained on data specific to their respective domains. Lacking enterprise private knowledge, these models cannot truly provide professional and reliable answers to enterprise questions and content generation. Traditional large-scale general knowledge models mostly use plain text data or manually compiled instruction data for domain fine-tuning. Enterprise private knowledge, on the other hand, is mostly structured data and private relational data stored in relational databases such as Oracle and MySQL. Traditional large-scale general knowledge models cannot directly train on this high-value private data. Furthermore, traditional large-scale general knowledge models are slow to update real-time information and cannot effectively distinguish between new and old knowledge. Therefore, they cannot perform corresponding inference, prediction, and content generation based on the latest facts, requiring costly retraining for each information update. Since enterprise content generation has high real-time requirements, traditional large-scale general knowledge models cannot truly meet the needs of enterprise users. Summary of the Invention

[0004] Therefore, it is necessary to provide a method for generating private knowledge content based on a general knowledge big model and transfer learning to address the above-mentioned technical problems.

[0005] A method for generating private knowledge content based on a general knowledge big data model and transfer learning, the method comprising:

[0006] Obtain the structured database corresponding to the enterprise private data required by the current business model, and perform foreign key associations between tables in the structured relational data in the structured database to obtain the table association data;

[0007] The table-related data is processed using the sql2text method and the sql2Instruction method respectively to obtain a text dataset and an instruction dataset. The pre-trained general knowledge model is then fine-tuned using LoRa with the text dataset and the instruction dataset to obtain a private knowledge model.

[0008] Acquire real-time private data, construct a reference material vector library based on several segmented text vectors corresponding to the real-time private data, acquire user input, perform vector transformation on the user input to obtain an input vector, select the reference material vector from the reference material vector library that is closest to the input vector, and obtain the reference material corresponding to the user input;

[0009] The user input and the reference materials are concatenated according to the instruction template, and the concatenated result is input into the private knowledge big model to obtain the corresponding generated content.

[0010] In one embodiment, the method further includes: obtaining a pre-set pair of related fields; the pair of related fields is related to the business model provided by the private knowledge big data model; and performing foreign key associations between tables on the structured relational data containing the key field pairs to obtain table-related data.

[0011] In one embodiment, the method further includes: exporting the table association data into SQL statements, constructing an instruction template set based on a variety of pre-set question and answer methods, and filling data according to the SQL statements and the instruction template set to obtain an instruction dataset.

[0012] In one embodiment, the method further includes: obtaining a hybrid dataset based on the text dataset and the instruction dataset; decomposing the parameters to be trained into a dimensionality-reduced matrix and an dimensionality-increasing matrix, and obtaining the parameters of a private knowledge model based on the parameters of the general knowledge model and the parameters to be trained; inputting the hybrid dataset into the general knowledge model, and iteratively training the parameters of the private knowledge model until the difference between the model output and the true answer is minimized, and then stopping the iteration to obtain the private knowledge model.

[0013] In one embodiment, the method further includes: converting the real-time private data into text using the sql2text method, dividing the text into several natural segments; converting each natural segment into a multi-dimensional vector to obtain a reference material vector, and obtaining a reference material vector library based on the reference material vectors corresponding to the several natural segments.

[0014] In one embodiment, the method further includes: calculating the cosine similarity between the input vector and reference material vectors in the reference material vector library; obtaining the reference material vector most similar to the input vector based on the relationship between the cosine similarity of each reference material vector and a preset threshold; and obtaining the reference material corresponding to the user input based on the natural segment corresponding to the most similar reference material vector.

[0015] In one embodiment, the method further includes: obtaining the maximum input length and similarity difference interval supported by the private knowledge large model; the similarity difference interval includes a minimum difference and a maximum difference; sorting the cosine similarity corresponding to each reference material vector in descending order to obtain a similarity list; traversing each cosine similarity in the similarity list, calculating the similarity difference between the current cosine similarity and the previous sorted similarity that meets preset conditions, and obtaining the reference material corresponding to the user input based on the relationship between the similarity difference and the similarity difference interval; the total length of the user input and the reference material is less than the maximum input length.

[0016] In one embodiment, the method further includes: if the similarity difference is within the similarity difference range, then the reference material corresponding to the current ranking similarity is retained.

[0017] In one embodiment, the method further includes: if the similarity difference is less than the minimum difference, then discarding the reference material corresponding to the current ranking similarity.

[0018] In one embodiment, the method further includes: if the similarity difference is greater than the maximum difference, discarding the reference material vector corresponding to the current ranking similarity and using the reference material corresponding to the previous ranking similarity that meets the preset conditions as the reference material corresponding to the user input.

[0019] The aforementioned method for generating private knowledge content based on a general knowledge model and transfer learning trains the general knowledge model using domain-specific private knowledge to obtain a private knowledge model. It acquires real-time private data, constructs a reference material vector library based on segmented text vectors corresponding to the real-time private data, obtains user input, performs vector transformation on the user input to obtain an input vector, selects the reference material vector from the reference material vector library that is closest to the input vector to obtain the reference material corresponding to the user input, and concatenates the user input and reference material according to an instruction template. The concatenated result is then input into the private knowledge model to obtain the corresponding generated content. This embodiment of the invention can improve the real-time performance and effectiveness of the generated content. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a method for generating private knowledge content based on a general knowledge big model and transfer learning in one embodiment.

[0021] Figure 2 This is a schematic diagram of the reasoning process of a private knowledge big model in one embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] In one embodiment, such as Figure 1 As shown, a method for generating private knowledge content based on a general knowledge model and transfer learning is provided, including the following steps:

[0024] Step 102: Obtain the structured database corresponding to the enterprise private data required by the current business model, and perform foreign key associations between tables in the structured relational data in the structured database to obtain the table association data.

[0025] Private knowledge refers to proprietary knowledge assets accumulated within an enterprise, including both internal experience and skills, as well as internal business intelligence. A large proportion of enterprise private knowledge is stored in structured databases such as Oracle and MySQL in the form of structured data, such as product parameter data, descriptive data, development trend data, customer data, financial record data, etc., existing in the form of relationships and attributes. Foreign key relationships are established between tables based on the related fields corresponding to the business model, which includes predictive analysis of future product parameters. Product parameters include technical parameters and descriptive parameters. Predictive analysis refers to the evolution of new product parameters based on the technical and descriptive parameters of historical products. Descriptive parameters include the product's applicable scope and research direction. In predictive analysis mode, the "previous model" and "modified model" fields of a product can be linked, so that in the same data, one can know the parameters of the current product as well as the parameters of its predecessor and modified products. The implementation methods of other business models are similar to those of predictive analysis mode, and will not be elaborated here. Step 104: The table-related data is processed using the sql2text method and the sql2Instruction method respectively to obtain the text dataset and the instruction dataset. The pre-trained general knowledge model is then fine-tuned using LoRa with the text dataset and the instruction dataset to obtain the private knowledge model.

[0026] General knowledge large models refer to machine learning models with a very large number of parameters. These models are usually trained using deep learning methods. General knowledge large models are applied to general domains and have not been incrementally trained on specific industry domain data. Examples include ChatGLM, ChatGLM2, MOSS, and LLAMA.

[0027] During the training of the general knowledge model, the table-related data is transformed into a text dataset using the sql2text method for incremental pre-training. This allows for the acquisition of more enterprise-specific knowledge. Based on the content of enterprise development, the required question and answer formats are generated, and a set of instruction templates is manually constructed. Then, the table-related data is transformed into an instruction dataset using the sql2Instruction method for fine-tuning the general knowledge model. This allows for a better understanding of the intent of enterprise users' input questions. Based on LoRa technology, combined with the text dataset and instruction dataset, the general knowledge model is further fine-tuned for specific domains.

[0028] Step 106: Obtain real-time private data, construct a reference material vector library based on several segmented text vectors corresponding to the real-time private data, obtain user input, perform vector transformation on the user input to obtain the input vector, select the reference material vector that is closest to the input vector from the reference material vector library, and obtain the reference material corresponding to the user input.

[0029] Real-time private data refers to highly real-time private data within the current business model. This means the data used for fine-tuning is not included in the training data used to train the general knowledge model. Furthermore, real-time private data is the latest available business data, and the specific content of the business data is determined based on the research objectives. User input includes questions or instructions from the user.

[0030] Step 108: Combine the user input and reference materials according to the instruction template, and input the combined result into the private knowledge big model to obtain the corresponding generated content.

[0031] Content generation technology refers to the use of artificial intelligence to automatically generate articles, news, reports, guides, and other content related to a specific field. This technology can automatically generate a large amount of high-quality content, providing timely, accurate, and useful information to practitioners in the field, improving the efficiency of information dissemination and industry competitiveness. In this invention, the technology can generate content describing future product parameters, enabling enterprise users to use these parameters to assist in the production of next-generation products and understand the possible parameters of the next-generation products, allowing them to deploy other products compatible with the next-generation products in advance.

[0032] In the aforementioned method for generating private knowledge content based on a general knowledge big model and transfer learning, a private knowledge big model is obtained by training the general knowledge big model using domain private knowledge. Real-time private data is acquired, and a reference material vector library is constructed based on several segmented text vectors corresponding to the real-time private data. User input is obtained, and the user input is vectorized to obtain an input vector. The reference material vector closest to the input vector is selected from the reference material vector library to obtain the reference material corresponding to the user input. The user input and the reference material are concatenated according to an instruction template, and the concatenated result is input into the private knowledge big model to obtain the corresponding generated content. This embodiment of the invention utilizes highly real-time domain data as reference material, effectively improving the problems of low real-time performance and inability to distinguish between new and old knowledge in traditional general knowledge big models.

[0033] In one embodiment, the step of performing foreign key associations between tables of structured relational data in a structured database to obtain table-related data includes: obtaining pre-set association field pairs; the association field pairs being related to the business model provided by the private knowledge big data model; and performing foreign key associations between tables of structured relational data containing key field pairs to obtain table-related data.

[0034] Specifically, taking predictive analytics as an example, foreign key relationships are established between tables in Oracle and MySQL structured databases, primarily through the "Previous Model" and "Modified Model" fields. This allows the same data record to provide information on the parameters, descriptions, and development trends of the current product, as well as the parameters, descriptions, and development trends of its predecessor and modified products.

[0035] In one embodiment, the step of processing table-related data using the sql2Instruction method to obtain a text dataset includes: exporting the table-related data into SQL statements, and populating the data according to the SQL statements and a pre-built transformation template to obtain the text dataset.

[0036] Specifically, the sql2text method converts the table-related data in a structured database into a natural language text dataset that the model can train on. First, the structured table-related data is exported as SQL statements. For example, exporting the data for product "Ship 1" yields the following SQL statement:

[0037] INSERT INTO table_name(“Category”,“Length”,...,“Development Trend”,“Previous Type-Category”,“Previous Type-Length”,...,“Previous Type-Development Trend”,“Modified Type-Category”,“Modified Type-Length”,...,“Modified Type-Development Trend”) VALUES

[0038] (value1,value2,...,valuen,previous-value1,previous-value2,...,previous-valuen,modified-value1,modified-value2,...,modified-valuen)

[0039] Then, construct a conversion template, for example:

[0040] Ship 1, xxxxxx (brief product description), has the following basic parameters: Length: xxx; Width: xxx; ...; Application scenario: xxx; Development trend: xxx. The predecessor of Ship 1 is Ship 10, whose basic parameters are as follows: ... The modified version of Ship 1 is Ship 12, whose basic parameters are as follows: ...

[0041] Finally, based on the exported SQL statements and conversion templates, data is populated to complete the construction of the text dataset.

[0042] In one embodiment, the step of processing table-related data using the sql2Instruction method to obtain an instruction dataset includes: exporting the table-related data into SQL statements; constructing an instruction template set based on various pre-set question and answer methods; and populating the data according to the SQL statements and the instruction template set to obtain the instruction dataset.

[0043] Specifically, the sql2Instruction method converts the table association data in the structured database into an instruction dataset that the model can train. First, the structured table association data is exported as SQL statements, and the process is the same as the corresponding steps for generating the text dataset.

[0044] Then, based on the question and answer formats required for content generation in enterprise development, a set of instruction templates is manually constructed. To enrich the diversity of question and answer formats during content generation, multiple instruction templates are manually constructed. For example, when a user inputs various parameters and development trend data of a ship, and wants to predict the parameters of the next generation of products for that ship, the following instruction template can be constructed:

[0045] Input: Given information: {Ship 1, xxxxxx (brief description of the product), its basic parameters are as follows: Length: xxx; Width: xxx; ...; Application scenario: xxx; Development trend: xxx.}. Based on the above given information, answer the user's question concisely and professionally. If you cannot find the answer, please say "Unable to answer the question based on the given information" or "Insufficient relevant information provided." Fabricated parts are not allowed in the answer, and please use Chinese for the answer. The question is: {What are the basic parameters of Ship 1's next-generation product, Ship 12?}

[0046] Output: Based on the provided parameters and development trends of Ship 1, the basic parameters of its next-generation product, Ship 12, can be inferred as follows: Length: xxx; Width: xxx; ...; Application scenario: xxx;

[0047] Finally, based on the exported SQL statements and instruction template set, data is populated to complete the construction of the instruction dataset.

[0048] In one embodiment, the step of fine-tuning a pre-trained general knowledge model using LoRa with a text dataset and an instruction dataset to obtain a private knowledge model includes: obtaining a hybrid dataset based on the text dataset and the instruction dataset; decomposing the parameters to be trained into a reduced-dimensional matrix and an increased-dimensional matrix, and obtaining the parameters of the private knowledge model based on the parameters of the general knowledge model and the parameters to be trained; inputting the hybrid dataset into the general knowledge model, and iteratively training the parameters of the private knowledge model until the difference between the model output and the true answer is minimized, at which point the iteration stops, and the private knowledge model is obtained.

[0049] In this embodiment, the text dataset helps the large model learn more enterprise-specific knowledge, while the instruction dataset helps the large model better understand the intent of user input questions or instructions, resulting in professional and reliable content generation.

[0050] LoRa is a lightweight large-scale model fine-tuning technique. The parameters of the original general knowledge large model are W0, and the parameters of the resulting private knowledge large model after fine-tuning are W. The parameters fine-tuned throughout the training process are ΔW. Therefore:

[0051] W = W0 + ΔW

[0052] At this point, the parameters ΔW to be trained are decomposed into two matrices, namely the reduced-dimensional matrix A and the increased-dimensional matrix B, as follows:

[0053] W = W0 + ΔW = W0 + BA

[0054] Where W0∈R d×k Denotes the initial general knowledge large model parameters, B∈R d×r Represents an upgraded matrix, initialized with a matrix of all zeros; A∈R r×k Let A represent the dimensionality reduction matrix, initialized with a random Gaussian distribution; r is the rank, a prior parameter, and r ≤ min(d,k). Throughout the training process, A and B are trainable parameters, while the others are prior parameters and fixed parameters.

[0055] During the model's forward training, both W0 and ΔW are multiplied by the same input x, that is:

[0056] h = W0x + ΔWx = W0x + BAx

[0057] The training objective of the model is:

[0058] min(h'-h)

[0059] Where h' is the true answer and h is the model output. This process is repeated multiple times until the model converges.

[0060] In one embodiment, such as Figure 2 As shown, a schematic diagram of the reasoning process of a private knowledge big data model is provided. The steps of constructing a reference material vector library based on several segmented text vectors corresponding to real-time private data include: converting real-time private data into text using the sql2text method, dividing the text into several natural segments; converting each natural segment into a multi-dimensional vector to obtain a reference material vector; and obtaining the reference material vector library based on the reference material vectors corresponding to several natural segments.

[0061] Specifically, the sql2text method is used to convert real-time private data into text. The text is then segmented into paragraphs of 200-300 characters each. Each paragraph is then converted into a vector using the open-source vector conversion model text2vec-base-chinese from the huggingface website. These vectors are then stored to obtain a reference material vector library.

[0062] In one embodiment, the step of selecting the reference material vector most similar to the input vector from the reference material vector library to obtain the reference material corresponding to the user input includes: calculating the cosine similarity between the input vector and the reference material vectors in the reference material vector library, and obtaining the reference material corresponding to the user input based on the relationship between the cosine similarity of each reference material vector and a preset threshold.

[0063] Specifically, the user inputs a question or instruction, which is then converted into a 768-dimensional vector using the open-source vector conversion model text2vec-base-chinese. A similarity threshold is dynamically designed based on the actual distribution characteristics of the private data. The cosine similarity between the input question vector and vectors in the reference material library is calculated one by one, and paragraphs of the original text whose cosine similarity meets the designed threshold are selected as the reference material for that question. The formula for calculating cosine similarity is as follows:

[0064]

[0065] Where, q i Given the input problem vector, y iThe input question and the original text of the reference material are then concatenated according to the instruction template to obtain the final input. This final input is then fed into a fine-tuned private knowledge model to generate the final content.

[0066] In one embodiment, the step of obtaining the reference material corresponding to the user input based on the relationship between the cosine similarity corresponding to each reference material vector and a preset threshold includes: obtaining the maximum input length and similarity difference interval supported by the private knowledge large model; the similarity difference interval includes the minimum difference and the maximum difference; sorting the cosine similarity corresponding to each reference material vector in descending order to obtain a similarity list; traversing each cosine similarity in the similarity list, calculating the similarity difference between the current cosine similarity and the previous sorted similarity that meets the preset conditions, and obtaining the reference material corresponding to the user input based on the relationship between the similarity difference and the similarity difference interval; the total length of the user input and the reference material is less than the maximum input length.

[0067] In this embodiment, not all data distribution features use the same threshold. The threshold definition is related to the specific business data, thereby improving the accuracy of reference materials in the model input. If too many segments in the private database have a similarity higher than the threshold to the input question vector, the final input characters after concatenation will be too long, severely affecting the performance of the large model. If multiple segments in the private database have a similarity almost identical to the input question vector, and all exceed the set threshold, concatenating these segments with the input question to form the final input will significantly increase the redundancy of the final input, affecting the performance of the large model. To address the above issues, a similarity threshold method is designed based on the actual distribution of the private data to improve the stability of the model input.

[0068] In one embodiment, the method further includes: if the similarity difference is within the similarity difference range, then retain the reference material corresponding to the current ranking similarity; if the similarity difference is less than the minimum difference, then discard the reference material corresponding to the current ranking similarity; if the similarity difference is greater than the maximum difference, then discard the reference material vector corresponding to the current ranking similarity, and use the reference material corresponding to the previous ranking similarity that meets the preset conditions as the reference material corresponding to the user input.

[0069] In this embodiment, the similarity threshold method is dynamically designed based on the actual distribution of private data as follows:

[0070] The model supports a maximum input length of n characters, and the similarity difference interval is set to (S). i S jFor example, (0.01, 0.1). First, obtain a list of cosine similarities between the input question vector and reference material vectors in the reference material library, and sort them in descending order. Taking the first reference material vector with the highest similarity as the benchmark, if the similarity difference between the second and first reference material vectors is within the interval (S... i S j In the second reference material vector, the third reference material vector and the second reference material vector are compared for similarity difference. If the similarity difference is less than S... i Then discard the second reference vector and continue comparing the similarity difference between the third and first reference vectors; if the similarity difference is greater than S... j Then, discard the second reference material vector, stop the comparison, and select the first paragraph as the final reference material. Continue this process until all paragraphs that meet the conditions are obtained, and the total length of the input question plus the paragraphs must be less than the maximum input length n. For private domain databases, in practical applications, multiple sets (S) are designed. i S j As a grid parameter, the best set (S) is selected based on the content generation effect. i S j )parameter.

[0071] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0072] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this invention should be determined by the appended claims.

Claims

1. A private knowledge content generation method based on a general-purpose large model and transfer learning, characterized in that, The method comprises: Obtaining the structured database corresponding to the enterprise private data required by the current business mode, performing foreign key association between tables on the structured relational data in the structured database to obtain table association data; Respectively using sql2text method and sql2Instruction method to process the table association data to obtain text data set and instruction data set, using the text data set and the instruction data set to fine-tune the pre-trained general large model to obtain a private knowledge large model; Obtaining real-time private data, constructing a reference material vector library according to a plurality of segmented text vectors corresponding to the real-time private data, obtaining user input, performing vector conversion on the user input to obtain an input vector, selecting a reference material vector most similar to the input vector from the reference material vector library to obtain a reference material corresponding to the user input; Splicing the user input and the reference material according to an instruction template, inputting the spliced result into the private knowledge large model to obtain corresponding generated content; The step of selecting a reference material vector most similar to the input vector from the reference material vector library to obtain a reference material corresponding to the user input comprises: Calculating the cosine similarity between the input vector and the reference material vector in the reference material vector library, and obtaining the reference material corresponding to the user input according to the size relationship between the cosine similarity corresponding to each reference material vector and the preset threshold value; The step of obtaining the reference material corresponding to the user input according to the size relationship between the cosine similarity corresponding to each reference material vector and the preset threshold value comprises: Obtaining the maximum input length supported by the private knowledge large model and the similarity difference interval; the similarity difference interval includes a minimum difference and a maximum difference; Sorting the cosine similarity corresponding to each reference material vector in descending order to obtain a similarity list; Traversing each cosine similarity in the similarity list, calculating the similarity difference between the current cosine similarity and the last sorted similarity that meets the preset condition, and obtaining the reference material corresponding to the user input according to the relationship between the similarity difference and the similarity difference interval; the total length of the user input and the reference material is less than the maximum input length; The step of using the text data set and the instruction data set to fine-tune the pre-trained general large model to obtain a private knowledge large model comprises: Obtaining a mixed data set according to the text data set and the instruction data set; Decomposing the to-be-trained parameters into a dimension reduction matrix and a dimension increase matrix, obtaining the parameters of the private knowledge large model according to the parameters of the general large model and the to-be-trained parameters; Inputting the mixed data set into the general large model, iteratively training the parameters of the private knowledge large model until the difference between the model output and the true answer is minimized, stopping iteration, and obtaining the private knowledge large model.

2. The method of claim 1, wherein, The step of performing foreign key association between tables on the structured relational data in the structured database to obtain table association data comprises: Setting the association field pair according to the current task mode The structured relational data containing the associated field pair is associated between tables by an external key to obtain table associated data.

3. The method of claim 1, wherein, The table associated data is processed by an sql2Instruction method to obtain an instruction data set, including: The table associated data is exported as an sql statement, and an instruction template set is constructed according to a plurality of pre-set question modes and answer modes; Data is filled according to the sql statement and the instruction template set to obtain an instruction data set.

4. The method of claim 1, wherein, The steps of constructing a reference material vector library according to the corresponding several segmented text vectors of the real-time private data include: The real-time private data is converted into text by an sql2text method, and the text is divided into several natural paragraphs; Each natural paragraph is converted into a multi-dimensional vector to obtain a reference material vector, and a reference material vector library is obtained according to the corresponding reference material vectors of the several natural paragraphs.

5. The method of claim 1, wherein, The method further includes: If the similarity difference value is within the similarity difference value interval, the reference material corresponding to the current ranking similarity is retained.

6. The method of claim 1, wherein, The method further includes: If the similarity difference value is less than the minimum difference value, the reference material corresponding to the current ranking similarity is discarded.

7. The method of claim 1, wherein, The method further includes: If the similarity difference value is greater than the maximum difference value, the reference material vector corresponding to the current ranking similarity is discarded, and the reference material corresponding to the last ranking similarity that meets the pre-set condition is taken as the reference material corresponding to the user input.

Images