A large language model optimization method and system based on big data driving
By calculating the deviation between test response data and response data, high-frequency and low-frequency errors are identified and processed, and the allocation of training resources for large language models is optimized. This solves the problem of low model optimization efficiency in existing technologies and achieves efficient model optimization and rapid iteration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 天创信用服务有限公司
- Filing Date
- 2026-04-15
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies for large language models suffer from outdated information interference and wasted computational resources during training, resulting in low optimization efficiency and difficulty in rapid iteration and updates.
By calculating the deviation between the test response data and the response data, the corrected text data is identified and similarity comparison is performed. The erroneous data groups are divided, and supplementation operations are performed for high-frequency errors, while replacement processing is performed for low-frequency errors, thus optimizing the allocation of model resources.
It achieves accurate localization and repair of large language models, optimizes the allocation of training resources, improves the training efficiency and quality of models, and avoids resource waste.
Smart Images
Figure CN122366664A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of large language model optimization technology, specifically involving a large language model optimization method and system based on big data. Background Technology
[0002] With the development of artificial intelligence technology, large language models serve as the technological foundation of natural language processing. By pre-training on massive amounts of text data, they can capture complex language rules and world knowledge. They also demonstrate outstanding performance in key tasks such as machine translation, intelligent question answering, and content creation, providing technical support for digital transformation.
[0003] In existing technologies, the original datasets used for training contain a large amount of outdated information; this increases unnecessary training overhead on the training model and interferes with the model's accurate learning of core knowledge; causing problems when the model generates content.
[0004] Furthermore, current mainstream model optimization methods, which employ full fine-tuning or correction strategies based on complex rules, require a significant amount of computing resources and time. This not only burdens the system but also makes rapid model iteration and knowledge updates difficult, leading to problems such as response lag.
[0005] In view of this, the present invention proposes a large language model optimization method and system based on big data. Summary of the Invention
[0006] The purpose of this invention is to provide a large language model optimization method and system based on big data, which solves the technical problems of complex correction logic, long correction time, and poor optimization effect in the existing technology when optimizing language models.
[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: A large language model optimization method based on big data, comprising: Input the publicly available text data into the test model to obtain test response data; calculate the test deviation between the test response data and the corresponding response data; when the test deviation meets the preset correction trigger condition, perform the following steps: Identify and correct the text data and correct response data corresponding to the test deviation; Based on the corrected text data and corrected response data, data supplementation processing is performed to generate supplementary data for model optimization; and based on the supplementary data, the test model is optimized to obtain the target language model. The data supplementation process based on the corrected text data and corrected response data includes: performing similarity comparison on multiple corrected text data to divide the multiple corrected text data and multiple corrected response data into at least one data group; generating multiple error registration data based on the division results of at least one data group; calculating statistical values based on the multiple error registration data; and performing a supplementation operation when the statistical values and a preset capacity threshold meet a preset supplementation judgment relationship to obtain new text data and new response data corresponding to at least one error registration data, wherein the supplemented data includes new text data and new response data.
[0008] Preferably, the test deviation meeting the preset correction trigger condition includes: quantifying the test deviation into a numerical difference; and determining the scoring level based on the numerical difference according to the preset comparison rules. The rating level is converted into a risk level; and the risk level is compared with the preset correction parameters to determine whether the correction trigger condition is met.
[0009] Preferably, the similarity comparison of multiple corrected text data includes: performing content decomposition operation on any two corrected text data to extract keywords; Based on keywords, the difference ratio between two corrected text data is calculated, and the difference ratio is determined as the preliminary judgment value; and the preliminary judgment value is calculated with preset weight parameters to obtain the final judgment value as the basis for similarity comparison.
[0010] Preferably, when the statistical value and the preset capacity threshold satisfy the preset supplementation judgment relationship, the supplementation operation includes: when the statistical value is greater than the capacity threshold, marking the corresponding error registration data as high-frequency error data and performing a supplementation operation on the high-frequency error data; and when the statistical value is not greater than the capacity threshold, marking the corresponding error registration data as low-frequency error data and performing a replacement process on the low-frequency error data.
[0011] Preferably, performing permutation processing on low-frequency erroneous data includes: From high-frequency error data, high-frequency content is extracted; the part of low-frequency error data that matches the high-frequency content is identified; and the matching part of low-frequency error data is replaced with high-frequency content to update the low-frequency error data.
[0012] Preferably, before optimizing the test model based on the supplementary data, the method further includes: performing matching verification on the newly added text data and the newly added response data; in the matching verification, identifying data pairs that are found to be missing or cannot be matched as discontinuous data; and converting the discontinuous data into standard response content, wherein the supplementary data further includes standard response content.
[0013] A big data-driven large language model optimization system includes: The deviation analysis module is used to acquire public text data and corresponding response data, input the public text data into the test model to obtain test response data, calculate the test deviation between the test response data and the response data, and generate a correction trigger signal when the test deviation meets the preset correction trigger conditions. The data supplementation module is used to identify the correction text data and correction response data corresponding to the test deviation in response to the correction trigger signal, and to perform data supplementation processing based on the correction text data and correction response data to generate supplementary data for model optimization. It also includes a model optimization module, which optimizes the test model based on supplementary data to obtain the target language model.
[0014] Preferably, the test deviation meeting the preset correction trigger condition includes: quantifying the test deviation into a numerical difference; and determining the scoring level based on the numerical difference according to the preset comparison rules. The rating level is converted into a risk level; and the risk level is compared with the preset correction parameters to determine whether the correction trigger condition is met.
[0015] Preferably, the data supplementation module is configured to: perform similarity comparison on multiple corrected text data to divide the multiple corrected text data and multiple corrected response data into at least one data group; and generate multiple error registration data based on the division results of at least one data group; Based on multiple error registration data, a statistical value is calculated; and when the statistical value and a preset capacity threshold satisfy a preset supplementation judgment relationship, a supplementation operation is performed to obtain new text data and new response data corresponding to at least one error registration data. The supplementation data includes new text data and new response data.
[0016] Preferably, the model optimization module is configured to receive supplementary data generated by the data supplementation module to optimize the test model.
[0017] Beneficial effects 1. The large language model optimization method provided by this invention calculates the test deviation between test response data and response data, and identifies the corrected text data when the test deviation meets the preset correction trigger condition; then, it performs similarity comparison to divide similar corrected text data into data groups, thereby accurately locating and repairing model defects; by starting from the test deviation and summarizing similar errors, it achieves a centralized solution to common problems existing in the model.
[0018] 2. This invention calculates statistical values based on error registration data and compares them with a capacity threshold to distinguish between high-frequency and low-frequency error data. For high-frequency error data, a supplementation operation is performed; for low-frequency error data, a replacement operation is performed. Through this differentiated processing mechanism, the allocation of training resources is optimized. The main computational and data supplementation resources are concentrated on solving high-frequency errors with a wide impact, while correcting low-frequency errors in a lower-cost manner, avoiding the waste of resources caused by supplementing all error types with the same intensity.
[0019] 3. Before optimizing the model, this invention performs matching verification on the supplementary data; the identified missing or mismatched data is identified as discontinuous data and converted into standard response content before being used for model optimization. Based on this matching verification and conversion process, the completeness and high quality of the supplementary data used for model optimization are ensured, non-standard data is filtered and repaired, and knowledge confusion or logical defects are prevented from arising due to the model learning incorrect text and response data pairs. Attached Figure Description
[0020] Figure 1 This is a flowchart of the method provided by the present invention; Figure 2 This is a system module diagram provided by the present invention. Detailed Implementation
[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0022] Example 1 like Figure 1 As shown, this embodiment provides a big data-driven method for optimizing large language models, which specifically includes the following steps: Execute optimization presets: Obtain publicly available text data from multiple data sources, including publicly available information on the Internet and internal enterprise knowledge bases, as well as response data as a reference standard.
[0023] The response data corresponds to the publicly available text data; after acquisition, this data is cleaned and deduplicated, and uniformly converted into a standardized data structure containing unique identifiers and preset fields. Then, based on this, we set the specific tasks for this optimization, such as improving the accuracy of question answering in specific domains, as well as the target quantitative indicators for measuring the optimization effect, and set the response data as the expected output benchmark in the evaluation process; The response data can be manually written standard answers, verified question and answer records, or output content from existing knowledge systems.
[0024] Further, the implementation testing phase begins: multiple publicly available text data are input into the test model one by one to obtain multiple test response data corresponding to each publicly available text data; then, through the text comparison calculation procedure, the test deviation between the test response data and the corresponding response data is calculated, which indicates the degree of content difference between the test response data generated by the language processing device to be optimized and the response data of the standard reference. The test deviation is mapped to a rating level according to the preset comparison rules, and then the rating level is converted into a corresponding risk level. Finally, the determined risk level is compared with the preset correction parameters, and the correction starting point is determined according to the comparison results. The preset comparison rules are as follows: a segmented mapping rule is adopted, based on the quantized values of the test deviation, such as the precise floating-point numbers corresponding to BLEU scores, ROUGE scores, or cosine similarity, to classify the scoring levels; the specific mapping relationship is as follows: when the quantized value ∈ [0.9, 1.0], the scoring level is level 1, i.e., minimal deviation; when the quantized value ∈ [0.7, 0.9), the scoring level is level 2, i.e., relatively small deviation; when the quantized value ∈ [0.5, 0.7), the scoring level is level 3, i.e., moderate deviation; when the quantized value ∈ [0.3, 0.5), the scoring level is level 4, i.e., relatively large deviation; when the quantized value ∈ [0, 0.3), the scoring level is level 5, i.e., extremely large deviation. This rule adopts a segmented linear mapping logic to ensure that different deviation ranges correspond to clear scoring levels, and the numerical thresholds of each segment can be adjusted according to the accuracy requirements of model optimization.
[0025] The preset correction parameter is specifically a risk level threshold parameter, set to "Level 3", which corresponds to a deviation of medium or above. Its parameter type is a level label, and its value can be dynamically adjusted according to the model application scenario. For example, in high-precision scenarios, the correction parameter can be set to Level 2, and in normal scenarios, it can be set to Level 3. The comparison logic is as follows: when the calculated risk level is ≥ the preset correction parameter / Level 3, it is determined that the correction trigger condition is met, and the correction process is started; when the risk level is < the preset correction parameter / Level 3, it is determined that the correction trigger condition is not met, and the correction process is not started.
[0026] Different handling strategies are adopted according to the risk level: deviations at high risk must be corrected immediately, and this serves as the starting point for correction; deviations at medium risk can be marked as points for delayed correction. Simultaneously, the deviation verification process also reflects the current response stability and knowledge coverage of the test model, providing a basis for prioritizing subsequent correction plans.
[0027] Further, the error classification and registration steps are performed to identify all correction starting points that require immediate correction operations; the corresponding corrected text data and corrected response data are extracted from the original public text data and response data; then, the similarity of multiple corrected text data is compared, and error samples with similar content are aggregated to divide multiple corrected text data and multiple corrected response data into at least one data group. The comparison process is as follows: For any two corrected text data, their similarity is first calculated, and the content is decomposed. By statistically analyzing the frequency of each word in a single text and its distribution across all texts, keywords that represent the core content of the text are selected. Then, based on the extracted keyword set, the content difference ratio between the two texts is calculated using Jaccard distance or edit distance as a preliminary judgment value. Finally, this preliminary judgment value is combined with preset weight parameters to obtain the final judgment value. The weight parameters can be pre-configured based on the professionalism or importance of the keywords, assigning higher weight values to specialized terms within the domain. This final judgment value is the similarity result. After obtaining the similarity result, it is compared with the preset classification judgment threshold. If the similarity result is greater than 0.85, the two corrected text data are determined to belong to the same type of error. Based on this comparison result, multiple corrected text data and multiple corrected response data with similarity are divided into the same data group. Based on the division result of at least one data group, multiple error registration data are generated, and error registration data are generated for each data group, and they are uniformly added to the queue to be optimized.
[0028] A similarity judgment function is used to calculate the content similarity between two corrected text data. The final judgment value is used as the basis for determining whether the two texts belong to the same type of error. The specific mathematical formula is defined as follows: Input: From the first corrected text data Keyword set extracted From the second corrected text data Keyword set extracted Preset weighting function Used to return any keyword Importance weights.
[0029] Output: Final judgment value , representing the similarity results of two corrected text data.
[0030]
[0031] In the formula, This represents corrected text data, meaning two corrected text data sets whose similarity is to be compared. This represents a set of keywords, meaning keywords extracted from the corrected text data. and The set of keywords extracted from it; This indicates a keyword, meaning a set. or The elements in the text are words; This represents the weighting function, which is a function that returns a single keyword. The importance value can be pre-configured based on the keyword's specialization or its importance in the domain knowledge system; for example, domain-specific terms have higher weight. This represents the final judgment value, which means that the content similarity score of the two corrected text data after weighting and taking into account the importance of keywords.
[0032] Furthermore, after completing the error registration, the data supplementation and optimization stage begins. Based on multiple error registration data, the number of data pairs contained in each data group is calculated and used as a statistical value. Then, the statistical value is compared with a preset capacity threshold. If the statistical value is greater than the capacity threshold, such as a data group containing more than 100 similar error samples, the corresponding error registration data is marked as high-frequency error data. Supplementation operations are then performed on it to enhance the subsequent optimization effect by expanding the data of similar scenarios.
[0033] Specifically, based on the word frequency, context, and semantic features of this set of high-frequency error data, more representative new text data will be automatically constructed through preset text generation rules and structural templates, and new response data will be constructed simultaneously. Conversely, if the statistical value is not greater than the capacity threshold, the corresponding error registration data is marked as low-frequency error data, and replacement processing is performed on the low-frequency error data; from the correction response data associated with high-frequency error data, standard content fragments with universal applicability are extracted, such as the key sentences of the standard answer to a question that has been repeatedly answered incorrectly, as high-frequency content; Then, in the low-frequency error data, the parts to be replaced that semantically correspond to the high-frequency content are located by keyword or sentence structure matching; finally, the parts to be replaced in the low-frequency error data are replaced with high-frequency content to update the low-frequency error data, unify the scattered low-frequency error paradigms into the high-frequency error correction framework, and improve the consistency of the data to be optimized.
[0034] Furthermore, a discontinuity detection and correction step is performed to match and verify the newly added text data and newly added reply data obtained through the supplementation operation; the newly added reply data is standardized by unifying all reply content into JSON objects containing specific fields, or by normalizing the text for capitalization and correcting punctuation marks to ensure the consistency of data format. The newly added response data after format conversion is then matched with the corresponding newly added text data to check their logical consistency and reasonableness in terms of syntax and context. During the matching verification, data pairs that are found to have missing content or logical inconsistencies are identified as discontinuous data. If the newly added text data is a question, but the newly added response data does not semantically answer the question, or the response content is empty, then the data pair is identified as discontinuous data. All identified discontinuous data are then summarized, and their common defects are analyzed. Based on this, standard response content is generated. If the analysis finds that a certain type of discontinuous data generally lacks explanation of the information source, standard response content can be generated, such as "This response was generated based on a general knowledge base."
[0035] Finally, the newly added response data in the discontinuous data is replaced with standard response content to form logically complete and formatted data pairs, thus avoiding interference from low-quality data in the subsequent optimization process.
[0036] Further, the optimization and correction output steps are performed. Based on the newly added text data, newly added response data, and standard response content, the internal processing logic and parameters of the test model are adjusted to obtain the target language model. New target text data not included in the above data is input into the target language model to generate the corresponding final target response data. The generation process employs a multi-stage correction and replacement output mechanism. The target language model generates initial target response data for the target text data. Then, it acquires a dynamically updated content library associated with the target text data, which includes rules such as factual errata, compliance requirements, or response taboos in specific scenarios. The initial target response data is compared with the content to be corrected. When the two contents are inconsistent, such as the initial response containing information listed as outdated in the content to be corrected, the inconsistent parts in the initial target response data are replaced with the content to be corrected to obtain the replaced response data.
[0037] The replacement response data is checked for consistency with the initial target response data to assess whether the replacement introduced new logical contradictions or awkward expressions. Based on the consistency check results, the final target response data is determined; if the consistency check passes, the replacement response data is output; if it fails, it may revert to the initial target response data or mark the output as requiring manual intervention. After generating the final target response data, a corresponding response quality level is generated based on its consistency with the content to be corrected, information content, and fluency, for subsequent result feedback and continuous monitoring.
[0038] Example 2 like Figure 2The present embodiment provides a large language model optimization system that can automatically identify test deviations generated by the large language model when processing public text data, and when the deviations meet the preset correction trigger conditions, it can perform data supplementation and model optimization processes.
[0039] In its implementation, this system can be deployed on computing devices including personal computers, servers, cloud computing platforms, or distributed computing clusters. It integrates a test model to be optimized and connects to a database storing public text data, response data, and various intermediate data. The system specifically includes the following modules: The deviation analysis module continuously monitors the performance of the test model and determines whether to initiate the optimization process. During execution, it retrieves publicly available text data and corresponding response data as a reference standard from the data source; then, it inputs the publicly available text data into the test model to obtain test response data; and finally, it calculates the test deviation between the test response data and the actual response data.
[0040] The test bias can be the result of the difference assessment on one or more dimensions such as semantics, syntax, factuality, or logical consistency.
[0041] To determine whether the test deviation is severe enough to require correction, the following judgment process is performed: the test deviation is quantified into one or more numerical differences; secondly, based on preset comparison rules, such as a set of rules that maps different difference ranges to different levels, the scoring level is determined according to the numerical differences. Then, the rating level is converted into a more intuitive risk level, such as "low risk", "medium risk" and "high risk". The risk level is then compared with the preset correction parameters. The correction parameters can be set as the trigger threshold for "high risk". When the preset correction trigger condition is met, the current risk level reaches or exceeds the threshold set by the correction parameters, and a correction trigger signal is generated to activate the subsequent data supplementation module.
[0042] The data supplementation module is used to respond to correction trigger signals and generate supplementary data for model optimization. When a correction trigger signal is received, the module identifies the correction text data and correction response data corresponding to the test deviation that triggered the trigger; that is, the original input that caused the deviation and the correct output. After acquiring multiple sets of correction text data and correction response data, the module begins to perform data supplementation processing.
[0043] The similarity comparison of multiple corrected text data is performed, and similarity error classification is performed. In the specific implementation, the comparison process includes: performing content decomposition on any two corrected text data, extracting their respective keywords through natural language processing technology; and calculating the difference ratio between the two corrected text data, such as using algorithms such as Jaccard similarity or word vector cosine similarity, and determining the difference ratio as the preliminary judgment value. Finally, the preliminary judgment value is calculated with preset weight parameters, such as weighting according to the importance of keywords, to obtain the final judgment value as the final basis for similarity comparison; based on the final judgment value, similar corrected text data and corresponding corrected response data are divided into at least one data group. Then, based on the data group partitioning results, error registration data is generated for each data group, which records the common patterns or characteristics of errors in the group; based on multiple error registration data, their respective statistical values are calculated, such as the frequency of each error pattern, and then the specific supplementation strategy is determined according to the statistical values. The statistical values are compared with the preset capacity threshold to determine whether the preset supplementation judgment relationship is met.
[0044] Specifically, when the statistical value of a certain error registration data exceeds the capacity threshold, it is marked as high-frequency error data, and a supplementary operation is performed on it. By performing synonym replacement, sentence transformation, or topic expansion on the original text, new text data and new response data that are related to the original error pattern but have diversified content are generated; when the statistical value of a certain error registration data does not exceed the capacity threshold, it is marked as low-frequency error data, and a replacement process is performed on it. Representative high-frequency content is extracted from high-frequency error data. Matching portions of this high-frequency content in low-frequency error data are identified and replaced with the high-frequency content, thus updating and enhancing the low-frequency error data. After generating new data, a verification step is performed to ensure data quality. All new text and response data are matched and verified. Missing or mismatched data pairs discovered during verification are identified as discontinuous data and converted into standardized, harmless response content, such as template responses indicating "unable to answer" or "insufficient information." Finally, all generated new text and response data, along with the converted standard response content, are integrated to form the final supplementary data, which is then provided to the model optimization module.
[0045] The model optimization module receives supplementary data and optimizes the test model. In the specific execution process, the supplementary data is used as training material to perform optimization operations on the test model, including but not limited to fine-tuning, instruction tuning, or reinforcement learning based on human feedback, which are techniques well known to those skilled in the art. The model learns and corrects the test biases that occurred previously. After optimization, the original test model is transformed into a target language model with improved performance.
[0046] Through the collaborative work of the above modules, the system in this embodiment dynamically monitors and quantifies the test deviation of the model, and generates high-quality supplementary data based on the statistical characteristics of the test deviation, thereby performing targeted and efficient optimization of the model.
Claims
1. A large language model optimization method based on big data, characterized in that, include: The publicly available text data is input into the test model to obtain test response data; Calculate the test deviation between the test response data and the corresponding response data; When the test deviation meets the preset correction trigger condition, the following steps are performed: identify the correction text data and correction response data corresponding to the test deviation; Based on the corrected text data and corrected response data, perform data supplementation processing to generate supplementary data for model optimization; And based on the supplementary data, the test model is optimized to obtain the target language model; The data supplementation process based on the corrected text data and corrected response data includes: performing similarity comparison on multiple corrected text data to divide the multiple corrected text data and multiple corrected response data into at least one data group; generating multiple error registration data based on the division results of at least one data group; calculating statistical values based on the multiple error registration data; and performing a supplementation operation when the statistical values and a preset capacity threshold meet a preset supplementation judgment relationship to obtain new text data and new response data corresponding to at least one error registration data, wherein the supplemented data includes new text data and new response data.
2. The method for optimizing a large language model based on big data as described in claim 1, characterized in that, The test deviation meets the preset correction trigger conditions, including: The test deviation is quantified into a numerical difference; based on the preset comparison rules, the scoring level is determined according to the numerical difference; and the scoring level is converted into a risk level. It also compares the risk level with the preset correction parameters to determine whether the correction trigger condition is met.
3. The method for optimizing a large language model based on big data as described in claim 1, characterized in that, Similarity comparison of multiple corrected text data includes: Perform content decomposition on any two corrected text data sets to extract keywords; Based on keywords, the difference ratio between two corrected text data is calculated, and the difference ratio is determined as the preliminary judgment value; and the preliminary judgment value is calculated with preset weight parameters to obtain the final judgment value as the basis for similarity comparison.
4. The method for optimizing a large language model based on big data as described in claim 1, characterized in that, When the statistical value and the preset capacity threshold meet the preset supplementation judgment relationship, the supplementation operation is performed, including: When the statistical value is greater than the capacity threshold, the corresponding error registration data is marked as high-frequency error data, and an addition operation is performed on the high-frequency error data; when the statistical value is not greater than the capacity threshold, the corresponding error registration data is marked as low-frequency error data, and a replacement process is performed on the low-frequency error data.
5. The method for optimizing a large language model based on big data as described in claim 4, characterized in that, Permutation processing for low-frequency erroneous data includes: From high-frequency error data, high-frequency content is extracted; the part of low-frequency error data that matches the high-frequency content is identified; and the matching part of low-frequency error data is replaced with high-frequency content to update the low-frequency error data.
6. The method for optimizing a large language model based on big data as described in claim 1, characterized in that, Before optimizing the test model based on the supplementary data, the method also includes: matching and validating the newly added text data and newly added response data; In the matching and validation process, data pairs that are found to be missing or cannot be matched are identified as discontinuous data; and the discontinuous data is converted into standard response content, wherein the supplementary data further includes the standard response content.
7. A large language model optimization system based on big data, characterized in that, include: The deviation analysis module is used to acquire public text data and corresponding response data, input the public text data into the test model to obtain test response data, calculate the test deviation between the test response data and the response data, and generate a correction trigger signal when the test deviation meets the preset correction trigger conditions. The data supplementation module is used to identify the correction text data and correction response data corresponding to the test deviation in response to the correction trigger signal, and to perform data supplementation processing based on the correction text data and correction response data to generate supplementary data for model optimization. It also includes a model optimization module, which optimizes the test model based on supplementary data to obtain the target language model.
8. The method for optimizing a large language model based on big data as described in claim 7, characterized in that, The test deviation meets the preset correction trigger conditions, including: The test deviation is quantified into a numerical difference; based on the preset comparison rules, the scoring level is determined according to the numerical difference; and the scoring level is converted into a risk level. It also compares the risk level with the preset correction parameters to determine whether the correction trigger condition is met.
9. The method for optimizing a large language model based on big data as described in claim 7, characterized in that, The data supplementation module is configured as follows: A similarity comparison is performed on multiple corrected text data to divide the multiple corrected text data and multiple corrected response data into at least one data group; Based on the partitioning results of at least one data group, generate multiple error registration data; Statistical values were calculated based on multiple error registration data. And when the statistical value and the preset capacity threshold meet the preset supplementation judgment relationship, the supplementation operation is performed to obtain new text data and new response data corresponding to at least one error registration data. The supplementation data includes new text data and new response data.
10. The method for optimizing a large language model based on big data as described in claim 7, characterized in that, The model optimization module is configured to receive supplementary data generated by the data supplementation module to optimize the test model.