Self-evolutionary methods, apparatus, devices, and media for training large models

By collecting evidence from the large model training process, determining the semantic iteration attribution, and generating multidimensional change evidence chains and failure mode labels, the problem of existing systems being unable to automatically analyze the causes of failure is solved, achieving accurate attribution and traceable analysis, and improving the automation of the training process and data quality.

CN123070192APending Publication Date: 2026-11-10BEIJING ZERO ONE EVERYTHING INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611046457.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-14
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing large model training systems lack self-evolution capabilities, cannot automatically analyze the root causes of training results and transform failure experiences into executable verification or prevention rules, resulting in the recurrence of similar problems during training, and there are security risks associated with agents modifying system logic.

Method used

By collecting evidence from the training process, determining the semantic iteration attribution, generating multidimensional change evidence chains and failure mode labels, identifying failure modes, generating candidate verification rules, and applying them to data verification before subsequent training, the training experiment can achieve accurate attribution and traceable analysis.

Benefits of technology

It effectively reduces the probability of similar problems recurring, improves the automation level and data quality of the large model training process, and ensures system security and controllability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123070192A_ABST
    Figure CN123070192A_ABST
Patent Text Reader

Abstract

This invention provides a self-evolutionary method, apparatus, device, and medium for training large-scale models. It involves collecting training process evidence from a first training task; determining the semantic iteration affiliation based on the relationship between the current training experiment and existing experiments, and changes in key evidence; if it is a new semantic iteration, collecting multiple types of change information to generate a multi-dimensional change evidence chain; identifying failure modes based on the multi-dimensional change evidence chain; generating failure mode labels and next-round suggestions; determining whether the failure mode is detectable before training; if so, generating candidate verification rules; adding the candidate verification rules to the verification rule library and applying them to data verification before initiating a second training task; and using the next-round suggestions as a decision reference for the second training task. This invention enables accurate attribution and traceable analysis of training experiments; effectively reduces the probability of similar problems recurring; and improves the automation level and data quality of the large-scale model training process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a self-evolutionary method, apparatus, device, and medium for training large models. Background Technology

[0002] Training large models is typically a complex, multi-stage iterative process, generally involving supervised fine-tuning (SFT), reinforcement learning (RL), preference optimization, and downstream task adaptation. Each training round involves the coordinated changes of various factors, including training data, configuration files, code versions, model checkpoints, operating environments, and evaluation metrics. To manage this complex multi-stage iterative process, various machine learning operation and maintenance platforms, experiment management systems, and training monitoring tools have been developed. However, with the exponential growth in the scale and complexity of large model training, existing systems exhibit increasing technical limitations in achieving efficient and reliable training process management. These limitations manifest as a lack of self-evolution based on historical training experience. For example, existing systems typically only passively record training results but cannot automatically analyze the root causes of those results, nor can they automatically transform these failures into executable verification or preventative rules for the next training round. When new problem patterns emerge during training, such as incorrect model output formats or missing tool call parameters, similar data quality issues repeatedly appear in subsequent training iterations, making optimization of the training process highly dependent on human intervention. Some systems introduce agents to assist in training decisions or answer questions, but these systems typically face a dilemma: if agents are completely prohibited from modifying system logic, a closed loop from failure experience to system improvement cannot be achieved; if agents are allowed to directly modify verification rules or analysis logic, it may introduce unauditable and unrecoverable security risks. Summary of the Invention

[0003] This invention provides a self-evolutionary method, apparatus, device, and medium for training large models, which addresses the shortcomings of existing models in that they lack the ability to automatically learn from historical training experience and transform failure patterns into reusable rules.

[0004] This invention provides a self-evolutionary method for training large models, comprising: Collect evidence of the training process for the first training task; Based on the parent-child relationship, stage relationship, target relationship and key evidence changes between the current training experiment and the existing experiments in the training process evidence, determine the semantic iteration affiliation of the first training task; If the semantic iteration belongs to a new semantic iteration, then collect multiple types of change information to generate a multi-dimensional change evidence chain, identify failure modes based on the multi-dimensional change evidence chain, and generate failure mode labels. Based on the multidimensional change evidence chain and the failure mode label, generate the next round of suggestions; Determine whether the failure mode is detectable before training; if so, generate candidate verification rules. The candidate verification rules are added to the verification rule base and applied to the data verification before the second training task is initiated; and the next round of suggestions are used as a decision reference for the second training task. The second training task is the training task executed after the first training task.

[0005] According to the self-evolutionary method for large model training provided by the present invention, the step of determining the semantic iteration affiliation of the first training task based on the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence includes: Obtain parent-child relationship information between the current training experiment and existing experiments; Obtain information on the stage relationship between the current training experiment and existing experiments; Obtain information on the target relationship between the current training experiment and existing experiments; Obtain key evidence changes between the current training experiment and existing experiments. The key evidence changes include at least one of the following: configuration changes, code changes, data changes, checkpoint changes, and runtime environment changes. Based on the parent-child relationship, stage relationship, target relationship, and key evidence change information, the semantic iteration attribution is determined according to the following judgment rules: If the current training experiment and the most recent experiment in the same phase belong to the same goal, and the previous experiment was in a failed or canceled state, and the key evidence has not changed substantially, then it is determined to be a retry attempt; If there are substantial changes in configuration, code, data, checkpoints, operating environment, or training objectives between the current training experiment and the most recent comparable experiment, it is determined to be a new semantic iteration.

[0006] According to the self-evolutionary method for training large models provided by the present invention, if the semantic iteration belongs to a retry attempt, then the historical multidimensional change evidence chain and failure mode label of the corresponding semantic iteration are inherited.

[0007] According to the self-evolutionary method for training large models provided by the present invention, the generation of failure mode labels includes: Extract training logs, evaluation outputs, failure samples, and tool responses from the multidimensional change evidence chain; The training logs are parsed to extract error information, exception stacks, and warning records, and information that matches the preset error characteristics is identified as a failure mode; The evaluation output is parsed to extract samples marked as failures and their corresponding model outputs, and cases where the output does not meet the expected format or content requirements are identified as failure modes. Each failed sample is analyzed to extract the difference between the model response and the standard answer in the sample, and the case where the difference exceeds a preset threshold is identified as a failure mode; The tool response is parsed to extract the return status, error code, and exception information during the tool invocation process, and the failure of the tool invocation or the return result is identified as a failure mode. The identified failure patterns are classified into model behavior anomaly labels and task execution error labels, wherein the model behavior anomaly labels are used to describe anomalies at the model output level, and the task execution error labels are used to describe execution failures at the task level.

[0008] The self-evolutionary method for training large models provided by the present invention further includes: The multidimensional alteration evidence chain is then standardized and serialized. The data after normalization and serialization is hashed to generate an evidence fingerprint; The evidence fingerprint is associated with and saved with the analysis results generated based on the multidimensional change evidence chain. The analysis results include at least one of the following: a list of key changes, training intent, training hypothesis, actual results, conclusions, and next round suggestions. When the multidimensional altered evidence chain changes, a new evidence fingerprint is recalculated; If the new evidence fingerprint is inconsistent with the original evidence fingerprint, the analysis result will be marked as expired.

[0009] According to the self-evolutionary method for training large models provided by the present invention, the generation of the next round of suggestions based on the multidimensional change evidence chain and the failure mode labels includes: The multidimensional change evidence chain is compared item by item with the multidimensional change evidence chain of the previous semantic iteration, and the change items with differences are extracted to generate a list of key changes. Based on the type and degree of impact of the changes in the list of key changes, infer the training intent of this round of training; Based on the correlation between the training intent and the list of key changes, a training hypothesis is generated; The training results-related indicators and sample evaluation data are extracted from the multidimensional change evidence chain, compared with the results of the previous iteration, and the actual results are generated. The training hypothesis is compared with the actual results. If the actual results match the training hypothesis, the hypothesis is marked as true; otherwise, the hypothesis is marked as false, and a conclusion is generated. Based on the list of key changes, training intent, training hypotheses, actual results, and conclusions, generate at least one of the following suggestions: whether to proceed to the next training phase, whether to rerun the current phase, whether to change the checkpoint, whether to modify the training data, whether to adjust the training parameters, whether to add or enable data validation rules, whether to record it as a known problem, and whether to generate a self-improvement proposal.

[0010] According to the self-evolutionary method for training large models provided by the present invention, the step of determining whether the failure mode is detectable before training, and if so, generating candidate validation rules, includes: Statistically analyze the distribution of failure modes in one or more training experiments; Determine whether the failure mode distribution is related to key factors of the training data, including at least one of sample structure, tool call format, field completeness, sample length, and sample distribution; If the failure mode distribution is related to the factors, then the failure mode is determined to be detectable before training. Obtain failure modes that are detectable before training, and generate corresponding candidate verification rules. The candidate verification rules include at least one of the following: rule name, triggering condition, applicable data type, detection logic, severity level, example, corresponding failure mode, and source experiment.

[0011] The present invention also provides a self-evolving device for training large models, comprising: The acquisition module is used to collect evidence of the training process for the first training task; The judgment module is used to determine the semantic iteration affiliation of the first training task based on the parent-child relationship, stage relationship, target relationship and key evidence changes between the current training experiment and the existing experiments in the training process evidence. The first generation module is used to collect multiple types of change information to generate a multi-dimensional change evidence chain if the semantic iteration belongs to a new semantic iteration, identify failure modes based on the multi-dimensional change evidence chain, and generate failure mode labels. The second generation module is used to generate the next round of suggestions based on the multidimensional change evidence chain and the failure mode label; The third generation module is used to determine whether the failure mode is detectable before training; if so, it generates candidate verification rules. The application module is used to add the candidate verification rules to the verification rule library and apply them to the data verification before the second training task is initiated; and to use the next round of suggestions as a decision reference for the second training task. The second training task is the training task executed after the first training task.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the self-evolutionary method for training large models as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the self-evolutionary method for training large models as described above.

[0014] The self-evolutionary method, apparatus, device, and medium for large model training provided by this invention collect training process evidence for a first training task; based on the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence, the semantic iteration affiliation of the first training task is determined; if the semantic iteration affiliation is a new semantic iteration, multiple types of change information are collected to generate a multi-dimensional change evidence chain; failure modes are identified based on the multi-dimensional change evidence chain, and failure mode labels are generated; based on the multi-dimensional change evidence chain and the failure mode labels, a next-round suggestion is generated; and it is determined whether the failure mode has pre-training detectability. If so, then candidate verification rules are generated; the candidate verification rules are added to the verification rule library and applied to the data verification before the second training task is initiated; and the next round suggestion is used as a decision reference for the second training task; wherein, the second training task is the training task executed after the first training task. This invention achieves accurate attribution and traceable analysis of training experiments by judging semantic iteration attribution and constructing a multi-dimensional change evidence chain; and then automatically converts the identified failure mode into candidate verification rules and applies them to the data verification before subsequent training, effectively reducing the probability of the same problem recurring and improving the automation level and data quality of the large model training process. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is one of the flowcharts of the self-evolutionary method for training large models provided by the present invention; Figure 2 This is a schematic diagram illustrating the distinction between ordinary retries and the new semantic iteration provided by the present invention; Figure 3 This is a schematic diagram illustrating the detection of whether old analyses have expired, provided by the present invention. Figure 4 This is the second flowchart of the self-evolutionary method for training large models provided by the present invention; Figure 5 This is a schematic diagram of the structure of the self-evolutionary device for training large models provided by the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] Figure 1 A flowchart of a self-evolutionary method for training large models provided in an embodiment of the present invention is shown below. Figure 1 As shown, the self-evolutionary method for training large models provided in this embodiment of the invention includes: Step 101: Collect evidence of the training process for the first training task; In this embodiment of the invention, basic information of training experiments can be obtained through training platform API, cluster scheduling API, file system scanning, log parsing, database synchronization, or user input. This basic information includes, but is not limited to: experiment identifier, training stage, parent experiment or upstream checkpoint, training task identifier, training status, training start time and end time, dataset used, configuration file used, code repository and commit version used, base model or checkpoint used, training log path, and evaluation results and key metrics.

[0019] Step 102: Based on the parent-child relationship, stage relationship, target relationship and key evidence changes between the current training experiment and the existing experiments in the training process evidence, determine the semantic iteration affiliation of the first training task; Step 103: If the semantic iteration belongs to a new semantic iteration, collect multiple types of change information to generate a multi-dimensional change evidence chain, identify failure modes based on the multi-dimensional change evidence chain, and generate failure mode labels. Step 104: Based on the multi-dimensional change evidence chain and the failure mode label, generate the next round of suggestions; Step 105: Determine whether the failure mode is detectable before training. If so, generate candidate verification rules. Step 106: Add the candidate verification rules to the verification rule library and apply them to the data verification before the second training task is initiated; and use the next round of suggestions as a decision reference for the second training task. The second training task is the training task executed after the first training task.

[0020] Traditional training data validation rules are typically hard-coded and cannot automatically evolve based on new failure modes that emerge in subsequent training. For example, if specific formatting errors, tool call errors, or business constraint errors are found in the model output after a training round, existing systems often require developers to analyze each error, write new data checking scripts, and manually execute them before the next training round. This process is slow, prone to omissions, and difficult to reuse. Existing experiment management tools often treat a single task as an experiment, lacking the abstraction of semantic iteration. Agent-assisted training systems can usually answer questions or call tools, but lack controllable self-improvement boundaries. If the agent directly modifies the analysis logic or validation rules, it may lead to unauditable, unrollbackable, and security risks; if agent modification is completely prohibited, a closed loop from training failure experience to checking rule improvement cannot be formed.

[0021] The self-evolutionary method for training large models provided in this invention collects training process evidence for a first training task; based on the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence, it determines the semantic iteration affiliation of the first training task; if the semantic iteration affiliation is a new semantic iteration, it collects multiple types of change information to generate a multi-dimensional change evidence chain, identifies failure modes based on the multi-dimensional change evidence chain, and generates failure mode labels; based on the multi-dimensional change evidence chain and the failure mode labels, it generates suggestions for the next round; it determines whether the failure mode is detectable before training, and if so... Then, candidate verification rules are generated; the candidate verification rules are added to the verification rule library and applied to the data verification before the second training task is initiated; and the next round of suggestions are used as a decision reference for the second training task; wherein, the second training task is the training task executed after the first training task. This invention achieves accurate attribution and traceable analysis of training experiments by judging semantic iteration attribution and constructing a multi-dimensional change evidence chain; and then automatically converts the identified failure mode into candidate verification rules and applies them to the data verification before subsequent training, effectively reducing the probability of the same problem recurring and improving the automation level and data quality of the large model training process.

[0022] Based on any of the above embodiments, determining the semantic iteration affiliation of the first training task according to the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence includes: Step 201: Obtain the parent-child relationship information between the current training experiment and existing experiments; Step 202: Obtain the stage relationship information between the current training experiment and existing experiments; Step 203: Obtain the target relationship information between the current training experiment and existing experiments; Step 204: Obtain key evidence change information between the current training experiment and existing experiments. The key evidence change includes at least one of configuration change, code change, data change, checkpoint change, and runtime environment change. Step 205: Based on the parent-child relationship, stage relationship, target relationship, and key evidence change information, determine the semantic iteration attribution according to the following judgment rules: Step 206: If the current training experiment and the most recent experiment in the same stage belong to the same goal, and the previous experiment was in a failed or canceled state, and the key evidence has not changed substantially, then it is determined to be a retry attempt. Step 207: If there are substantial changes in configuration, code, data, checkpoints, operating environment, or training objective between the current training experiment and the most recent comparable experiment, it is determined to be a new semantic iteration.

[0023] like Figure 2 As shown, this method obtains information about the relationship between the current training experiment and existing experiments, distinguishing between execution-level retries and semantic-level iterations. Specifically, it includes: S1. If the current training experiment and the most recent experiment in the same phase belong to the same goal, and the previous experiment failed or was canceled, and the key evidence such as configuration, code, data, and checkpoints have not changed substantially, then the current training experiment will be classified into the same iteration as a retry attempt. S2. If the current training experiment has substantial changes in configuration, code, data, dependencies, checkpoints, or training objectives relative to the most recent comparable experiment, then create a new iteration; S3. If the current training experiment belongs to a new stage in the pipeline and inherits the output checkpoint of the previous stage, then create or associate an iteration based on the stage relationship and parent-child experiment relationship. S4. If the user specifies that the current experiment belongs to a certain iteration, the user-specified result will be used first, and the user's overwriting trace will be recorded.

[0024] This module helps the system avoid misjudging simple reruns as new research rounds and prevents meaningful configuration or data changes from being buried in ordinary task records.

[0025] In this embodiment of the invention, if the semantic iteration belongs to a retry attempt, then the historical multidimensional change evidence chain and failure mode label of the corresponding semantic iteration are inherited.

[0026] Based on any of the above embodiments, generating the failure mode label includes: Step 301: Extract training logs, evaluation outputs, failure samples, and tool responses from the multidimensional change evidence chain; Step 302: Parse the training log, extract error information, exception stacks and warning records, and identify information that meets the preset error characteristics as failure modes; Step 303: Analyze the evaluation output, extract the samples marked as failures and their corresponding model outputs, and identify the cases where the output does not meet the expected format or content requirements as failure modes. Step 304: Analyze each of the failed samples, extract the differences between the model response and the standard answer in the samples, and identify the cases where the differences exceed a preset threshold as failure modes; Step 305: Analyze the tool response, extract the return status, error code and exception information during the tool call process, and identify the failure mode as the situation where the tool call fails or the return result is abnormal. Step 306: Classify the identified failure modes into model behavior anomaly labels and task execution error labels, wherein the model behavior anomaly labels are used to describe anomalies at the model output level, and the task execution error labels are used to describe execution failures at the task level.

[0027] In this embodiment of the invention, failure modes are automatically identified from training output, evaluation samples, tool call responses, logs, and validation results. Failure modes can be categorized into at least two types of labels: model behavior anomaly labels, used to describe anomalies at the model output level, such as empty output, repeated inference, output truncation, degenerate output, failure to call necessary tools, incomplete format, etc.; and business or task execution error labels, used to describe task-level failures, such as missing keys, no feasible solutions, illegal format, object mismatch, incomplete toolchain, general execution errors, etc. Identification can be based on rules, regular expressions, parsers, classification models, or large model discriminators. Identification results are saved in the form of sample-level labels, experimental-level statistics, and iterative-level trends.

[0028] Based on any of the above embodiments, the self-evolutionary method for training large models further includes: Step 401: Perform standardized serialization processing on the multidimensional alteration evidence chain; Step 402: Perform hash calculation on the normalized serialized data to generate evidence fingerprints; Step 403: Associate and save the evidence fingerprint with the analysis results generated based on the multidimensional change evidence chain. The analysis results include at least one of the following: a list of key changes, training intent, training hypothesis, actual results, conclusions, and suggestions for the next round. Step 404: When the multidimensional alteration evidence chain changes, recalculate the new evidence fingerprint; Step 405: If the new evidence fingerprint is inconsistent with the original evidence fingerprint, then the analysis result is marked as expired.

[0029] like Figure 3 As shown, the evidence fingerprinting and failure detection module is used to generate deterministic evidence fingerprints for change bundles. Specific steps include: S1. Sort the change bundle according to the preset field order or normalized serialization method; S2. Perform hash calculations on the sorted structured data, for example, using SHA-256 or other deterministic digest algorithms; S3. Extract or save the complete hash value as an evidence hash; S4. Associate and save the evidence hash with the attribution results, review draft, and list of key changes. S5. Recalculate the evidence hash when configuration, code, data, checkpoints, runtime results, or other evidence changes; S6. If the new evidence hash is inconsistent with the saved evidence hash, mark the existing attribution results as stale or analysis expired; S7. The system prompts the user to regenerate key changes, intent inferences, or review drafts.

[0030] This mechanism can prevent natural language analysis based on old evidence from being misused after the evidence changes, thus improving the reliability of training and review conclusions.

[0031] The multidimensional change evidence chain construction module is used to automatically collect change packages within a semantic iteration. These change packages include at least: configuration changes, such as training parameters, context length, output length, learning rate, batch size, reward weights, KL coefficients, sampling parameters, and validation frequency; code changes, such as code repository name, baseline commit version, current commit version, commit scope, changed files, difference statistics, and key file summaries; dependency changes, such as dependency lists, container versions, runtime framework versions, submodule references, and key package versions; data changes, such as dataset paths, dataset versions, number of samples, field structure, data sources, data filtering rules, and sample distribution; checkpoint changes, such as base model, parent experiment, parent checkpoint, model directory, and checkpoint steps; runtime environment changes, such as node type, number of GPUs, distributed training parameters, environment variables, and scheduling parameters; runtime result changes, such as training status, training duration, key performance indicator curves, validation scores, failure rate, error rate, and sample-level evaluation results; and user or agent actions, such as approval records, configuration modification records, training submission records, training cancellation records, and checkpoint selection records.

[0032] This invention not only preserves the original differences but also normalizes some of them into structured field differences. For example, it normalizes configuration file text differences (diff) into key-level differences, code changes into repository-level and file-level changes, and data changes into additions, deletions, replacements, or distribution changes.

[0033] Based on any of the above embodiments, generating the next round of recommendations based on the multidimensional change evidence chain and the failure mode label includes: Step 501: Compare the multidimensional change evidence chain with the multidimensional change evidence chain of the previous semantic iteration item by item, extract the change items with differences, and generate a list of key changes. Step 502: Based on the type and degree of impact of the changes in the list of key changes, infer the training intent of this round of training; Step 503: Generate training hypotheses based on the correlation between the training intent and the list of key changes; Step 504: Extract the indicators and sample evaluation data related to the training results from the multidimensional change evidence chain, compare them with the results of the previous iteration, and generate the actual results; Step 505: Compare the training hypothesis with the actual results. If the actual results match the training hypothesis, mark the hypothesis as true; otherwise, mark the hypothesis as false and generate a conclusion. Step 506: Based on the list of key changes, training intent, training hypothesis, actual results and conclusions, generate at least one of the following suggestions: whether to proceed to the next training phase, whether to rerun the current phase, whether to change the checkpoint, whether to modify the training data, whether to adjust the training parameters, whether to add or enable data validation rules, whether to record it as a known problem, and whether to generate a self-improvement proposal.

[0034] This invention automatically generates structured review content based on a multi-dimensional change evidence chain, transforming scattered change information, indicator data, and failure modes into attribution conclusions that R&D personnel can directly understand and make decisions, thereby solving the problems of traditional reviews relying on manual analysis, low efficiency, and easy omissions. The structured debriefing output of this module includes at least the following: the training objective for this round, i.e., the core task expected to be achieved in this training; the training hypothesis for this round, i.e., the expected causal effects based on key changes; a list of key changes, summarizing substantial changes in configuration, code, data, checkpoints, environment, etc.; the source of evidence, importance assessment, and confidence score for each key change, making the attribution conclusions traceable and verifiable; expected behavioral or indicator changes, describing the positive effects that should be observed when the hypothesis is valid; actual behavioral or indicator changes, recording the actual indicator changes and sample-level performance after training; the status of hypothesis validation, clarifying whether the hypothesis is supported, refuted, or lacks sufficient evidence; remaining uncertainties, pointing out anomalies or fluctuations that cannot be fully explained by current evidence; suggestions for the next round of training, providing specific operational directions such as whether to advance the phase, rerun, change checkpoints, adjust parameters, or modify data; and whether new data validation rules or analysis rules are needed, directly transforming the problems found in this round into preventive measures for subsequent training. Through the structured debriefing output described above, the system helps researchers quickly identify key factors affecting training results, reduce repetitive analysis work, and distill experience into reusable rules, achieving a closed loop from single debriefing to continuous optimization. The system retains original evidence and structured fields while generating natural language debriefings, preventing natural language summaries from replacing the original evidence.

[0035] Based on any of the above embodiments, determining whether the failure mode is detectable before training, and if so, generating candidate verification rules, includes: Step 601: Statistically analyze the distribution of failure modes in one or more training experiments; Step 602: Determine whether the failure mode distribution is related to key factors of the training data. The key factors include at least one of the following: sample structure, tool call format, field completeness, sample length, and sample distribution. Step 603: If the failure mode distribution is related to the factor, then the failure mode is determined to be detectable before training. Step 604: Obtain failure modes that are detectable before training and generate corresponding candidate verification rules. The candidate verification rules include at least one of the following: rule name, triggering condition, applicable data type, detection logic, severity level, example, corresponding failure mode, and source experiment.

[0036] In this embodiment of the invention, the failure patterns discovered after training are transformed into data validation rules before training. The specific process includes: S1. Statistically analyze the distribution of failure modes in one or more training experiments; S2. Determine whether a certain failure mode is related to the training data, sample structure, tool call format, field completeness, sample length, or sample distribution; S3. If the failure mode is detectable before training, then generate candidate data verification rules; S4. Candidate data validation rules include rule name, triggering conditions, applicable data types, detection logic, severity level, examples, corresponding failure modes, and source experiments; S5. The system submits the candidate rules to the user for confirmation or automatically enables them based on the trust level; S6. Once the rule is enabled, it will be executed automatically before subsequent training sessions begin; S7. If the data validation fails, the system will prevent the training submission, issue a warning, or require the user to confirm. S8. The verification results are further correlated with subsequent training results to determine the validity of the rules.

[0037] For example, when the system finds missing tool call parameter fields in several failed samples, and this deficiency can be directly detected from the training data samples, the system generates a "tool call parameter integrity check" rule. Before the next round of training, if similar defective samples exist in the dataset, the system can detect them in advance and prevent them from entering the training process.

[0038] Some embodiments provided by the present invention also include

[0039] Step 701: Preset multiple trust levels, wherein the trust levels include at least one of the following: Level 1: Automatic reading and analysis are allowed only; writing is not allowed. Level 2: Allows the creation of experimental records and review drafts; Level 3: Allows submission of low-risk data verification rules; Level 4: Allows modification of training configuration or selection of checkpoints, but requires approval; Level 5: Allows for the proposal and execution of analyzer or validator improvements within the whitelist. Step 702: Match the corresponding trust level according to the current operation type and determine whether automatic execution is allowed; Step 703: If the operation requires approval, an approval item is generated and submitted to the user for confirmation. The approval item includes the operation type, target file or target rule, change summary, risk level, source of evidence, expected impact, and rollback method. Step 704: Receive the user's confirmation result for the approval item, and perform one of the following operations based on the confirmation result: approve execution, reject execution, or request modification and resubmission; Step 705: Record all write operations and their approval results to form a traceable operation audit log.

[0040] Through the above steps, the system can achieve differentiated authorization for operations of different risk levels while ensuring controllability and auditability. This avoids the closed-loop break caused by completely prohibiting the agent from modifying the system, and also prevents the security risks caused by arbitrary modifications by the agent.

[0041] In some embodiments provided by the present invention, a self-improvement audit log step is also included to record improvements automatically proposed or implemented by the system, specifically including: Step 801: Set a whitelist of modifiable scopes. The whitelist limits low-risk objects that are allowed to be automatically modified, including data verification rules, failure mode analyzers, log parsers, or knowledge bases. Automatic modification is prohibited for high-risk paths such as training core code, permission systems, and infrastructure control scripts. Step 802: When the system automatically proposes or implements improvements, an improvement record is generated. The improvement record includes at least: improvement title, improvement target, target file or rule or module, improvement type, source experiment or failure mode, improvement description, difference summary, test or verification results, approver or approval strategy, application status, and recording time. Step 803: Store the improved records in the audit log for subsequent querying and backtracking; Step 804: When it is necessary to modify an object within the whitelist, the system automatically proposes an improvement plan, which is then implemented after approval; when it is necessary to modify an object outside the whitelist, the system generates a prompt message requesting manual intervention from the user.

[0042] Through the above steps, the system can achieve limited self-improvement while ensuring security. All improvement behaviors are fully recorded and auditable, which not only meets the needs of closed-loop optimization, but also avoids the risks brought about by uncontrollable modifications.

[0043] like Figure 4 The self-evolutionary method for training large models provided in this embodiment of the invention includes the following steps: S1. Receive or collect information about the current training experiment, including training phase, configuration file, training data, code version, model checkpoints, running status, log path, and evaluation results.

[0044] S2. Based on the parent-child relationship, stage relationship, target relationship and key evidence changes between the current training experiment and the existing experiments, determine whether the current training experiment belongs to the retry attempt in the existing semantic iteration or constitutes a new semantic iteration.

[0045] S3. For a given semantic iteration, automatically collect configuration changes, code changes, dependency changes, data changes, checkpoint changes, runtime environment changes, and result changes to generate a multi-dimensional change evidence chain.

[0046] S4. Standardize and serialize the multidimensional alteration evidence chain, calculate the deterministic evidence fingerprint, and save the evidence fingerprint in association with the current analysis results.

[0047] S5. Identify failure patterns from training logs, evaluation outputs, failure samples, and tool responses, and generate labels for abnormal model behavior and task execution errors.

[0048] S6. Based on the multidimensional change evidence chain and failure mode labels, automatically generate a list of key changes, training intent, training hypothesis, actual results, conclusions, and suggestions for the next round.

[0049] S7. Determine whether the failure mode can be converted into a data validation rule that can be detected before training; if it can be converted, generate a candidate validation rule.

[0050] S8. Submit candidate validation rules, review drafts, or self-improvement proposals to users for confirmation, or automatically execute low-risk operations based on preset trust levels.

[0051] S9. Enable the confirmed data verification rules and execute them automatically before subsequent training tasks are submitted; if verification fails, output a block, warning, or approval request.

[0052] S10. When the underlying evidence changes, recalculate the evidence fingerprint; if the evidence fingerprint changes, mark the existing attribution results as expired and trigger a reanalysis.

[0053] Based on any of the above embodiments, a training system includes a supervised fine-tuning stage, a reinforcement learning stage, and a downstream task optimization stage. Before each training task is submitted, the system reads the training dataset and executes enabled data validation rules, including sample format checks, field integrity checks, length distribution checks, duplicate sample checks, and tool call structure checks.

[0054] S1: After the Nth round of reinforcement learning training is completed, the evaluation samples are analyzed and it is found that a large number of samples have errors due to missing necessary fields.

[0055] S2: The error of missing necessary field can be detected directly from the task input field in the training data, which is a detectable problem before training.

[0056] S3: Automatically generate candidate data validation rules, which include: rule name "Necessary Field Integrity Check"; applicable scope: datasets containing structured task inputs and tool call targets; trigger condition: missing target field, constraint field, or tool call parameter field in the sample; severity level: high; source evidence: failed samples, error label statistics, and specific error responses in the Nth round of training.

[0057] S4: Show the candidate rule to the user. After the user approves it, the rule is officially added to the validation rule base.

[0058] S5: Before the N+1th round of training task submission, the system automatically scans the new dataset, executes the enabled validation rules, and finds that 2.7% of the samples are missing necessary fields. Therefore, the system blocks the training submission and prompts the user to repair the data or confirm to continue training.

[0059] Through the above steps, the system can automatically transform the data problems exposed in the previous training round into defense measures for the next training round, effectively preventing the recurrence of similar data quality problems in subsequent training iterations, and significantly reducing the cost and risk of omissions in manually writing verification rules.

[0060] Based on any of the above embodiments, this embodiment uses a specific training iteration attribution scenario as an example for illustration. In a certain training iteration, the following changes occurred compared to the previous comparable experiment: the context length in the configuration increased from 16384 to 32768, and the maximum output length increased from 8192 to 12288; new context samples were added to the dataset; the code repository commit version changed, with logs indicating truncation handling logic was involved; the parent checkpoint remained unchanged; the training results showed that the truncation rate decreased from 18% to 7%, but the training time increased.

[0061] S1: Automatically collects the above configuration changes, data changes, code changes, checkpoint changes, and runtime result changes, organizes all types of change information according to a preset structure, and generates a multi-dimensional change evidence chain, i.e., a change package. This change package records all substantial differences between this iteration and the previous comparable experiment, providing a complete evidentiary basis for subsequent attribution analysis.

[0062] S2: The multidimensional change evidence chain is normalized and serialized to generate a deterministic evidence fingerprint, which is then associated with and saved in relation to the attribution results of this iteration. This evidence fingerprint is used to subsequently detect whether the underlying evidence has changed, preventing the erroneous reuse of old analysis conclusions.

[0063] S3: Based on a multi-dimensional change evidence chain, automatically perform attribution analysis: compare the current change package with the previous evidence chain item by item to extract a list of key changes, including increased context length, increased output length, new context samples in the dataset, and changes to code truncation logic; based on the type and impact of the changes in the list of key changes, infer that the training intent for this round is to reduce the truncation problem in long context samples; based on the correlation between the training intent and the key changes, generate a training hypothesis, namely, that the truncation rate can be reduced by increasing the context length and output length, introducing long context data, and optimizing the truncation logic.

[0064] S4: While keeping the truncation rate under control, consider optimizing the sampling parameters and batch processing parameters to reduce the training time; at the same time, it is recommended to record the key changes and conclusions of this round as effective experience for reference in subsequent iterations.

[0065] S5: If engineers subsequently modify the configuration file associated with this iteration or supplement new evaluation results, the system will recalculate the evidence fingerprint. If the new fingerprint is found to be inconsistent with the original fingerprint, the system will automatically mark the original review results and prompt the user to update the attribution conclusion.

[0066] By using a multi-dimensional change evidence chain, the system can integrate scattered information such as configuration, code, data, and results into traceable attribution evidence, so that changes in the quality of training results no longer depend on the user's subjective memory. Through the evidence fingerprint mechanism, the system can automatically mark old conclusions as expired after changes in underlying evidence, avoiding the incorrect reuse of historical analysis results, and significantly improving the credibility of training review and the reliability of long-term training decisions.

[0067] The system constructs the above information into a change bundle and calculates the evidence hash. The iterative intent inference module outputs the following debriefing draft based on the structured evidence: The goal of this round is likely to mitigate the truncation problem in long context samples; the main evidence is the increase in both context length and maximum output length, along with the addition of long context data. Code changes may further impact truncation handling. Since the parent checkpoint remains unchanged, the main variables in this round are more likely to be configuration, data, and truncation logic, rather than changes to the underlying model. Actual results show a significant decrease in the truncation rate, supporting this hypothesis, but training time has increased. The next round could consider optimizing sampling and batching parameters while maintaining a controllable truncation rate.

[0068] Once the user confirms the review, the system saves it as the attribution result for that iteration. If an engineer subsequently modifies the configuration associated with that iteration or adds new evaluation results, the system recalculates the evidence hash. If the hash is found to be inconsistent with the original hash, the original review is marked as "evidence has changed and needs to be reanalyzed."

[0069] Based on any of the above embodiments, this embodiment will be used as an example to illustrate a scenario where the system automatically proposes and executes analyzer improvements.

[0070] S1: After a round of training, the system analysis revealed that the existing failure mode recognizer could not identify the "nested tool call" error. Although this error occurs frequently in the samples, the current analyzer only categorizes it as a general format error and cannot separately analyze its occurrence trend, making it difficult to analyze and prevent this problem in a targeted manner.

[0071] S2: Generate a structured self-improvement proposal, which includes: the improvement goal is to add nested tool call detection logic to the failure mode analyzer; the target scope is the failure mode analyzer module, which belongs to a pre-defined whitelist path, allowing the system to automatically propose modifications; the source evidence is several failure samples containing nested tool call errors and their corresponding error responses; the expected effect is to separate nested tool call errors from general format errors, facilitating independent trend analysis and targeted generation of data validation rules; the risk level is assessed as low; the verification method is to run the test set of the failure mode analyzer and check whether the statistical results of the newly added labels meet expectations.

[0072] S3: Submit the above self-improvement proposal to the user, who will then review and approve it. This user approval process ensures that the system's self-improvement is within a controllable range and will not cause unforeseen risks.

[0073] S4: After user approval, the system automatically modifies the logic of the failure mode analyzer, adds nested tool call detection functionality, and runs a preset test suite to verify the modified analyzer. If the test passes, the statistical results of the newly added tags are confirmed to meet expectations.

[0074] S5: After the test passes, the system executes a self-improvement audit log step, saving detailed information about the improvement, including the improvement title, target file, change summary, test results, source experiment identifier, and application status. This audit log can be queried and reviewed later, ensuring that all automated improvement actions are traceable and auditable.

[0075] S6: In subsequent training, the system can independently count the frequency and trend of nested tool call errors, and based on the statistical results of the failure mode, further determine whether it is necessary to generate data validation rules before training, such as checking whether the training data contains nested tool call samples with incorrect format, thereby realizing a complete closed loop from failure mode recognition to data validation rule generation.

[0076] This invention employs a whitelist mechanism, allowing automatic modifications only to low-risk modules (such as failure mode analyzers and data validation rules), while prohibiting automatic modifications to high-risk paths such as core training code and permission systems. This achieves system self-evolution while avoiding uncontrollable security risks. Through user approval processes and audit log mechanisms, all improvement actions have clearly defined responsible parties and complete operation records, meeting the stringent requirements of enterprise-level systems for auditability and rollback. This embodiment demonstrates that this invention, while ensuring security and controllability, enables the training system to possess the evolutionary characteristics of learning from failures and automatically optimizing its analytical capabilities.

[0077] Based on any of the above embodiments, this embodiment takes two different training retry scenarios as examples to illustrate how the system distinguishes between retry attempts at the execution level and semantic iterations at the research level.

[0078] Scene 1: S1: A training task failed after running for two hours due to a cluster node failure. The system recorded the status of the training as failed, and the reason for the failure was execution failure (cluster node abnormality).

[0079] S2: The user has not modified any configuration, data, code, or checkpoints, and immediately resubmits the training task with the exact same configuration, dataset, code commit version, and parent checkpoint as the first training.

[0080] S3: The semantic iteration recognition module obtains information about the relationship between the current training experiment and the existing experiments, compares the key evidence of the two training sessions, and finds that there are no substantial differences in configuration, code, data, checkpoint, or environment changes, and that the previous failure was due to an execution failure rather than a failure in the research sense.

[0081] S4: According to the judgment rules, the current training experiment and the most recent experiment in the same stage belong to the same training objective. The previous experiment was in a failed state and the key evidence has not changed substantially. Therefore, the system judges the second training as a retry attempt and classifies it into the semantic iteration to which the first training belongs, instead of creating a new semantic iteration.

[0082] Scene 2: S1: After a training task is completed, the user analyzes the results and finds that the training effect is not as expected, so they decide to adjust the training strategy.

[0083] S2: The user modified the data filtering rules and reward weights, keeping other configurations unchanged, and then resubmitted the training task.

[0084] S3: The semantic iteration recognition module obtains information about the relationship between the current training experiment and the existing experiments, compares the key evidence of the two training sessions, and discovers two types of substantial changes: data changes and configuration changes.

[0085] S4: According to the judgment rules, the current training experiment has substantial changes in data and configuration compared to the most recent comparable experiment. Therefore, the system creates a new semantic iteration and records and attributes this training as a training round with independent research significance.

[0086] S5: After training is completed, the system automatically generates debriefing content based on the multidimensional change evidence chain, and performs attribution analysis on the data filtering rules and reward weights as key changes to evaluate the actual impact of these two changes on the training results.

[0087] The embodiments of this invention fully demonstrate its technical effectiveness in distinguishing between execution failure retries and semantic changes in research. For retries caused by execution-level issues such as cluster failures, network anomalies, and insufficient resources, the system categorizes them as attempts within the same semantic iteration, avoiding misjudging simple reruns as new research rounds and preventing experimental record inflation and confused review conclusions. For retries caused by adjustments to research parameters such as data, configuration, code, and checkpoints, the system classifies them as new semantic iterations, ensuring that every meaningful experimental change is independently recorded and fully attributed. Through this mechanism, training records can more accurately reflect the actual research process and decision-making logic, improving the semantic quality of experimental management and the traceability of long-term training.

[0088] The self-evolutionary method for large model training provided in this invention takes training experiments as the basic execution object and semantic iteration as the attribution object. It automatically collects multi-dimensional training evidence during each training round and automatically constructs a change evidence package when training ends or a key event occurs. Based on the evidence package, it infers the training intent and key changes of this round and generates a structured debriefing draft. Simultaneously, it extracts failure patterns from training output, evaluation results, failure samples, and error responses. Reusable failure patterns are transformed into data validation rules or analyzer improvement suggestions. After user confirmation or approval at a preset trust level, these rules are written into the validation rule base or analysis module, thereby automatically intercepting similar problems before subsequent training begins. This invention can automatically transform data problems exposed after training into pre-training verification rules, enabling similar data quality problems to be proactively intercepted in subsequent training, thus improving the efficiency of training data quality control. By automatically collecting configuration, code, data, checkpoints, dependencies, environment, and result changes through a multi-dimensional change evidence chain, the attribution of training results no longer relies on the user's subjective memory, significantly improving the accuracy of debriefing. Through evidence fingerprinting and failure detection mechanisms, old conclusions are automatically marked as expired after changes in underlying evidence, avoiding the erroneous reuse of historical conclusions and improving the reliability of analysis conclusions.

[0089] The self-evolving device for training large models provided by the present invention will be described below. The self-evolving device for training large models described below can be referred to in correspondence with the self-evolving method for training large models described above.

[0090] Figure 5 This is a schematic diagram of the structure of the self-evolutionary device for training large models provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the self-evolutionary device for training large models provided in this embodiment of the invention includes: The acquisition module 501 is used to acquire evidence of the training process of the first training task; The judgment module 502 is used to determine the semantic iteration affiliation of the first training task based on the parent-child relationship, stage relationship, target relationship and key evidence changes between the current training experiment and the existing experiments in the training process evidence. The first generation module 503 is used to collect multiple types of change information to generate a multi-dimensional change evidence chain if the semantic iteration belongs to a new semantic iteration, identify failure modes based on the multi-dimensional change evidence chain, and generate failure mode labels. The second generation module 504 is used to generate a next round of suggestions based on the multidimensional change evidence chain and the failure mode label; The third generation module 505 is used to determine whether the failure mode is detectable before training. If so, it generates candidate verification rules. Application module 506 is used to add the candidate verification rules to the verification rule library and apply them to the data verification before the second training task is initiated; and to use the next round of suggestions as a decision reference for the second training task. The second training task is the training task executed after the first training task.

[0091] The self-evolving device for large model training provided in this invention collects training process evidence for a first training task; based on the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence, it determines the semantic iteration affiliation of the first training task; if the semantic iteration affiliation is a new semantic iteration, it collects multiple types of change information to generate a multi-dimensional change evidence chain, identifies failure modes based on the multi-dimensional change evidence chain, and generates failure mode labels; based on the multi-dimensional change evidence chain and the failure mode labels, it generates a next-round suggestion; it determines whether the failure mode is detectable before training, and if so... Then, candidate verification rules are generated; the candidate verification rules are added to the verification rule library and applied to the data verification before the second training task is initiated; and the next round of suggestions are used as a decision reference for the second training task; wherein, the second training task is the training task executed after the first training task. This invention achieves accurate attribution and traceable analysis of training experiments by judging semantic iteration attribution and constructing a multi-dimensional change evidence chain; and then automatically converts the identified failure mode into candidate verification rules and applies them to the data verification before subsequent training, effectively reducing the probability of the same problem recurring and improving the automation level and data quality of the large model training process.

[0092] The device can automatically generate a list of key changes, training intentions, training hypotheses, actual results, and suggestions for the next round. Researchers only need to review and correct these changes, effectively reducing review costs. The system can continuously accumulate failure patterns, verification rules, review conclusions, and improvement records across multiple rounds of experiments, enabling the training process to have continuous learning capabilities and improving the long-term training closed-loop capability. Through a whitelist mechanism, user approval, and audit records, the system can self-improve within a controlled scope, improving the security of agent self-improvement. By distinguishing between retry attempts and semantic iteration, the training records are made more consistent with the real research process, improving the semantic quality of experimental records. The system can combine historical failure patterns, changes in evidence, and actual indicators to propose more evidence-based suggestions for data modification, parameter adjustment, checkpoint selection, or stage advancement, which can improve the quality of decisions for the next round of training.

[0093] This invention also provides an electronic device, which may include a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus. The processor can call logical instructions in the memory to execute a self-evolutionary method for training a large model. This method includes: collecting training process evidence for a first training task; determining the semantic iteration affiliation of the first training task based on the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence; if the semantic iteration affiliation is a new semantic iteration, collecting multiple types of change information to generate a multi-dimensional change evidence chain, identifying failure modes based on the multi-dimensional change evidence chain, and generating failure mode labels; generating a next-round suggestion based on the multi-dimensional change evidence chain and the failure mode labels; determining whether the failure mode is detectable before training, and if so, generating candidate verification rules; adding the candidate verification rules to a verification rule base and applying them to data verification before initiating a second training task; and using the next-round suggestion as a decision reference for the second training task; wherein the second training task is a training task executed after the first training task.

[0094] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a self-evolutionary method for large model training provided by the methods described above. This method includes: collecting training process evidence for a first training task; determining the semantic iteration affiliation of the first training task based on the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence; if the semantic iteration affiliation is a new semantic iteration, collecting multiple types of change information to generate a multi-dimensional change evidence chain, identifying failure modes based on the multi-dimensional change evidence chain, and generating failure mode labels; generating a next-round suggestion based on the multi-dimensional change evidence chain and the failure mode labels; determining whether the failure mode is detectable before training, and if so, generating candidate verification rules; adding the candidate verification rules to a verification rule base and applying them to data verification before the initiation of a second training task; and using the next-round suggestion as a decision reference for the second training task; wherein the second training task is a training task executed after the first training task.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A self-evolutionary method for training large models, characterized in that, include: Collect evidence of the training process for the first training task; Based on the parent-child relationship, stage relationship, target relationship and key evidence changes between the current training experiment and the existing experiments in the training process evidence, determine the semantic iteration affiliation of the first training task; If the semantic iteration belongs to a new semantic iteration, then collect multiple types of change information to generate a multi-dimensional change evidence chain, identify failure modes based on the multi-dimensional change evidence chain, and generate failure mode labels. Based on the multidimensional change evidence chain and the failure mode label, generate the next round of suggestions; Determine whether the failure mode is detectable before training; if so, generate candidate verification rules. The candidate verification rules are added to the verification rule base and applied to the data verification before the second training task is initiated; and the next round of suggestions are used as a decision reference for the second training task. The second training task is the training task executed after the first training task.

2. The self-evolutionary method for training large models according to claim 1, characterized in that, The step of determining the semantic iteration affiliation of the first training task based on the parent-child relationship, stage relationship, target relationship, and key evidence changes between the current training experiment and existing experiments in the training process evidence includes: Obtain parent-child relationship information between the current training experiment and existing experiments; Obtain information on the stage relationship between the current training experiment and existing experiments; Obtain information on the target relationship between the current training experiment and existing experiments; Obtain key evidence changes between the current training experiment and existing experiments. The key evidence changes include at least one of the following: configuration changes, code changes, data changes, checkpoint changes, and runtime environment changes. Based on the parent-child relationship, stage relationship, target relationship, and key evidence change information, the semantic iteration attribution is determined according to the following judgment rules: If the current training experiment and the most recent experiment in the same phase belong to the same goal, and the previous experiment was in a failed or canceled state, and the key evidence has not changed substantially, then it is determined to be a retry attempt; If there are substantial changes in configuration, code, data, checkpoints, operating environment, or training objectives between the current training experiment and the most recent comparable experiment, it is determined to be a new semantic iteration.

3. The self-evolutionary method for training large models according to claim 1 or 2, characterized in that, If the semantic iteration belongs to a retry attempt, then the historical multidimensional change evidence chain and failure mode label of the corresponding semantic iteration are inherited.

4. The self-evolutionary method for training large models according to claim 1, characterized in that, The generation of failure mode labels includes: Extract training logs, evaluation outputs, failure samples, and tool responses from the multidimensional change evidence chain; The training logs are parsed to extract error information, exception stacks, and warning records, and information that matches the preset error characteristics is identified as a failure mode; The evaluation output is parsed to extract samples marked as failures and their corresponding model outputs, and cases where the output does not meet the expected format or content requirements are identified as failure modes. Each failed sample is analyzed to extract the difference between the model response and the standard answer in the sample, and the case where the difference exceeds a preset threshold is identified as a failure mode; The tool response is parsed to extract the return status, error code, and exception information during the tool invocation process, and the failure of the tool invocation or the return result is identified as a failure mode. The identified failure patterns are classified into model behavior anomaly labels and task execution error labels, wherein the model behavior anomaly labels are used to describe anomalies at the model output level, and the task execution error labels are used to describe execution failures at the task level.

5. The self-evolutionary method for training large models according to claim 1, characterized in that, Also includes: The multidimensional alteration evidence chain is then standardized and serialized. The data after normalization and serialization is hashed to generate an evidence fingerprint; The evidence fingerprint is associated with and saved with the analysis results generated based on the multidimensional change evidence chain. The analysis results include at least one of the following: a list of key changes, training intent, training hypothesis, actual results, conclusions, and next round suggestions. When the multidimensional altered evidence chain changes, a new evidence fingerprint is recalculated; If the new evidence fingerprint is inconsistent with the original evidence fingerprint, the analysis result will be marked as expired.

6. The self-evolutionary method for training large models according to claim 1, characterized in that, The generation of the next round of recommendations based on the multidimensional change evidence chain and the failure mode labels includes: The multidimensional change evidence chain is compared item by item with the multidimensional change evidence chain of the previous semantic iteration, and the change items with differences are extracted to generate a list of key changes. Based on the type and degree of impact of the changes in the list of key changes, infer the training intent of this round of training; Based on the correlation between the training intent and the list of key changes, a training hypothesis is generated; The training results-related indicators and sample evaluation data are extracted from the multidimensional change evidence chain, compared with the results of the previous iteration, and the actual results are generated. The training hypothesis is compared with the actual results. If the actual results match the training hypothesis, the hypothesis is marked as true; otherwise, the hypothesis is marked as false, and a conclusion is generated. Based on the list of key changes, training intent, training hypotheses, actual results, and conclusions, generate at least one of the following suggestions: whether to proceed to the next training phase, whether to rerun the current phase, whether to change the checkpoint, whether to modify the training data, whether to adjust the training parameters, whether to add or enable data validation rules, whether to record it as a known problem, and whether to generate a self-improvement proposal.

7. The self-evolutionary method for training large models according to claim 1, characterized in that, The step of determining whether the failure mode is detectable before training, and if so, generating candidate verification rules, includes: Statistically analyze the distribution of failure modes in one or more training experiments; Determine whether the failure mode distribution is related to key factors of the training data, including at least one of sample structure, tool call format, field completeness, sample length, and sample distribution; If the failure mode distribution is related to the factors, then the failure mode is determined to be detectable before training. Obtain failure modes that are detectable before training, and generate corresponding candidate verification rules. The candidate verification rules include at least one of the following: rule name, triggering condition, applicable data type, detection logic, severity level, example, corresponding failure mode, and source experiment.

8. A self-evolving device for training large models, characterized in that, include: The acquisition module is used to collect evidence of the training process for the first training task; The judgment module is used to determine the semantic iteration affiliation of the first training task based on the parent-child relationship, stage relationship, target relationship and key evidence changes between the current training experiment and the existing experiments in the training process evidence. The first generation module is used to collect multiple types of change information to generate a multi-dimensional change evidence chain if the semantic iteration belongs to a new semantic iteration, identify failure modes based on the multi-dimensional change evidence chain, and generate failure mode labels. The second generation module is used to generate the next round of suggestions based on the multidimensional change evidence chain and the failure mode label; The third generation module is used to determine whether the failure mode is detectable before training; if so, it generates candidate verification rules. The application module is used to add the candidate verification rules to the verification rule library and apply them to the data verification before the second training task is initiated; and to use the next round of suggestions as a decision reference for the second training task. The second training task is the training task executed after the first training task.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the self-evolutionary method for training large models as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the self-evolutionary method for training large models as described in any one of claims 1 to 7.