Large power transmission and distribution model robustness evaluation method based on adversarial training and automatic modeling

Abstract

The invention discloses a power transmission and distribution large model robustness evaluation method based on adversarial training and automatic modeling. Obtaining a multi-modal reference data set of a power transmission and distribution production command scene, and generating an adversarial sample set based on an automatic modeling mechanism; performing robustness disturbance injection on the adversarial sample set through an adversarial training technology to generate a robustness evaluation instruction; the prediction precision of the large model is verified through a robustness evaluation instruction in combination with an adaptive defending distillation mechanism, and a robustness evaluation record is generated; and the robustness evaluation record is stored in an evaluation data system, and power transmission and distribution large model robustness quantitative scoring is completed according to the robustness evaluation record. According to the method, the robustness and the security of the large power transmission and distribution model under the attack resistance are improved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and power system technology, specifically relating to a robustness evaluation method for a large-scale power transmission and distribution model based on adversarial training and automatic modeling. Background Technology

[0002] With the deep application of artificial intelligence technology in the power industry, Large Language Models (LLMs) have been widely used in power transmission and distribution production command scenarios, including equipment health diagnosis, fault early warning, and production scheduling optimization. However, existing large power transmission and distribution models are significantly vulnerable to adversarial attacks. Attackers can mislead the model output through minor perturbations (such as adversarial examples generated by FGSM and PGD), leading to misjudgments of equipment status, errors in production command, and even power grid safety accidents. Meanwhile, traditional robustness assessment methods largely rely on general benchmark datasets and lack targeted testing for multimodal expertise in the power transmission and distribution field (such as technical standards, maintenance procedures, inspection images, and operation tables), making it difficult to quantify the model's actual defense capabilities in the subdivided business domains of substation, transmission, and distribution. Furthermore, existing assessment processes are mostly static offline testing, lacking a complete mechanism for automatic modeling and generation of adversarial examples, dynamic defense distillation verification, and closed-loop continuous optimization, making it difficult to support the safe and reliable deployment of large models in complex production environments. Therefore, there is an urgent need for a robustness assessment method for large-scale power transmission and distribution models that integrates adversarial training and automatic modeling, in order to achieve precise disturbance injection, real-time accuracy verification, and full life-cycle robustness quantification, thereby improving the intelligent and safe level of power production command. Summary of the Invention

[0003] The purpose of this invention is to provide a robustness evaluation method for large-scale power transmission and distribution models based on adversarial training and automatic modeling. This method addresses the problems of existing large-scale models being susceptible to adversarial attacks, lacking specificity for power transmission and distribution applications in evaluation, and being difficult to achieve dynamic closed-loop optimization. By generating dedicated multimodal adversarial samples, injecting robust perturbations, and combining adaptive defensive distillation to verify prediction accuracy, this method achieves quantitative scoring and continuous iteration, thereby improving the defensive capabilities and security of large-scale models in substation, transmission, and distribution production command scenarios. This ensures accurate equipment health diagnosis and reliable production command, ultimately improving the overall resilience and security level of intelligent power system operation.

[0004] Specifically, to achieve the above objectives, this invention provides a robustness evaluation method for large-scale power transmission and distribution models based on adversarial training and automatic modeling, characterized in that the method includes: Acquire a multimodal benchmark dataset of power transmission and distribution production command scenarios, and generate an adversarial sample set based on an automatic modeling mechanism; Robust perturbation is injected into the adversarial sample set using adversarial training techniques to generate robust evaluation instructions; The robustness evaluation instructions are combined with an adaptive defensive distillation mechanism to verify the prediction accuracy of the large model and generate robustness evaluation records. The robustness assessment records are stored in the assessment data system, and the robustness quantification score of the large-scale power transmission and distribution model is completed based on the robustness assessment records.

[0005] Optionally, the step of acquiring a multimodal benchmark dataset of power transmission and distribution production command scenarios and generating an adversarial example set based on an automatic modeling mechanism includes: Receive a multimodal benchmark dataset for power transmission and distribution production command scenarios. The multimodal benchmark dataset includes text, images, tables, and labeled question-and-answer pairs from subdivided business domains of substation, transmission, and distribution, with each subdivided business domain covering diverse samples. Multi-dimensional feature extraction is performed on the multimodal benchmark dataset, including language semantic features, visual entity features, table structure features, and reasoning logic labels, to generate feature vectors; Based on the automatic modeling mechanism, an adversarial sample set is calculated according to the feature vector and threat model parameters. The automatic modeling mechanism uses AutoML and NAS to adaptively optimize the perturbation generation network. The threat model parameters include the FGSM gradient perturbation magnitude, PGD iteration steps, and norm constraint range.

[0006] Optionally, the step of injecting robust perturbations into the adversarial sample set using adversarial training techniques to generate robust evaluation instructions includes: A multi-threat adversarial generation array is deployed, which includes a white-box attack module, a black-box attack module, and a migration attack module to generate perturbed input samples in real time. The adversarial sample set is processed by an adversarial training algorithm to generate a robust state evaluation report. The adversarial training algorithm uses TRADES or MART to implement the regularization and boundary constraints of the loss function. Based on the robustness status assessment report and the preset robustness threshold, the defense adjustment amount of the model parameters is calculated through the assessment instruction generation algorithm to generate robustness assessment instructions.

[0007] Optionally, the steps of the evaluation instruction generation algorithm include: Based on the robustness assessment report, identify the prediction bias and confidence decline state of the current model; The optimal adjustment trajectory of the defense strategy is calculated by a predictive robust control model, which comprehensively considers the evaluation accuracy requirements, training dynamic response characteristics and security constraints. Generate robustness evaluation instructions containing multi-dimensional perturbation parameters, including L2 norm perturbation budget, attack success rate threshold, and defense distillation temperature; The robustness assessment instructions are sent to the training execution system, and the defense feedback is monitored in real time.

[0008] Optionally, the step of verifying the prediction accuracy of the large model by combining the robustness evaluation command with the adaptive defensive distillation mechanism and generating a robustness evaluation record includes: The robustness evaluation command invokes the adaptive defense distillation mechanism to calculate the accuracy deviation based on the adversarial sample set and real-time training feedback, thereby verifying the prediction accuracy of the large model. After successful verification, a robustness evaluation record is generated, which includes sample identifier, perturbation path, accuracy deviation, confidence distribution, and robustness state, wherein the robustness evaluation record is associated with the adversarial sample set. The robustness assessment records are subjected to data integrity verification and timestamp marking to ensure the traceability and immutability of the records.

[0009] Optionally, the step of storing the robustness assessment records to the assessment data system and completing the robustness quantification score of the large-scale power transmission and distribution model based on the robustness assessment records includes: The robustness assessment records are formatted and standardized using data processing algorithms to generate a robustness assessment report that conforms to industry standards. The robustness assessment report is stored in the assessment data system, and an associated index is established with the model version number, assessment batch information, and security profile. Based on the robustness status in the robustness assessment record, subsequent assessment dimension scores are triggered, including language understanding robustness, image understanding robustness, multimodal reasoning robustness, and security. An evaluation and coordination mechanism ensures the transmission of parameters and the synchronization of status for scores across all dimensions.

[0010] Optionally, the steps of the adaptive defensive distillation mechanism include: An adaptive defense model for robustness assessment is constructed. Based on knowledge distillation and real-time adversarial data, the defense distillation value D is calculated using the following formula:

[0011] Constraints:

[0012] Where D is the defensive distillation adjustment value. To predict probability bias, For probability bias weights, Let j be the current robustness index for the j-th dimension. For the target robustness, To robustly control weights, Adjust the upper limit for distillation. This is the acceptable robustness lower bound. The adaptive defense model comprehensively considers both adversarial strength and modal interference, and uses a correction factor. For prediction bias Perform environmental adaptive correction; The robustness threshold is determined based on the defensive distillation value D. When the robustness threshold is met, the evaluation verification is confirmed to be successful, and a robustness evaluation record associated with the adversarial sample set is generated.

[0013] Optionally, before the step of acquiring the multimodal benchmark dataset of the power transmission and distribution production command scenario, the method further includes: A benchmark quality assessment system based on multimodal consistency detection is used to automatically detect multimodal benchmark datasets. The detection content includes text factual consistency, image entity annotation accuracy, table structure integrity, and question-answering logic rationality. A benchmark quality assessment report is generated based on the test results. The quality assessment report includes the pass / fail determination, error location marking, and actual measured data of features. Standardized sample identifiers are generated from qualified benchmark data, and these standardized sample identifiers are associated with business domain labels, modality types, and quality levels. The measured data of the features are used as the correction input for the generation of adversarial examples to compensate for the impact of data noise on robustness assessment.

[0014] Optionally, after the step of storing the robustness evaluation record to the evaluation data system, the method further includes: Based on the robustness assessment records, model robustness traceability information is generated, which includes adversarial sample batch tracing, assessment personnel records, disturbance condition logs, security parameter archives, and defense change records. The robust traceability information of the model is securely stored using data encryption technology to generate a traceability data archive; Establish a role-based access control mechanism and provide a source tracing query interface to authorized parties, including model developers, security audit departments, and operations and maintenance service teams; Based on the aforementioned source tracing query interface, robust tracing, attack cause analysis, and responsibility identification are supported throughout the entire lifecycle of the model.

[0015] Optionally, the method further includes a continuous optimization step based on the robustness quantification score, including: Based on the robustness assessment report, the model retraining process is automatically triggered, which integrates adversarial example incremental updates and adaptive adjustment of defense parameters. The actual adversarial performance of the model after going live is collected in real time through a closed-loop feedback mechanism, generating dynamic robust monitoring logs. Based on the dynamic robustness monitoring logs and historical evaluation records, model version iteration and defense strategy optimization are performed. The optimization includes dynamic allocation of disturbance budget, adaptive scheduling of distillation temperature, and multimodal robust weight rebalancing. The optimized model parameters are linked and archived with the evaluation records to support the continuous robust evolution of the power transmission and distribution large model in production command scenarios.

[0016] This invention addresses the technical challenges of existing large-scale power transmission and distribution models, such as susceptibility to adversarial attacks leading to misjudgments, lack of domain specificity in traditional assessments, and the inability of static testing to support dynamic deployment. It provides a precise, automated, and robust end-to-end assessment method that effectively generates and injects perturbations specific to power transmission and distribution, combined with adaptive defensive distillation to verify prediction accuracy in real time. This enables quantitative scoring of security across subdivided business domains of substations, transmission, and distribution, avoiding risks of equipment status misdiagnosis and command errors. Simultaneously, through automatic modeling and closed-loop optimization mechanisms, it significantly improves assessment efficiency and model defense capabilities, supports continuous iteration and full lifecycle traceability, ensuring stable and reliable operation of the large model in complex production environments. This substantially enhances the intelligent security level and system resilience of power production command. Attached Figure Description

[0017] Figure 1 The present invention provides a flowchart of a robustness evaluation method for a large-scale power transmission and distribution model based on adversarial training and automatic modeling. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; in the description of this application, unless otherwise stated, "multiple" means two or more.

[0020] To more clearly illustrate the technical solution of the present invention, the present invention will be described in detail below with reference to specific embodiments, but it should not be construed as a limitation on the scope of protection of the present invention.

[0021] like Figure 1 As shown, this embodiment realizes an automated closed-loop process for robustness assessment of large-scale power transmission and distribution models through the aforementioned method, effectively improving the model's predictive stability and security under adversarial attacks, reducing the risk of equipment misdiagnosis, and ensuring reliable production command.

[0022] Step S01: Obtain a multimodal benchmark dataset of power transmission and distribution production command scenario, and generate an adversarial sample set based on an automatic modeling mechanism.

[0023] Specifically, multimodal benchmark datasets are collected from the power grid dispatch center and inspection database, including substation operation log text, drone inspection images of transmission lines, sensor table data of distribution cabinets, and corresponding manually labeled question-and-answer pairs, to ensure that the data covers actual production environments such as high temperature, high humidity, and electromagnetic interference.

[0024] Furthermore, an AutoML (Automated Machine Learning) platform is employed to automatically search for perturbations to generate network structures. Simultaneously, NAS (Neural Architecture Search) is used to optimize the number of network layers and connections. Based on the semantic and visual entity features of the dataset, an adversarial example set containing gradient perturbations and pixel offsets is generated to simulate both white-box and black-box attack scenarios. The adversarial example set preserves the integrity of the original multimodal information, adding perturbations only within a perceptible range to ensure the evaluation process closely approximates real-world attack threats.

[0025] Step S02: Inject robust perturbation into the adversarial sample set using adversarial training techniques to generate robust evaluation instructions.

[0026] Specifically, the adversarial sample set is input into the pre-trained power transmission and distribution model. The TRADES (TRadeoff-inspired Adversarial Defense via Surrogate-loss minimization) algorithm is used to calculate the loss difference between clean and perturbed samples, injecting robust perturbations to allow the model to learn the adversarial boundary distribution. During the injection process, the gradient norm and confidence changes are monitored in real time, and the perturbation intensity is dynamically adjusted to avoid excessive perturbation that could cause model collapse. The generated robustness evaluation instructions contain the perturbation parameter sequence and the expected defense target, directly guiding the adjustment of the execution mechanism in the subsequent validation phase.

[0027] Step S03: Verify the prediction accuracy of the large model by combining the robustness evaluation command with the adaptive defensive distillation mechanism, and generate a robustness evaluation record.

[0028] Specifically, robustness assessment instructions are executed, transferring soft labels from the teacher model to the student model, while adaptively adjusting the distillation temperature to balance cleanliness accuracy and robustness accuracy. The consistency of the equipment health diagnosis output is verified under multimodal input. The predicted probability distributions before and after perturbation are compared, and the accuracy deviation and confidence level decrease are calculated. If a preset threshold is met, the verification is passed.

[0029] Understandably, generating robustness assessment records includes sample identifiers, perturbation paths, accuracy deviation values, confidence distribution curves, and final robustness state markers, facilitating subsequent traceability analysis.

[0030] Step S04: Store the robustness assessment record in the assessment data system, and complete the robustness quantification score of the large power transmission and distribution model based on the robustness assessment record.

[0031] Specifically, robustness assessment records are uploaded to a distributed assessment data system via an encrypted channel, establishing an index linking the model version number and assessment batch, supporting cross-regional data synchronization. Based on multi-dimensional indicators in the records, robustness scores for language understanding, image understanding, multimodal reasoning, and security are automatically calculated, generating a comprehensive quantitative score report.

[0032] Understandably, the scoring result is directly fed back into the model iteration process, enabling continuous safety optimization of large models in production command scenarios.

[0033] In some embodiments, the following methods can be used to achieve accurate and automated generation of multimodal adversarial samples specific to power transmission and distribution, improve the targeting of the evaluation process and the realism of attack simulation, and significantly enhance the robustness testing efficiency of large models in subdivided business domains.

[0034] The system receives a multimodal benchmark dataset for power transmission and distribution production command scenarios. The multimodal benchmark dataset includes text, images, tables, and labeled question-and-answer pairs from subdivided business domains of substation, transmission, and distribution, with each subdivided business domain covering diverse samples.

[0035] Specifically, multimodal benchmark datasets are synchronously acquired from the China Southern Power Grid's unified data platform. The substation domain includes transformer status report text and infrared thermal imaging images; the transmission domain includes tower tilt angle tables and drone aerial images; and the distribution domain includes switchgear operation logs and fault waveform diagrams, along with expert-annotated health diagnosis Q&A pairs. A data sampling strategy ensures a balanced distribution of samples across each sub-business domain, covering various operating conditions such as normal operation, minor defects, and severe faults, simulating diverse production scenarios such as high temperatures, rainy seasons, and lightning interference. All samples are anonymized and stored uniformly in a Hadoop distributed file system, supporting high-concurrency reads and version management.

[0036] Multi-dimensional feature extraction is performed on the multimodal benchmark dataset, including language semantic features, visual entity features, table structure features, and reasoning logic labels, to generate feature vectors.

[0037] Specifically, the text part uses a domain-pre-trained BERT variant to extract language semantic features and identify embedded professional terms such as "overheating" and "insulation degradation"; the image part uses ResNet-50 to extract visual entity features and locate target areas such as equipment corrosion and bird nests; the table part uses Table Transformer to parse structural features and capture numerical anomalies and trend labels.

[0038] Furthermore, the inference logic labels are converted into one-hot vectors, which are then concatenated with the aforementioned features to form a unified multi-dimensional feature vector. The dimension is controlled within 2048 to balance the computational load. The feature extraction process runs in parallel on a GPU cluster, with a single batch processing latency of less than 2 seconds, ensuring efficient preprocessing of large-scale datasets.

[0039] Based on the automatic modeling mechanism, an adversarial sample set is calculated according to the feature vector and threat model parameters. The automatic modeling mechanism adaptively optimizes the perturbation generation network through AutoML (Automated Machine Learning) and NAS (Neural Architecture Search). The threat model parameters include the gradient perturbation magnitude of FGSM (Fast Gradient Sign Method), the number of iteration steps of PGD (Projected Gradient Descent), and the norm constraint range.

[0040] Specifically, the AutoML framework is activated to search for the optimal perturbation generator hyperparameters in the feature vector space. Simultaneously, NAS dynamically constructs a convolutional-attention hybrid network structure, outputting a perturbation mapping function for multimodal features. The FGSM perturbation amplitude is set to 0.01 to 0.03, the PGD iteration steps are 10 to 40, and the norm constraint range is limited to a combination of L∞ and L2. The perturbation budget is adaptively allocated based on feature sensitivity, generating pixel-level and semantic-level composite adversarial examples.

[0041] Understandably, the entire computation process runs in a containerized environment, automatically saving the optimal network weights and perturbation samples to the object storage service to form a reusable set of adversarial examples, supporting batch calls of subsequent evaluation instructions.

[0042] In some embodiments, the following methods are used to achieve accurate injection and instruction generation of robust perturbations in multi-threat scenarios, thereby improving the defense response speed and assessment controllability of large models against white-box, black-box, and migration attacks.

[0043] A multi-threat adversarial generation array is deployed, which includes a white-box attack module, a black-box attack module, and a migration attack module to generate perturbed input samples in real time.

[0044] Specifically, a containerized adversarial generation array is deployed on a Kubernetes cluster. The white-box attack module connects to the model gradient interface, the black-box attack module uses query-based perturbation estimation, and the migration attack module uses the source model to generate cross-domain samples.

[0045] Each module processes the multimodal benchmark dataset in parallel and outputs perturbed text embedding offsets, image pixel noise, and table numerical fine-tuning samples in real time, with perturbation generation latency controlled within 500 milliseconds.

[0046] Understandably, the array supports dynamic expansion and automatically schedules computing resources based on the assessed load to ensure real-time requirements in high-pressure production command scenarios.

[0047] The adversarial sample set is processed by an adversarial training algorithm to generate a robust state evaluation report. The adversarial training algorithm uses TRADES (TRadeoff-inspired Adversarial Defense via Surrogate-loss minimization) or MART (Misclassification Aware Regularized Adversarial Training) to implement the regularization and boundary constraints of the loss function.

[0048] Specifically, the perturbation samples and the original samples are input into the power transmission and distribution large model in pairs. TRADES is used to calculate the trade-off loss between natural error and robust error, or MART is enabled to introduce a misclassification confidence regularization term and perform multiple rounds of small-batch training.

[0049] Furthermore, L2 regularization and early stopping mechanisms are applied during training to monitor the decision surface distance of boundary samples and generate a robust state evaluation report that includes clean accuracy, robust accuracy, loss curve, and gradient statistics.

[0050] The report is serialized in JSON format, which facilitates automated parsing and decision-making by the subsequent instruction generation algorithm.

[0051] Based on the robustness status assessment report and the preset robustness threshold, the defense adjustment amount of the model parameters is calculated through the assessment instruction generation algorithm to generate robustness assessment instructions.

[0052] Specifically, the robust accuracy and confidence decline indicators in the analysis report are compared with preset thresholds (such as robust accuracy not less than 85%), triggering the evaluation instruction generation algorithm. The algorithm uses proportional-integral control logic to calculate the learning rate decay coefficient, distillation temperature increment, and regularization weight adjustment to form an optimized defense parameter scheme.

[0053] Understandably, the final robustness evaluation instructions are encapsulated as an executable script, containing parameter adjustment sequences and verification trigger conditions, and are directly sent to the training execution system to achieve closed-loop defense enhancement.

[0054] In some embodiments, the following methods are used to achieve predictive optimization of defense strategies and refined instruction issuance, thereby improving the robustness adjustment accuracy and execution feedback efficiency of large models in dynamic adversarial environments.

[0055] Based on the robustness assessment report, identify the prediction bias and confidence decline state of the current model.

[0056] Specifically, the JSON fields in the robustness assessment report are parsed to extract the difference in predicted probabilities before and after the disturbance and the softmax confidence decay curve, marking misclassified samples of "normal → minor fault" or "minor → severe" in equipment health diagnosis. A sliding window statistical method is used to calculate the mean and variance of the deviation, identify high-risk modalities (such as visual misjudgments caused by image noise), and generate a deviation heatmap and a confidence decline state summary.

[0057] Understandably, this identification process runs on edge computing nodes, with a single sample processing latency of less than 100 milliseconds, supporting real-time early warning on the production site.

[0058] The optimal adjustment trajectory of the defense strategy is calculated by a predictive robust control model, which comprehensively considers the evaluation accuracy requirements, training dynamic response characteristics, and security constraints.

[0059] Specifically, a pre-trained robust control model (based on an LSTM sequence predictor) is loaded, the current bias state and historical training trajectory are input, and the defense parameter adjustment sequence for the next 10 steps is output. The model integrates a multi-objective optimization layer, which simultaneously meets the evaluation accuracy requirements (e.g., a 5% improvement in robust accuracy), training dynamic response characteristics (e.g., a learning rate convergence time of less than 2 hours), and safety constraints (e.g., the parameter adjustment range does not exceed 10% of the pre-trained weights).

[0060] Understandably, the adjustment trajectory is saved in the form of a timestamp sequence, supporting visual playback, which facilitates the auditing of the defense process by operation and maintenance personnel.

[0061] Generate robustness evaluation instructions containing multi-dimensional perturbation parameters, including L2 norm perturbation budget, attack success rate threshold, and defense distillation temperature.

[0062] Specifically, based on the optimal adjustment trajectory, a combination of parameters is encapsulated, including the L2 norm perturbation budget (e.g., 0.5), the attack success rate threshold (e.g., below 15%), and the defense distillation temperature (e.g., initially 4.0 and gradually decaying to 2.0), to generate a structured robustness evaluation instruction.

[0063] Furthermore, the instructions include execution priority and timeout mechanisms to ensure that high-risk samples are processed first, with the overall generation cycle kept within 1 second. The instructions use the ProtoBuf serialization format for lightweight transmission to the training execution system, supporting cross-datacenter synchronization.

[0064] The robustness evaluation instructions are sent to the training execution system, and the execution feedback is monitored in real time.

[0065] Specifically, robustness evaluation instructions are pushed to the distributed training execution system via message queues (such as Kafka) to trigger hot parameter updates and incremental training tasks.

[0066] Furthermore, the system transmits execution status in real time, including parameter application success rate, training loss convergence curve, and anomaly alarms. The monitoring panel displays three statuses: green (normal), yellow (delay), and red (failure). Feedback data is automatically archived to the evaluation data system, forming a closed-loop log to support subsequent evaluation iterations and fault tracing.

[0067] In some embodiments, the following methods are used to achieve adaptive verification and standardized generation of records for large model prediction accuracy, thereby improving the traceability and consistency of robustness assessment.

[0068] The robustness evaluation command invokes the adaptive defense distillation mechanism to calculate the accuracy deviation based on the adversarial sample set and real-time training feedback, thereby verifying the prediction accuracy of the large model.

[0069] Specifically, upon receiving the robustness assessment instruction, an adaptive defensive distillation mechanism is activated to extract soft labels from the teacher model and inject them into the student model. Simultaneously, the distillation weights are dynamically adjusted based on the perturbation intensity of the adversarial example set. The loss value and gradient norm in the training feedback are collected in real time and compared sample by sample with the original clean sample output. The accuracy deviation (such as the decrease in F1 score) is calculated, and it is verified whether the production command threshold is met (e.g., equipment diagnostic accuracy is not less than 90%).

[0070] Understandably, the verification process is executed in parallel on a highly available GPU cluster, supporting synchronous comparison of multiple version models to ensure real-time performance and fault tolerance in power transmission and distribution scenarios.

[0071] After successful verification, a robustness evaluation record is generated, which includes sample identifier, perturbation path, accuracy deviation, confidence distribution, and robustness state, wherein the robustness evaluation record is associated with the adversarial sample set.

[0072] Specifically, once the accuracy deviation is below the threshold, the verification is considered successful, and the record generation module is automatically triggered to fill in the sample unique identifier (such as UUID), perturbation path trajectory (FGSM step size sequence), accuracy deviation value, confidence histogram, and robust status label (pass / needs optimization).

[0073] Furthermore, a hash chain technique is used to establish an association index between each record and the corresponding set of adversarial examples, ensuring data integrity and immutability. Records are stored in structured Parquet format, facilitating subsequent statistical analysis and visualization on the big data platform.

[0074] The robustness assessment records are subjected to data integrity verification and timestamp marking to ensure the traceability and immutability of the records.

[0075] Specifically, the SHA-256 algorithm is used to verify the integrity of the recorded content, generating a digital fingerprint which is then appended to the end of the record. A Trusted Timestamp Service (TSA) is introduced to assign an authoritative timestamp to each record, accurate to the millisecond level, and embedded in a blockchain sidechain to achieve distributed consensus.

[0076] Understandably, the automated verification and marking process is embedded into the record generation pipeline, supporting audit departments to verify the records with a single click, ensuring the legal validity of robust evidence throughout the entire lifecycle and meeting the needs of production traceability.

[0077] In some embodiments, the following methods are used to achieve standardized storage of robustness assessment records and automated triggering of multi-dimensional quantitative scoring, thereby improving the standardization of assessment reports and the efficiency of model security iteration.

[0078] The robustness assessment records are formatted and standardized using data processing algorithms to generate a robustness assessment report that conforms to industry standards.

[0079] Specifically, the data processing algorithm pipeline is initiated. First, the diverse fields of the robustness assessment records (such as accuracy deviation values ​​and confidence distribution) are uniformly converted into the power grid industry standard format (based on IEC 61850 extension), redundant metadata is removed, and missing items are filled in. Further, a template engine is applied to generate a visual robustness assessment report, including line graphs showing the accuracy changes before and after disturbances, and heat maps annotating high-risk business areas (such as misjudgments of distribution faults), ensuring that the report complies with the State Grid safety assessment specifications.

[0080] Understandably, the entire process runs on the Spark distributed computing framework, with a single batch of thousands of records processing latency of less than 30 seconds, supporting high throughput for evaluation tasks during peak periods.

[0081] The robustness assessment report is stored in the assessment data system, and an associated index is established with the model version number, assessment batch information, and security profile.

[0082] Specifically, the robustness assessment report is uploaded to the assessment data system through the object storage interface, and the current model version number (e.g., v2.3.1), assessment batch identifier (e.g., Batch-20251117), and corresponding security profile number are bound using metadata tags.

[0083] Furthermore, an inverted index and relational tables are constructed to enable three-dimensional relational queries of versions, batches, and archives, allowing operations and maintenance personnel to trace historical reports with a single click via a web interface. Storage employs a hot / cold tiered strategy, with recent reports stored hot for faster access and archived reports stored cold to reduce costs and ensure long-term compliant retention.

[0084] Based on the robustness status in the robustness assessment record, subsequent assessment dimension scores are triggered, including language understanding robustness, image understanding robustness, multimodal reasoning robustness, and security.

[0085] Specifically, the robustness status field is parsed. If marked as "passed," the scoring engine is automatically triggered to calculate robustness in language understanding (BLEU decline rate after text-based adversarial question answering), robustness in image understanding (mAP decay in visual entity recognition), robustness in multimodal reasoning (F1 score in image-text joint diagnosis), and security (attack success rate). The scores for each dimension are weighted and summed to generate a total score, and weaker dimensions (such as image robustness below 80%) are marked to provide priority guidance for model optimization.

[0086] Understandably, the scoring process integrates a rules engine, supports custom weight configurations, and adapts to the evaluation focus of different business domains such as substation, transmission, and distribution.

[0087] An evaluation and coordination mechanism ensures the transmission of parameters and the synchronization of status for scores across all dimensions.

[0088] Specifically, the deployment assessment coordination mechanism is based on a message bus, which broadcasts intermediate parameters of each dimension of scoring (such as confidence threshold and perturbation budget) to downstream modules in real time to ensure consistency of language, image, and multimodal scoring status.

[0089] Furthermore, distributed locks and heartbeat detection are introduced to prevent concurrent scoring conflicts, and automatic retries and logging are performed when synchronization fails. This mechanism supports cross-regional data center collaboration, ensuring the global consistency and real-time availability of scoring results under the unified national power grid evaluation standard.

[0090] In some embodiments, the following method is used to achieve adaptive fine control and verification closed loop for defensive distillation, thereby improving the robustness and reliability of large models under multi-dimensional adversarial conditions.

[0091] An adaptive defense model for robustness assessment is constructed. Based on knowledge distillation and real-time adversarial data, the defense distillation adjustment value D is calculated using the following formula:

[0092] Constraints: .

[0093] Specifically, in the formula The preset probability bias weights are derived from the configuration parameters of the adaptive defense model and are used to balance the impact of prediction probability bias. The prediction probability bias is calculated from the difference between the predicted probabilities of perturbed samples and clean samples in the real-time training feedback. The robust control weights are derived from the hyperparameter settings during model training and are used to adjust the weights that control the deviation of the robustness index. The current robustness index for the j-th dimension is derived from the real-time measurement values ​​of each dimension in the robustness state assessment report. The target robustness rate is derived from the preset robustness threshold configuration.

[0094] Furthermore, the calculation process employs a gradient descent optimizer to iteratively minimize the objective function until convergence, while simultaneously verifying the constraints at each iteration. (in, The upper limit for distillation adjustment (derived from model safety constraint parameters) and (in The acceptable robust lower limit (derived from the evaluation accuracy requirements) ensures that the adjustment value meets the specified range.

[0095] Understandably, this formula is calculated directly based on the joint optimization of knowledge distillation soft tag transfer and real-time adversarial data, and outputs a single defense distillation adjustment value D for subsequent steps.

[0096] The adaptive defense model comprehensively considers both adversarial strength and modal interference, and uses a correction factor. For prediction bias Perform environmental adaptive correction.

[0097] Specifically, the correction factor The data is derived from real-time monitoring values ​​of adversarial strength and modal interference, and is calculated through weighted fusion to account for prediction bias. Perform a multiplicative adjustment, i.e., update to In order to compensate for the impact of the external environment.

[0098] Furthermore, the revised By directly substituting the values ​​into the formula, the optimization process is ensured to use the corrected input values. This correction step serves as a preprocessing step before formula calculation, guaranteeing the input accuracy of the adaptive defense model.

[0099] The robustness threshold is determined based on the defensive distillation adjustment value D.

[0100] Specifically, based on the defensive distillation adjustment value D output by the formula, an absolute value comparison is directly performed to verify it. And examine each dimension. .

[0101] Furthermore, if all constraints are satisfied, the evaluation process is deemed to meet the robustness threshold; otherwise, the specific constraints violated are recorded. The judgment logic strictly relies on the formula results and constraints to ensure the objectivity of the evaluation.

[0102] When the robustness threshold is met, the evaluation verification is confirmed to be successful, and a robustness evaluation record associated with the adversarial sample set is generated.

[0103] Specifically, once the threshold is met, the verification is confirmed, and a robustness assessment record is generated, which includes the defense distillation adjustment value D calculated by the formula and the correction factor. and constraint verification status.

[0104] Furthermore, sample identifiers are used to establish a link between records and adversarial sample sets, supporting traceability. Records generate fixed formula optimization and constraint verification results, directly serving the evaluation closed loop.

[0105] In some embodiments, the following methods are used to achieve pre-processing quality control and noise compensation for multimodal benchmark datasets, thereby improving the purity of adversarial sample generation and the baseline reliability of robustness assessment.

[0106] A benchmark quality assessment system based on multimodal consistency detection is used to automatically detect multimodal benchmark datasets. The detection content includes text factual consistency, image entity annotation accuracy, table structure integrity, and question-answer logic rationality.

[0107] Specifically, after launching the benchmark quality assessment system and loading the multimodal benchmark dataset, the text part calls the fact-checking model to compare the equipment parameters with the standard procedures, and the image part uses the object detection algorithm to verify the overlap between the entity annotation box and the actual defect location.

[0108] Furthermore, the table section parses cell dependencies to check structural integrity, while the question-and-answer section verifies the causal consistency of the question-and-answer relationship through logical reasoning chains, automatically marking inconsistent samples. The detection process runs in parallel within containerized microservices, supporting second-level scanning of thousands of samples in batches, ensuring the professional accuracy of power transmission and distribution production data.

[0109] A benchmark quality assessment report is generated based on the test results. The quality assessment report includes the pass / fail determination, error location marking, and actual feature measurement data.

[0110] Specifically, the test markers are summarized to generate a baseline quality assessment report. The pass / fail criteria are presented as pass / fail labels, and error locations are marked with text line numbers, image coordinates, and table cell paths. The report also includes additional measured data, such as text factual error rate, mean IoU of image annotations, table missing rate, and the proportion of logical errors in question-and-answer sessions, forming a quantitative quality profile.

[0111] Understandably, the report is output in both PDF and JSON formats to facilitate manual review and automated system parsing.

[0112] Standardized sample identifiers are generated for qualified benchmark data, and these standardized sample identifiers are associated with business domain labels, modality types, and quality levels.

[0113] Specifically, after screening qualified benchmark data, standardized sample identifiers (such as T&D-Substation-Text-LevelA-001) are automatically assigned, and associated business domain labels (substation / transmission / distribution), modal types (text / image / table), and quality levels (A / B / C) are associated.

[0114] Furthermore, the identifier is embedded in metadata fields, supporting database indexing and fast retrieval. This identifier system ensures the uniqueness and traceability of qualified data in subsequent processes.

[0115] The measured data of the features are used as the correction input for the generation of adversarial examples to compensate for the impact of data noise on robustness assessment.

[0116] Specifically, noise statistics (such as image blurring and text misspelling rate) in the measured feature data are input into the automatic modeling mechanism as a correction factor for the perturbation budget, dynamically scaling the FGSM amplitude or PGD steps.

[0117] Furthermore, when generating the adversarial example set, high-noise samples are compensated first to reduce the interference of baseline data defects on robustness assessment. This compensation mechanism forms a closed-loop control of quality and perturbation, ensuring that the assessment results reflect the model's true defensive capabilities rather than data flaws.

[0118] In some embodiments, the following methods are used to achieve secure archiving and full lifecycle traceability of robustness assessment records, thereby improving the compliance and accountability efficiency of model security audits.

[0119] Based on the robustness assessment records, model robustness traceability information is generated, which includes adversarial sample batch tracing, assessment personnel records, perturbation condition logs, security parameter archives, and defense change records.

[0120] Specifically, after parsing the robustness assessment records, model robustness traceability information is automatically generated. The adversarial sample batch traceability includes sample set hash and generation timestamp, and the assessment personnel record is bound to the operator's employee number and digital signature.

[0121] The disturbance condition log records the FGSM amplitude, PGD steps, and norm constraints; the safety parameter archive saves the distillation temperature and threshold configurations; and the defense change log lists comparisons before and after parameter adjustments. Traceability information is organized in a chain structure, supporting backtracking of the entire evaluation chain along a timeline.

[0122] The robust traceability information of the model is securely stored using data encryption technology to generate a traceability data archive.

[0123] Specifically, the AES-256 algorithm is used to symmetrically encrypt the robust traceability information of the model. The key is managed by the hardware security module, and then a traceability data archive containing ciphertext and metadata is generated.

[0124] Furthermore, files are appended with digital fingerprints and version numbers to ensure confidentiality and integrity during transmission and storage. Encrypted storage complies with the power grid information security level protection requirements and supports offline backup and disaster recovery.

[0125] Establish a role-based access control mechanism and provide a source tracing query interface to authorized parties, including model developers, security audit departments, and operations and maintenance service teams.

[0126] Specifically, a Role-Based Access Control (RBAC) mechanism is deployed to assign read and write permissions to model developers, read-only audit permissions to the security audit department, and query and export permissions to the operations and maintenance service team. A source tracing query interface is exposed through an API gateway, supporting conditional filtering (such as by batch, time, or personnel) and paginated returns. Access logs record operation behavior in real time, supporting compliance auditing and anomaly alerts.

[0127] Based on the aforementioned source tracing query interface, robust tracing, attack cause analysis, and responsibility identification are supported throughout the entire lifecycle of the model.

[0128] Specifically, users input sample identifiers or time ranges through the source tracing query interface, and the system returns a complete evaluation chain, including the perturbation injection path, defense adjustment records, and final score. The built-in analysis module automatically highlights successful attack samples and defense failure points, generates a responsibility delineation report, and clarifies the responsibilities of the evaluation personnel and model versions.

[0129] Understandably, this function covers the entire lifecycle of the model, from data collection to scoring and publication, ensuring rapid location and closed-loop rectification of safety incidents in the large-scale power transmission and distribution model.

[0130] In some embodiments, the following methods are generally used to achieve closed-loop feedback of robustness quantification scoring and continuous model evolution, thereby improving the long-term safety adaptability and defense resilience of the power transmission and distribution large model in production command scenarios.

[0131] Based on the robustness assessment report, the model retraining process is automatically triggered, which integrates adversarial example incremental updates and adaptive adjustment of defense parameters.

[0132] Specifically, the comprehensive quantitative score in the robustness evaluation report is analyzed. If it is lower than the preset safety baseline (e.g., 85 points), the model retraining process is automatically triggered by the scheduler, and the latest adversarial example set is loaded for incremental updates.

[0133] Furthermore, defense parameters are adaptively adjusted during retraining, including increasing the distillation temperature, tightening the perturbation budget, or increasing the regularization strength, to ensure that new samples are seamlessly integrated into existing knowledge. The process runs on an automated CI / CD pipeline, supporting minimal downtime and enabling hot model updates.

[0134] A closed-loop feedback mechanism is used to collect the actual adversarial performance of the model after it goes live in real time, generating dynamic robust monitoring logs.

[0135] Specifically, after the online model is connected to production traffic, a closed-loop feedback mechanism captures adversarial inputs (such as abnormal inspection images and tampered operation logs) in real time, compares the model output with expectations, and calculates the actual robust accuracy and attack success rate. Hourly data aggregation generates dynamic robust monitoring logs, recording time-series metrics and details of abnormal events.

[0136] Understandably, logs are pushed to a unified monitoring platform, supporting alarm threshold triggering and visualization dashboard display.

[0137] Based on the dynamic robustness monitoring logs and historical evaluation records, model version iterations and defense strategy optimizations are performed. These optimizations include dynamic allocation of disturbance budgets, adaptive scheduling of distillation temperature, and multimodal robust weight rebalancing.

[0138] Specifically, dynamic robustness monitoring logs and historical evaluation records are integrated and input into the strategy optimization engine to automatically reallocate perturbation budgets (e.g., increasing the image domain by 20%), schedule distillation temperature curves, and adjust multimodal weights (text:image:table = 0.4:0.4:0.2). This generates a new version of the model configuration file and optimization report, triggering automated testing and canary releases.

[0139] Understandably, the optimization process uses A / B testing to verify the effects and ensure that the robustness of the iterative version is improved by no less than 5%.

[0140] The optimized model parameters are linked and archived with the evaluation records to support the continuous robust evolution of the power transmission and distribution large model in production command scenarios.

[0141] Specifically, the optimized model parameters (such as weight files and configuration JSON) are associated with the corresponding evaluation records via version numbers and archived in the evaluation data system, forming a complete evolutionary chain. The archive supports snapshot rollback and difference comparison, making it easy to trace the input and output of each optimization.

[0142] Understandably, this archiving mechanism ensures the continuous safety evolution and compliance auditing of the model during the long-term operation of substations, transmission lines, and distribution systems.

[0143] The above description is merely an exemplary embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention specification and drawings under the technical concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A robustness evaluation method for a large-scale power transmission and distribution model based on adversarial training and automatic modeling, characterized in that, The method includes: Acquire a multimodal benchmark dataset of power transmission and distribution production command scenarios, and generate an adversarial sample set based on an automatic modeling mechanism; Robust perturbation is injected into the adversarial sample set using adversarial training techniques to generate robust evaluation instructions; The robustness evaluation instructions are combined with an adaptive defensive distillation mechanism to verify the prediction accuracy of the large model and generate robustness evaluation records. The robustness assessment records are stored in the assessment data system, and the robustness quantification score of the large-scale power transmission and distribution model is completed based on the robustness assessment records.

2. The method as described in claim 1, characterized in that, The steps of acquiring a multimodal benchmark dataset of power transmission and distribution production command scenarios and generating an adversarial example set based on an automatic modeling mechanism include: Receive a multimodal benchmark dataset for power transmission and distribution production command scenarios. The multimodal benchmark dataset includes text, images, tables, and labeled question-and-answer pairs from subdivided business domains of substation, transmission, and distribution, with each subdivided business domain covering diverse samples. Multi-dimensional feature extraction is performed on the multimodal benchmark dataset, including language semantic features, visual entity features, table structure features, and reasoning logic labels, to generate feature vectors; Based on the automatic modeling mechanism, an adversarial sample set is calculated according to the feature vector and threat model parameters. The automatic modeling mechanism uses AutoML and NAS to adaptively optimize the perturbation generation network. The threat model parameters include the FGSM gradient perturbation magnitude, PGD iteration steps, and norm constraint range.

3. The method as described in claim 1, characterized in that, The step of robustly perturbing the adversarial sample set using adversarial training techniques to generate robust evaluation instructions includes: A multi-threat adversarial generation array is deployed, which includes a white-box attack module, a black-box attack module, and a migration attack module to generate perturbed input samples in real time. The adversarial sample set is processed by an adversarial training algorithm to generate a robust state evaluation report. The adversarial training algorithm uses TRADES or MART to implement the regularization and boundary constraints of the loss function. Based on the robustness status assessment report and the preset robustness threshold, the defense adjustment amount of the model parameters is calculated through the assessment instruction generation algorithm to generate robustness assessment instructions.

4. The method as described in claim 3, characterized in that, The steps of the evaluation instruction generation algorithm include: Based on the robustness assessment report, identify the prediction bias and confidence decline state of the current model; The optimal adjustment trajectory of the defense strategy is calculated by a predictive robust control model, which comprehensively considers the evaluation accuracy requirements, training dynamic response characteristics and security constraints. Generate robustness evaluation instructions containing multi-dimensional perturbation parameters, including L2 norm perturbation budget, attack success rate threshold, and defense distillation temperature; The robustness assessment instructions are sent to the training execution system, and the defense feedback is monitored in real time.

5. The method as described in claim 1, characterized in that, The step of verifying the prediction accuracy of the large model by combining the robustness evaluation command with the adaptive defensive distillation mechanism and generating a robustness evaluation record includes: The robustness evaluation command invokes the adaptive defense distillation mechanism to calculate the accuracy deviation based on the adversarial sample set and real-time training feedback, thereby verifying the prediction accuracy of the large model. After successful verification, a robustness evaluation record is generated, which includes sample identifier, perturbation path, accuracy deviation, confidence distribution, and robustness state, wherein the robustness evaluation record is associated with the adversarial sample set. The robustness assessment records are subjected to data integrity verification and timestamp marking to ensure the traceability and immutability of the records.

6. The method as described in claim 1, characterized in that, The steps of storing the robustness assessment records in the assessment data system and completing the robustness quantification score of the large-scale power transmission and distribution model based on the robustness assessment records include: The robustness assessment records are formatted and standardized using data processing algorithms to generate a robustness assessment report that conforms to industry standards. The robustness assessment report is stored in the assessment data system, and an associated index is established with the model version number, assessment batch information, and security profile. Based on the robustness status in the robustness assessment record, subsequent assessment dimension scores are triggered, including language understanding robustness, image understanding robustness, multimodal reasoning robustness, and security. An evaluation and coordination mechanism ensures the transmission of parameters and the synchronization of status for scores across all dimensions.

7. The method according to any one of claims 1 to 6, characterized in that, The steps of the adaptive defensive distillation mechanism include: An adaptive defense model for robustness assessment is constructed. Based on knowledge distillation and real-time adversarial data, the defense distillation value D is calculated using the following formula: ； Constraints: ;; Where D is the defensive distillation adjustment value. To predict probability bias, For probability bias weights, Let j be the current robustness index for the j-th dimension. For the target robustness, To robustly control weights, Adjust the upper limit for distillation to an acceptable robust lower limit; The adaptive defense model comprehensively considers both adversarial strength and modal interference, and uses a correction factor. For prediction bias Perform environmental adaptive correction; The robustness threshold is determined based on the defense distillation value D. If the robustness threshold is met, the evaluation is confirmed to be successful, and a robustness evaluation record associated with the adversarial sample set is generated.

8. The method as described in claim 1, characterized in that, Before the step of acquiring the multimodal benchmark dataset of the power transmission and distribution production command scenario, the method further includes: A benchmark quality assessment system based on multimodal consistency detection is used to automatically detect multimodal benchmark datasets. The detection content includes text factual consistency, image entity annotation accuracy, table structure integrity, and question-answering logic rationality. A benchmark quality assessment report is generated based on the test results. The quality assessment report includes the pass / fail determination, error location marking, and actual measured data of features. Standardized sample identifiers are generated from qualified benchmark data, and these standardized sample identifiers are associated with business domain labels, modality types, and quality levels. The measured data of the features are used as the correction input for the generation of adversarial examples to compensate for the impact of data noise on robustness assessment.

9. The method as described in claim 1, characterized in that, After the step of storing the robustness evaluation records to the evaluation data system, the method further includes: Based on the robustness assessment records, model robustness traceability information is generated, which includes adversarial sample batch tracing, assessment personnel records, disturbance condition logs, security parameter archives, and defense change records. The robust traceability information of the model is securely stored using data encryption technology to generate a traceability data archive; Establish a role-based access control mechanism and provide a source tracing query interface to authorized parties, including model developers, security audit departments, and operations and maintenance service teams; Based on the aforementioned source tracing query interface, robust tracing, attack cause analysis, and responsibility identification are supported throughout the entire lifecycle of the model.

10. The method as described in claim 1, characterized in that, The method further includes a continuous optimization step based on the robustness quantification score, including: Based on the robustness assessment report, the model retraining process is automatically triggered, which integrates adversarial example incremental updates and adaptive adjustment of defense parameters. The actual adversarial performance of the model after going live is collected in real time through a closed-loop feedback mechanism, generating dynamic robust monitoring logs. Based on the dynamic robustness monitoring logs and historical evaluation records, model version iteration and defense strategy optimization are performed. The optimization includes dynamic allocation of disturbance budget, adaptive scheduling of distillation temperature, and multimodal robust weight rebalancing. The optimized model parameters are linked and archived with the evaluation records to support the continuous robust evolution of the power transmission and distribution large model in production command scenarios.

Images