An electric power graph paper understanding large model self-evolution training method

Abstract

The application belongs to the technical field of large model training, and particularly relates to a power drawing understanding large model self-evolution training method. Based on original power engineering drawings, a game and consensus mechanism of multi-model reasoning is used to construct a supervised training set, a preference training set and an expert fine-tuning set, and negative samples are explicitly labeled; based on the supervised training set, the training task is deconstructed and data is arranged, a parameter efficient fine-tuning and a gradual injection mode are used to train a base multi-modal large model, and a basic training model is obtained; based on the basic training model and the preference training set, direct preference optimization is performed on the preference data containing explicitly labeled negative samples, and general data is mixed to anchor the generalization ability of the model, forming an alignment model special for power drawing review; the alignment model special for power drawing review is deployed to actual application, its effect is evaluated, and a data backflow mechanism is established, and a new round of iteration cycle is driven by continuously collecting low confidence and rejected samples.

Description

Technical Field

[0001] This invention belongs to the field of large model training technology, specifically relating to a self-evolutionary training method for large models of power drawing interpretation. Background Technology

[0002] In the field of power engineering, design drawings are the core carrier throughout the entire process, encompassing various forms and massive amounts of complex information. However, current drawing review mainly relies on manual labor, which has problems such as long processing time, strong subjectivity, and easy omissions and errors. Moreover, power engineering has extremely high safety requirements, and the omission of key review points may lead to major accidents and losses. Therefore, realizing automatic drawing understanding, risk detection, and review assistance has become an urgent issue to be addressed.

[0003] Previously, the industry attempted to improve review efficiency using methods based on computer vision and rule engines, such as using OCR and image processing technologies to recognize text symbols, constructing graph structures, and then using rule engines to check electrical logic. However, such methods have obvious limitations. First, they are sensitive to drawing formats and layouts, have poor generalization ability, are difficult to adapt to changes, and are insufficiently compatible with non-standard drawings; second, they lack deep semantic understanding and comprehensive reasoning capabilities, and cannot support the judgment of complex review points; third, the system is complex to build and maintain, has high costs, and is difficult to iterate sustainably.

[0004] Multimodal large model technology brings new possibilities to the intelligent review of power drawings, but direct application or simple fine-tuning faces many challenges. First, the model lacks professional knowledge in the power field, which can easily lead to "professional illusions." Second, simple mixed fine-tuning can easily cause negative interference between tasks and catastrophic amnesia, reducing overall robustness. Third, conventional supervision and fine-tuning are difficult to optimize for high-risk categories and cannot meet the zero-tolerance requirements for power safety. Fourth, the lack of a systematic data closed loop and self-evolution mechanism makes it difficult to form a positive "data-model" cycle.

[0005] Data-driven AI methods and model alignment techniques also have limitations when applied to the field of power drawings. High-quality data annotation relies on domain experts, which is costly and time-consuming; relying on a single strong model to generate pseudo-labels for self-training introduces uncertainty risks; course learning and uncertainty modeling methods have low coupling with business risks, making it impossible to focus on training high-risk samples; and mainstream model alignment targets lack preference optimization mechanisms for strong engineering safety constraints.

[0006] In summary, current technologies face key bottlenecks in the intelligent review of power engineering drawings. There is a general lack of an integrated self-evolving mechanism for data and models specific to power engineering drawings; a lack of uncertainty measurement and sample stratification strategies based on multi-model game consensus; and the absence of a complete, integrated training framework. This makes it difficult to improve the ability to identify and process high-risk review points while ensuring the model's general capabilities. Therefore, the industry urgently needs a new technological solution that utilizes multi-model game consensus to construct high-quality hierarchical datasets, achieves progressive domain knowledge injection through data-driven curriculum learning and phased training, and combines preference alignment technology to build a sustainable data and model closed-loop system, endowing large models with self-evolving capabilities. Summary of the Invention

[0007] To overcome the problems in the prior art, this invention proposes a self-evolutionary training method for large-scale power drawing comprehension models.

[0008] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0009] This invention provides a self-evolutionary training method for a large-scale model of power drawing understanding, comprising the following steps:

[0010] Step 100: Based on the original power engineering drawings, construct a supervised training set, a preference training set, and an expert fine-tuning set through a game and consensus mechanism of multi-model reasoning, and explicitly label negative samples;

[0011] Step 200: Based on the supervised training set, the training task is deconstructed and the data is arranged. The multimodal large model of the base is trained by using efficient parameter fine-tuning and progressive injection to obtain the basic training model.

[0012] Step 300: Based on the basic training model and preference training set, direct preference optimization is performed using preference data containing explicitly labeled negative samples. At the same time, general data is mixed to anchor the generalization ability of the model, and finally an alignment model for power drawing review is formed.

[0013] Step 400: Deploy the alignment model specifically for power drawing review into practical applications, evaluate its effectiveness, and establish a data feedback mechanism to drive a new round of iterative cycles by continuously collecting low-confidence and rejected samples.

[0014] Further, step 100 includes:

[0015] Obtain original power engineering design drawings of different voltage levels, preprocess them to obtain power engineering drawings to be reviewed; set up multimodal tasks and configure review instructions for each task;

[0016] Based on the power engineering drawings to be reviewed and their corresponding review instructions, a multi-way reasoning game is performed and standardized to obtain a standardized multi-way reasoning result.

[0017] Based on standardized multi-way inference results, a consensus discrimination and diversion mechanism is constructed to obtain a supervised training set, a preference training set, and an expert fine-tuning set.

[0018] Furthermore, based on the power engineering drawings to be reviewed and their corresponding review instructions, a multi-way reasoning game is performed and standardized to obtain standardized multi-way reasoning results, including:

[0019] For each power engineering drawing to be reviewed and its corresponding review instructions, a basic model to be trained is selected as the current model, and at the same time, at least one group of models with stronger performance or different preferences are selected as the teacher model group.

[0020] Based on the current model and the teacher model group, parallel and independent reasoning is performed on the same power engineering drawing to be reviewed and its corresponding review instructions, generating structured review results respectively.

[0021] The structured review results output by the current model and the teacher model group are standardized to obtain standardized multi-way inference results.

[0022] Furthermore, based on standardized multi-way inference results, a consensus discrimination and triage mechanism is constructed to obtain a supervised training set, a preference training set, and an expert fine-tuning set, and negative samples are explicitly labeled, including:

[0023] A consensus-based discrimination and distribution mechanism is constructed. This mechanism is based on semantic consistency and rule verification, and divides power engineering design drawings into different data subsets and explicitly labels positive and negative samples.

[0024] If the results of multi-way inference are consistent, the power engineering design drawings are considered as highly reliable samples. Their consensus conclusions and logical chains are extracted, combined into structured supervised samples, and stored in the supervised training set.

[0025] If there are discrepancies in the results of multi-way inference, an automatic mechanism is used to determine the superiority or inferiority. Those that conform to the automatic mechanism are marked as positive samples, and those that do not are marked as negative samples. The two constitute the preference training set.

[0026] If the results of multi-way inference are inconsistent and cannot be judged by an automatic mechanism, the power engineering design drawings are marked as difficult samples and submitted to human experts with experience in power drawing review for review and correction. The correction results will be used as expert fine-tuning samples to form an expert fine-tuning set, which will be added to the supervision training set.

[0027] Furthermore, if there are discrepancies in the multi-way inference results, an automatic mechanism is used to determine their superiority or inferiority. Results conforming to the automatic mechanism are marked as positive samples, otherwise they are marked as negative samples. The two constitute the preference training set, which includes:

[0028] Automatic mechanisms include majority voting, rule-based business validation, or external constraints; samples that conform to the automatic mechanisms are labeled as positive samples, and those that do not are labeled as negative samples, and the two constitute the preference training set.

[0029] The rule-based business verification refers to checking whether the content output by the model conforms to the basic common sense and logical chain of power engineering, including contradictions between the conclusion and the reasoning process, and errors in numerical calculation results; the external constraints refer to whether the model outputs according to the preset JSON format and whether the grounding coordinates are within the range of image resolution.

[0030] Further, step 200 includes:

[0031] Based on the actual process of power drawing review, the training task is deconstructed and the data is arranged. The deconstructed task includes the recognition and description of basic visual elements of drawings, the extraction of structured information of drawings, and basic review logic reasoning.

[0032] The base multimodal large model is trained using a method of efficient parameter fine-tuning and progressive injection to obtain a basic training model. The efficient parameter fine-tuning includes targeted fine-tuning of the visual encoding, text decoding and multimodal alignment modules.

[0033] Furthermore, the progressive injection method includes: the entire training process is divided into a basic stage and an advanced stage; in the basic stage, drawing recognition and structured extraction samples are used to enable the model to establish an understanding of the visual composition and text format of power drawings; in the advanced stage, the proportion of samples containing review conclusions and logical chains is gradually increased to guide the model to learn to explain and judge according to power specifications.

[0034] Further, step 300 includes:

[0035] Based on the basic training model and preference training set, by comparing the output probabilities or likelihood differences of positive and negative samples, the model's ranking ability on business preferences is directly optimized, so that it tends to produce a response close to that of the positive sample when faced with the same input.

[0036] General data is introduced during the preference training process, and the general dataset and preference training set are mixed to anchor the generalization ability of the model, ultimately forming an alignment model specifically for power grid drawing review.

[0037] Further, step 400 includes:

[0038] The alignment model specifically designed for power drawing review will be upgraded in the next round of data synthesis. It will be added to the teacher model group or replace the original basic model to participate in the new multi-way reasoning game and evaluate its effectiveness.

[0039] Establish a data feedback mechanism to drive a new round of iterative cycles by continuously collecting low-confidence and rejected samples.

[0040] Compared with the prior art, the present invention has the following technical effects:

[0041] First, this invention, through a multi-expert game consensus mechanism, can automatically transform a large number of unlabeled drawings and review instructions into high-quality supervision samples and preference samples. Only a small number of difficult samples with serious disagreements among multiple models need to be reviewed by human experts, thereby greatly reducing the reliance on highly qualified experts and the high cost of labeling, and effectively solving the core problem of data scarcity in professional fields such as power drawing review.

[0042] Secondly, this invention significantly improves the safety and reliability of the model by systematically mining and explicitly utilizing negative samples. During the data synthesis phase, it identifies "incorrect" answers by comparing differences between models and constructs them as preference pairs. During the alignment training phase, high-risk negative samples such as "missed detections" and "misjudged compliance" are mapped as strong penalty signals. This mechanism enables the model not only to learn "what is the correct answer" but also to deeply understand "which answers are absolutely unacceptable from an engineering safety perspective," thereby significantly reducing the risk of missed detections and serious misjudgments in high-safety-requirement scenarios such as power grid drawing review.

[0043] Furthermore, the phased gradient injection and generalization anchoring strategy employed in this invention ensures that the model, while deeply mastering professional skills, does not lose its general generalization ability. Through task deconstruction and hierarchical training, the model's capability building path is clear and stable, avoiding knowledge confusion that may result from simple mixed training. Simultaneously, the introduction of general data mixing and low-amplitude learning rate strategies in the reinforcement alignment stage effectively suppresses catastrophic forgetting, enabling the final model to not only deeply understand the professional specifications of power drawings but also retain general graphic and textual understanding and reasoning abilities, demonstrating good robustness and transferability.

[0044] More importantly, this invention constructs a self-evolving closed-loop mechanism, enabling continuous improvement of model capabilities. By reintegrating the enhanced model into the data generation system and continuously collecting online low-confidence feedback and user error correction feedback, a virtuous cycle of "data generation-training-deployment-feedback-retraining" is formed. This makes the model no longer a static system that is trained once and fixed for a long time, but one that can continuously iterate and optimize with the accumulation of business data, thereby adapting to new specifications, new drawing formats, and new business needs in a timely manner, significantly improving the system's lifecycle value. In addition, the core idea of ​​this invention, namely, using multi-model consensus to drive data layering and negative sample mining, using phased training and preference alignment to achieve domain capability enhancement, and achieving self-evolution through a closed-loop mechanism, has strong versatility and can be extended to other multimodal domains that require strict security constraints and deep professional knowledge, such as industrial design review, medical image report verification, and legal document compliance review. Attached Figure Description

[0045] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the overall system structure of the present invention;

[0047] Figure 2 This is a schematic diagram of the overall process of the method of the present invention;

[0048] Figure 3 This is a flowchart illustrating the data synthesis and distribution process based on multi-model game consensus in this invention.

[0049] Figure 4 A schematic diagram of the hierarchical domain knowledge gradient injection (SFT) training structure;

[0050] Figure 5 This is a schematic diagram of DPO training and generalization anchoring based on preference alignment;

[0051] Figure 6 This is a schematic diagram of a self-evolving data flywheel and closed-loop iteration.

[0052] Figure 7 For the overall performance comparison of different training models;

[0053] Figure 8 This shows the evolution trend of recall rate for key audit labels. Detailed Implementation

[0054] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solutions proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0055] This invention proposes a self-evolving training method for a multimodal large-scale model for understanding power drawings, with the technical solution unfolding sequentially along a timeline. The entire process begins with structured data synthesis in step 100, followed by hierarchical domain knowledge gradient injection in step 200, business alignment and generalization anchoring based on negative sample mining in step 300, and finally constructing a self-evolving data flywheel in step 400. This method starts with a multimodal base multimodal large-scale model and combines it with a multi-expert model system composed of model instances with different architectures, training stages, or configurations. Through the organic combination of game theory and consensus mechanisms in multi-model inference and a phased training strategy, this invention achieves a complete technical closed loop from unlabeled raw data to a continuously evolving production-grade model.

[0056] In a specific embodiment, the hardware and software environment configuration for the training and experimentation process of the present invention is as follows: On the hardware side, a server configured with eight NVIDIA RTX 4090 GPUs (24GB of VRAM per GPU) is used. This server is equipped with a multi-core x86_64 architecture processor and at least 256GB of system memory. Training data and model checkpoints are stored using a local NVMe solid-state drive to ensure high-speed read and write operations. On the software side, the operating system is Linux (Ubuntu Server 20.04 or later), and the programming environment is Python 3.10. The deep learning framework stack consists of PyTorch 2.8, Transformers 4.57, DeepSpeed ​​0.17, and the ms-swift 3.9 framework for encapsulating supervised fine-tuning (SFT), human feedback-based reinforcement learning (RLHF), and direct preference optimization (DPO) training processes. To improve the efficiency of long-context training, flash_attn is enabled as the attention mechanism implementation during training.

[0057] This embodiment selects Qwen2.5-VL-32B-Instruct as the base multimodal large model. This model was chosen for its powerful image and text understanding and instruction following capabilities, and it can process image input and generate text output. To balance image clarity and memory usage, the maximum number of pixels in the input image is constrained to 1,003,520 during training. For efficient parameter fine-tuning, Low-Rank Adaptive (LoRA) and its enhanced version LoRA+ are used. Typical parameter configurations include: setting the LoRA rank (r) to 8 during the basic training phase using SFT, and increasing it to 16 during the alignment enhancement phase; LoRA Alpha (α) is uniformly set to 32. To improve adaptation efficiency, the lorap_lr_ratio parameter of LoRA+ is set to 16, i.e., an amplified learning rate is applied to the B matrix of LoRA. The target modules for fine-tuning are set to all-linear to uniformly inject LoRA weights into all linear layers in the model. During most training phases, the visual encoder (ViT) remains frozen, and the parameters of the language model and the multimodal fusion layer are adjusted first. For distributed training and optimization, the multi-process launcher built into torchrun or ms-swift is used to fully utilize eight GPUs for parallel training, with eight processes per node. To support sequence lengths from 8192 to 10240 within the 24GB VRAM limit, the key optimization strategy is to enable DeepSpeed ​​ZeRO Stage 3 in conjunction with CPU Offload. With the above environment configuration, the training and evaluation of all processes from steps 100 to 400 involved in this invention were completed.

[0058] In this embodiment, refer to Figures 1-8 This paper provides a self-evolutionary training method for understanding multimodal large models in power drawing analysis, including the following steps:

[0059] Step 100: Based on the original power engineering drawings, construct a supervised training set, a preference training set, and an expert fine-tuning set through a game and consensus mechanism of multi-model reasoning, and explicitly label negative samples;

[0060] Step 200: Based on the high-confidence supervision set and expert fine-tuning set, the training task is deconstructed and the data is arranged. The training is carried out by using efficient parameter fine-tuning and progressive injection to obtain the basic training model.

[0061] Step 300: Based on the basic training model and preference training set, direct preference optimization is performed using preference data containing explicit negative samples. At the same time, general data is mixed to anchor the generalization ability of the model, and finally an alignment model for power drawing review is formed.

[0062] Step 400: Deploy the alignment model specifically for power drawing review into practical applications, evaluate its effectiveness, and establish a data feedback mechanism to drive a new round of iterative cycles by continuously collecting low-confidence and rejected samples.

[0063] The following is a detailed explanation of each of the above steps:

[0064] Step 100: Based on the original power engineering drawings, construct a supervised training set, a preference training set, and an expert fine-tuning set through a game and consensus mechanism of multi-model reasoning, and explicitly label negative samples.

[0065] This step is the data generation engine of the entire method. Its core objective is to automatically construct a hierarchical structured training dataset from massive amounts of power engineering drawings in the absence of large-scale manual annotation, and to explicitly introduce "negative samples" in the process for use in the subsequent alignment optimization stage.

[0066] As an example, this step specifically includes:

[0067] Step 110: Obtain the original power engineering design drawings for different voltage levels, and preprocess them to obtain the power engineering drawings to be reviewed; set up multimodal tasks and configure review instructions for each task.

[0068] To ensure the model can grasp the key points of reviewing power engineering design drawings at different voltage levels, original power engineering design drawings of different voltage levels (such as 10kV and 0.4kV) from multiple projects were collected and uniformly converted into PNG image format. Each power engineering drawing to be reviewed serves as a sample. For review-related tasks, each sample also includes a specific review point description, such as "the rated current of all incoming line cabinets should not be less than 100A." The review points are mainly determined based on typical designs and depth requirements issued by the power grid company, with some data referencing national standards.

[0069] To achieve structured data synthesis, multimodal tasks are divided into four types, and each type is configured with dedicated review instructions. Review instructions are prompts specific to a particular task type. Setting dedicated review instructions requires first determining the task category, and then writing prompts based on the characteristics of that category and the drawing to guide the model in completing the task.

[0070] The first type is drawing comprehension tasks. The core objective of this task is to enable the model to provide a concise natural language description of the overall content, key visual elements, and engineering scenarios of power engineering design drawings, focusing on visual understanding rather than in-depth engineering judgment. For example, for a high-voltage system configuration diagram, the model needs to describe which main equipment is included in the diagram, the layout of the equipment, and the general function of the entire system. This type of task setting helps the model develop a preliminary understanding of the drawings.

[0071] When setting prompts for drawing comprehension tasks, the focus should be on guiding the model to provide a comprehensive and concise natural language description of the drawing's content. Here is an example prompt: "You are now given a power engineering design drawing. Please carefully observe the drawing and briefly describe its overall structure using natural language. Specifically, you should cover the main equipment included in the drawing, how these devices are arranged, and the general function of the entire engineering scenario. No in-depth engineering judgment is required during the description; simply present a clear initial understanding of the drawing." With such prompts, the model can clearly define the task objective, describe the key visual elements and engineering scenario of the drawing, and thus establish an initial understanding of the drawing.

[0072] The second category is drawing classification and entity recognition tasks. In this task, the model needs to undertake two important responsibilities. First, it needs to determine the type of drawing, such as accurately distinguishing between different types of drawings, such as high-voltage system configuration drawings, equipment installation drawings, and low-voltage power distribution system drawings. Different types of drawings have different characteristics and review focuses, and accurate classification helps the model to perform subsequent processing more effectively. Second, it needs to identify standardized entities according to preset rules, such as incoming line cabinets, compensation cabinets, and circuit breakers. The model needs to locate these entities in the power engineering design drawings and identify and label them according to requirements. To standardize the model's output and facilitate subsequent data processing and analysis, the output of this task is strictly limited to a JSON object containing the analysis process and metadata. This structured output format can clearly demonstrate the model's recognition approach and results, improving the usability and interpretability of the data.

[0073] For the drawing classification and entity recognition task, the example prompts are as follows: "Please carefully examine this power engineering design drawing. First, determine its type, such as a high-voltage system configuration drawing, equipment installation drawing, or low-voltage power distribution system drawing. Different types of drawings have different characteristics, and accurate classification is crucial. Second, according to preset rules, locate and identify standardized entities in the drawing, such as incoming line cabinets, compensation cabinets, and circuit breakers, and label them as required. Finally, output your analysis process and recognition results in a JSON object format containing the analysis process and metadata, ensuring a clear output structure for easy subsequent data processing and analysis." Such prompts allow the model to clearly understand the task requirements and complete the classification and entity recognition work according to the specifications.

[0074] As a professional drawing classification assistant, your task is to determine which of the predefined categories provided below this drawing belongs to.

[0075] Please find specific evidence to support this classification based on the drawing content (especially the drawing title in the lower right corner) and the classification rules provided below.

[0076] Classification rules and predefined types (used to find evidence):

[0077] 1. Drawing Catalog: Characteristics: Usually contains a list or index of multiple drawings, and may not have a drawing name in the lower right corner, which needs to be determined based on the content.

[0078] 2. Design Description: Features: Includes extensive textual descriptions such as project overview, design basis, technical requirements, and construction requirements. Note: Cover page, disclaimer, etc., are not part of the design description.

[0079] 3. Calculation Sheet: Characteristics: Includes electrical calculation process, formulas, results, etc., such as drawing titles or content that clearly involve load calculation, short-circuit current calculation, grounding resistance calculation, cable selection calculation, etc.

[0080] 4. List of Major Equipment and Materials: Characteristics: List / table format, including the name, model, specifications, quantity, unit, and remarks of the equipment / materials. The drawing title often includes "Major Equipment List," "Materials List," or "Equipment List."

[0081] 5. Access Scheme Diagram: Characteristics: Indicates how a user or project connects to the upstream power grid. The diagram title often includes terms such as "Access Scheme," "Access System," "Connection Method," and "Power Supply Scheme." Examples: 10kV access scheme system diagram, high-voltage system access method diagram.

[0082] 6. Main Wiring Diagram: Characteristics: Shows the primary electrical wiring relationships of the entire project or its main parts; the topology is the key point. The diagram title often includes "Main Wiring Diagram," "Primary Main Wiring Diagram," "Electrical Main Wiring Diagram," or "System Wiring Diagram" (referring to the overall system).

[0083] 7. Wiring / Power Connection Configuration and Primary System Diagram: Characteristics: Shows the detailed primary wiring of a specific device (such as a transformer substation or switchgear) or a partial system. The diagram title often includes "Primary System Diagram" (referring to a partial diagram), "Power Distribution System Diagram," "Wiring Configuration Diagram," "Connection Diagram," "Schematic Diagram," etc. Examples: Transformer substation primary system diagram, switchgear primary schematic diagram, power distribution system diagram. Note: Secondary circuit diagrams and terminal block diagrams are not included.

[0084] 8. Equipment Layout Plan: Characteristics: Shows the horizontal position and layout of equipment in a planar area (such as a substation, power distribution room, or floor). The drawing title often includes "plan layout," "equipment layout," or "layout plan."

[0085] 9. Prefabricated Substation Plan / Elevation Views: Characteristics: Specifically refers to the internal or external structural and layout views of a prefabricated substation. The drawing title often includes phrases such as "Prefabricated Substation Plan," "Prefabricated Substation Elevation," "Prefabricated Substation Layout," or "Prefabricated Substation Structure." Examples: Prefabricated substation equipment layout diagram, prefabricated substation plan and elevation views.

[0086] 10. Grounding Grid Diagram: Characteristics: Shows the arrangement and connection of grounding devices (grounding electrodes, grounding wires). The diagram title often includes "Grounding Plane", "Grounding Grid Arrangement", "Grounding Electrode Arrangement", etc.

[0087] 11. High and Low Voltage Switchgear Installation Drawings: Characteristics: These drawings show the installation dimensions, fixing methods, foundation requirements, and cross-sectional structure of high and low voltage switchgear or other electrical cabinets. The drawing title often includes terms such as "Installation Drawing," "Foundation Drawing," "Cross-section Drawing," "Sectional Drawing," or "Cabinet Dimension Drawing." Note the distinction from "Layout Drawings"; "Installation" focuses on how to install and fix the cabinets and the specific structural dimensions.

[0088] 12. Other: If the content or title of the drawing does not conform to any of the above categories, or if it is a cover, title page, standard drawing reference, or other auxiliary page, it shall be classified as such.

[0089] Explanation of process and requirements:

[0090] 1. Confirm Input Category: Understand the drawing category name provided by the user.

[0091] 2. Find evidence: Carefully examine the drawings, especially the title block in the lower right corner, to find the drawing title. If the title is unclear or missing, check the main features of the drawing (such as charts, text descriptions, and graphic types).

[0092] 3. Matching Evidence and Rules: Determine whether the found image name or content features match the description or example of the provided category in the rules.

[0093] 4. Generate Explanation: Output in JSON format, containing two fields:

[0094] `category`: Must be a category name provided by the user in `user_template`.

[0095] `reason`: Provide detailed evidence found on the drawing (e.g., "The drawing title in the lower right corner is 'XX,' which matches the characteristics of this category" or "The drawing content is a list of XX / contains XX calculations, which matches the characteristics of this category"). The reason needs to clearly link the drawing information to the classification rules.

[0096] Example output:

[0097] json

[0098] {

[0099] "category": "main wiring diagram",

[0100] "reason": "The drawing title in the lower right corner is 'Electrical Main Wiring Diagram,' which perfectly matches the definition and example of the 'Main Wiring Diagram' category (representing the overall primary electrical wiring relationship)."

[0101] }

[0102] ```

[0103] The third category is drawing review tasks. For a given review point, the model needs to output a detailed review process and structured conclusions. The review process should record in detail how the model analyzes and judges based on the drawing information and review standards. The structured conclusions are presented in a fixed JSON format. This JSON format includes detailed audit information for each review object, such as a record of checking whether the rated current of each incoming cabinet meets the requirements, and the final overall assessment, such as "Review passed" or "Review failed, the following problems exist...". The prompts specifically emphasize the distinction between situations such as "information missing, unable to assess," requiring the model to accurately identify and truthfully respond to incomplete information. Speculation and subjective judgment by the model are strictly prohibited to ensure the accuracy and reliability of the review results.

[0104] The example prompt for the drawing review task is: "Review this electrical engineering design drawing based on the given review points. During the review process, record in detail the steps you take to analyze and judge based on the drawing information and review standards, ensuring the review process is complete and clear. The review conclusion should be presented in a fixed JSON format, which must include detailed audit information for each reviewed object, such as checking and recording whether the rated current of each incoming cabinet meets the requirements, and finally giving an overall assessment, such as 'Review passed' or 'Review failed, the following problems exist...'. In particular, if you encounter a situation where information is missing and cannot be evaluated, you must accurately identify it and provide truthful feedback. Speculation and subjective judgment are strictly prohibited to ensure the accuracy and reliability of the review results." With such prompts, the model can complete the review task as required and output results that meet the specifications.

[0105] Role: You are an experienced electrical engineering drawing review expert, specializing in reviewing the compliance of power receiving engineering design drawings.

[0106] Task: Based on the provided drawings and audit point requirements, generate a JSON object containing a detailed logical reasoning process, a brief description of the problem, improvement suggestions, and a final conclusion, explaining whether the drawing meets the audit point requirements.

[0107] Please follow these steps to perform the reasoning and organize the results into the specified JSON format:

[0108] 1. Parse the input information:

[0109] Carefully analyze the provided power receiving engineering design drawings and extract key visual information (such as legends, symbols, connections, annotations, and values) related to the review points.

[0110] Read and understand the given audit points and their design specifications or safety requirements.

[0111] 2. Matching, Comparison, and Reasoning:

[0112] Locate the elements in the drawings that are relevant to the audit requirements.

[0113] Compare the information in the drawings with the requirements of the review points.

[0114] Checkpoint: Clearly lists the original text of the given checkpoint.

[0115] Logical chain (`logic_chain`):

[0116] Describe in detail the reasoning process, from identifying information on the drawings to drawing a conclusion.

[0117] Explicitly cite drawing evidence (e.g., "according to the legend...", "in the equipment list...", "line...connection...").

[0118] Use numbered lists or logical connectors to ensure the process is clear and coherent.

[0119] Cover all situations: Satisfied (indicate the content that meets the criteria), Not Satisfied (explain the discrepancies, reasons, and evidence), Irrelevant (explain why it cannot be evaluated).

[0120] Problem summary:

[0121] A concise summary of the compliance status between the drawings and the requirements of the review points.

[0122] Improvement suggestions:

[0123] Provide specific, actionable modification suggestions only if the final `label` is "Does not meet the review criteria".

[0124] When the final `label` is "Accepted Audit Point" or "Drawings are not relevant to the audit point", this field should be a brief conclusion (e.g., "The drawing meets the requirements of the current audit point.", "The content of the drawing is not relevant to this audit point and no modification is required.").

[0125] Label: Based on the analysis, the final audit conclusion is given ("Audit points are met", "Audit points are not met", "Drawings are not relevant to audit points").

[0126] 3. Generate JSON output:

[0127] Output the results strictly according to the JSON structure and field names specified below.

[0128] json

[0129] {

[0130] "checkpoint": "String, the original text of the checkpoint."

[0131] "logic_chain": [

[0132] "An array of strings, each item representing a detailed reasoning step."

[0133] ],

[0134] "problem_summary": "A string that provides a one-sentence summary of the reasons that are correct / incorrect / irrelevant."

[0135] "improvement_suggestion": "String, specific modification suggestions or brief conclusions."

[0136] "label": "String, final conclusion ('Meets audit criteria', 'Does not meet audit criteria', 'Drawings are irrelevant to audit criteria')."

[0137] }

[0138] ```

[0139] 4. Restrictions:

[0140] Analysis and conclusions must be strictly based on the content of the drawings and the given review points.

[0141] Do not speculate or add information that does not exist in the drawings.

[0142] Ensure that the reasoning in `logic_chain` is rigorous and verifiable. The fourth category is grounding or review result location tasks, which require the model to select highly relevant review areas in the drawing based on the review conclusion, and output a JSON array containing the bounding box coordinates. The training samples for all tasks are organized into JSON line format containing fields such as system prompts, image paths, task type, and model output.

[0143] This task prompt should explicitly instruct the model to select relevant review areas on the drawing based on the review conclusions, and to specify the output format. An example prompt would be: "Based on the previously given review conclusions, locate the highly relevant review areas in this power engineering design drawing. Use bounding boxes to select these areas and output the bounding box coordinates as a JSON array. Ensure accurate selection and a compliant output format to clearly demonstrate the correspondence between the review results and the drawing areas." Such prompts help the model clearly understand the task's focus, accurately locate the review results, and output the results in the prescribed format.

[0144] You are a multimodal location assistant proficient in electrical engineering drawings. Based on given review points and review results, locate all directly related key areas in the drawings.

[0145] Please select the review area based on the review points and review content.

[0146] The review points are as follows:

[0147] ```

[0148] xxx

[0149] ```

[0150] xxx

[0151] ```

[0152] To facilitate model training and management, all training samples for each task are organized into a JSON line format containing fields such as system prompts, image paths, task types, and model outputs. System prompts provide the model with specific requirements and guidance for the task; image paths specify the storage location of the sample images; the task type clarifies the task category to which the current sample belongs; and the model output is structured according to the requirements of the corresponding task. This unified sample organization format enables the model to read and process data more efficiently, improving training stability and effectiveness.

[0153] Step 120: Based on the power engineering drawings to be reviewed and their corresponding review instructions, perform multi-way reasoning game and standardize the process to obtain standardized multi-way reasoning results.

[0154] For each power engineering drawing to be reviewed and its corresponding review instructions, a base model to be trained is selected as the current model, and at the same time, at least one group of models with stronger performance or different preferences is selected as the teacher model group. Specifically, the teacher model group consists of at least two external models with stronger performance or better instruction compliance capabilities (such as larger-scale internal versions or commercial closed-source models).

[0155] Based on the current model and the teacher model group, parallel and independent reasoning is performed on the same power engineering drawing to be reviewed and its corresponding review instructions, generating structured review results. The structured review results include judgments on each review point, corresponding explanations or logical chains, and optional intermediate structured representations.

[0156] The structured audit results output by each model are standardized to obtain standardized multi-way inference results. The standardization process includes field alignment and label normalization, and the reasoning process in natural language form is extracted into a comparable logical chain representation, laying the foundation for subsequent consensus judgment.

[0157] Step 130: Based on the standardized multi-way inference results, a consensus discrimination and diversion mechanism is constructed to obtain a supervised training set, a preference training set, and an expert fine-tuning set.

[0158] After obtaining standardized multi-way inference results, a consensus discrimination and splitting mechanism is constructed. This mechanism, based on semantic consistency and rule verification, automatically divides samples into different data subsets and explicitly labels positive and negative samples.

[0159] If the results of multi-way inference are highly consistent within a preset threshold range, the sample is considered a high-confidence sample. Its consensus conclusion and the underlying logical chain are extracted and combined into a structured supervised sample. This high-confidence sample is stored in a high-confidence supervised training set used for basic capability learning. The logical chain refers to the intermediate inference process in which the model gives its final judgment in each type of task, especially in the third type of drawing review task, as this type of task requires the most complex inference.

[0160] If discrepancies arise in the results of multi-way inference, the "winning answer" will be determined using majority voting, rule-based business validation, or external constraints. Answers deemed more in line with engineering specifications are marked as positive samples (Chosen), while obviously erroneous or potentially unsafe answers are marked as negative samples (Rejected). Both constitute the preference training set used for alignment optimization. Rule-based business validation checks whether the model's output conforms to basic common sense and logical chains in power engineering, including contradictions between conclusions and reasoning processes, and errors in numerical calculations. External constraints ensure the model outputs according to a preset JSON format and that the grounding coordinates are within the image resolution range.

[0161] Of particular note is that for special samples where "the current model is wrong while the teacher model is correct", the system will set them as key sampling objects, and through targeted learning and correction, it will specifically address the error patterns in the current model, making the model's output more accurate and reliable.

[0162] If the results of multi-way inference are inconsistent and cannot be judged by an automatic mechanism, the sample is marked as a difficult sample and submitted to a human expert with experience in power grid drawing review for review and correction. The decision will be used as a high-value expert fine-tuning sample and added to the supervisory fine-tuning data to obtain the expert fine-tuning set.

[0163] Through this diversion mechanism, this step ultimately generates three core data subsets: a high-confidence supervised training set, a preference training set containing explicit negative samples, and an expert-tuned set.

[0164] The methods for measuring consensus vary across different task scenarios. For drawing classification tasks, if the drawing type labels in the multi-way inference results are completely consistent and the JSON structure is valid, it is considered a label consensus. Furthermore, if the set similarity of the entity recognition results is greater than a preset threshold (e.g., Jaccard similarity greater than or equal to 0.8), the sample is considered a high-confidence sample. In drawing review tasks, if the final review labels of the three outputs are completely consistent, it is considered a label consensus. If they are inconsistent but there is a majority opinion, such as two models judging "not satisfied" and one judging "satisfied," the majority opinion label is considered a candidate correct answer. For drawing understanding and grounding tasks, consensus is measured by the semantic similarity of the text content and the intersection-over-union (IoU) ratio of the bounding boxes, respectively.

[0165] Based on the consensus determination results, the samples are distributed to three different datasets. When the multi-way inference results reach a high-confidence consensus, the sample is constructed as a Supervised Fine-tuning (SFT) sample, with its target output being the most structurally complete answer from the teacher model, and stored in the supervised training set Dataset_SFT. When the multi-way inference results are divergent but have a clear majority opinion, the sample is constructed as a Direct Preference Optimization (DPO) preference pair, with the answer that conforms to the majority opinion and is logically superior being designated as a "Chosen" response, and the answer of the minority opinion being designated as a "Rejected" response, and stored in the preference training set Dataset_DPO. If the multi-way inference results are completely inconsistent or have serious structural errors, and cannot be automatically judged by rules, the sample is assigned to the difficult sample set Dataset_Expert, which is then manually reviewed and labeled by power engineers. The high-quality conclusions produced by these engineers will be added to the supervised training set Dataset_SFT as SFT samples. This process ensures that step 100 can stably output high-quality positive and negative samples for subsequent training stages.

[0166] Step 200: Based on the supervised training set, the training task is deconstructed and the data is arranged. The multimodal large model of the base is trained by using efficient parameter fine-tuning and progressive injection to obtain the basic training model.

[0167] This step, SFT, serves as the foundational training phase of the entire method. Its goal is to enable the model to systematically master the basic visual elements, structured representation methods, and basic review logic of power drawings. Specifically, it includes two levels: "basic visual semantic understanding" and "injection of complex logical reasoning".

[0168] As an example, step 200 specifically includes:

[0169] Step 210: Based on the actual process of power grid drawing review, deconstruct the training task and arrange the data.

[0170] Based on the actual process of power drawing review, the capabilities that the model needs to learn are broken down into several levels of tasks, including recognition and description of basic visual elements of drawings, extraction of structured information from drawings, and basic review logic reasoning.

[0171] Specifically, for the task of identifying and describing basic visual elements in drawings, power engineering drawings include many different types of visual elements, such as various electrical symbols, lines, equipment graphics, etc. The model needs to identify these visual elements and be able to clearly describe them in appropriate language. For example, it can identify that a certain symbol represents a circuit breaker and describe its shape, location and other characteristics.

[0172] Next is the task of extracting structured information from the drawings. After completing the basic visual element recognition, the model needs to further explore the connections and logical relationships between these elements and present the information in the drawings in a structured way. For example, it can extract the connection relationships between various devices in the power system, the direction of the lines, and the division of different areas to form a complete and organized knowledge framework, which will provide strong support for the subsequent review work.

[0173] Finally, there's the basic logical reasoning task for review. Based on the structured information extracted earlier, and combined with relevant power industry regulations, standards, and review rules, the model needs to conduct a preliminary review and judgment of the drawings. For example, it needs to determine whether the line connection methods comply with safety regulations and whether the equipment selection is reasonable. Through logical reasoning, it identifies potential problems and hidden dangers in the drawings. This progressively challenging task breakdown constructs a clear and orderly learning path for the model, enabling it to gradually master the key capabilities required for power drawing review.

[0174] In terms of data arrangement, the supervised training set and expert fine-tuning set generated in step 100 are mainly used, and they are mixed and rearranged according to task type to achieve course-style training from easy to difficult. For example, the focus is first on basic recognition and structured extraction, and then the proportion of audit judgment samples is gradually increased.

[0175] Specifically, in the initial training phase, the focus is on inputting samples for basic recognition and structured extraction, allowing the model to concentrate on learning methods for recognizing basic visual elements of drawings and techniques for extracting structured information. For example, a large number of samples containing simple electrical symbol recognition and line routing extraction are added to the dataset, enabling the model to quickly become familiar with the basic structure and common elements of electrical drawings.

[0176] As training progresses, the proportion of review and judgment samples is gradually increased. At this point, the model has acquired certain basic capabilities, enabling it to better understand and process review-related information. By introducing more samples involving the application of review rules and logical reasoning, such as drawing samples containing issues like unreasonable equipment selection and incorrect wiring connections, the model continuously improves the accuracy and rigor of its review judgments in practice. This gradual data arrangement method aligns with the cognitive patterns of human learning, enabling the model to improve its performance more efficiently and stably, gradually meeting the practical application requirements of power drawing review.

[0177] Step 220: Train the base multimodal large model using efficient parameter fine-tuning and progressive injection to obtain the basic training model.

[0178] In terms of training strategy, this step employs efficient parameter fine-tuning and progressive injection. While keeping most parameters of the base multimodal model frozen, trainable adaptation components are introduced only into some key modules. Targeted fine-tuning of the visual encoding, text decoding, and multimodal alignment modules is achieved by setting differentiated learning rates or training weights for different modules. Specifically, the target modules for fine-tuning are set to all-linear to uniformly inject LoRA weights into all linear layers in the model. During most training phases, the visual encoder (ViT) remains frozen, prioritizing the adjustment of parameters for the language model and the multimodal fusion layer.

[0179] The entire training process is divided into different stages: the basic stage focuses on using drawing recognition and structured sample extraction to enable the model to establish a stable understanding of the visual composition and text format of power drawings; the advanced stage gradually increases the proportion of samples containing review conclusions and logical chains, guiding the model to learn how to explain and judge according to power regulations. At the end of each stage, the system selects the optimal model parameters as the starting point for the next stage based on the performance on the validation set, thus forming a gradient injection trajectory of domain knowledge from shallow to deep. After completing this step, an intermediate version model will be obtained, which is significantly superior to the base multimodal large model in terms of basic understanding and preliminary review capabilities of power drawings.

[0180] Specifically, in organizing the training data, a data mixing and curriculum design strategy was adopted to construct a merged supervised fine-tuning SFT training set, Dataset_SFT_Merged. In this dataset, samples from drawing comprehension, classification and entity recognition, auditing, and grounding tasks were roughly mixed in a ratio of 20%, 30%, 40%, and 10%. To enhance the model's learning of difficult examples, samples from the expert-tuned set were given higher weights during sampling (e.g., sampling probability increased by 2 to 3 times). The training process was automated through a script, with key hyperparameter settings as follows: the learning rate was set to 1e-4, suitable for LoRA fine-tuning; the LoRA rank was set to 8; and alpha was set to 32. The equivalent global batch size was set to 16 through a gradient accumulation step, supporting a maximum sequence length of up to 10240 to handle complex auditing logic. To ensure training stability and resource utilization, optimization techniques such as learning rate preheating, DeepSpeed ​​Zero3 Offload, bfloat16 mixed precision, and flash_attn were employed. The SFT phase can be conducted in rounds. The initial phase focuses on basic understanding and classification tasks, while subsequent rounds gradually introduce more complex review and positioning tasks.

[0181] Step 300: Based on the basic training model and preference training set, direct preference optimization is performed using preference data containing explicit negative samples. At the same time, general data is mixed to anchor the generalization ability of the model, and finally an alignment model for power drawing review is formed.

[0182] This step is the Enhanced Alignment Phase DPO. Based on the basic trained model, this step introduces a hybrid training method that combines explicit negative samples with general data. It focuses on solving the "illusion" and misjudgment problems of the model in the power industry scenario, and aims to reduce the false negative rate of high-risk audit points. It also ensures that while strengthening domain behavior norms, it maximizes the model's generalization ability for general graphic and text tasks.

[0183] The first step is business value alignment. This process uses the base training model trained in step 200 as a starting point and the preference training set generated in step 100 as the main training data. By comparing the output probabilities or likelihood differences of positive and negative samples, the model's ability to rank business preferences is directly optimized, making it more likely to produce answers closer to positive samples (Chosen) when faced with the same input. This process pays special attention to scenarios where negative samples represent missed risks or misjudged compliance. By explicitly "penalizing" these negative samples, the model deeply understands which answers are unacceptable in engineering safety and tends to give stricter or more conservative recommendations when uncertain.

[0184] While aligning with business requirements, this step also performs generalization robustness anchoring. Specifically, a certain proportion of general-purpose text and image data or cross-domain multimodal data is introduced during preference training. The "regularization" effect of this general-purpose data is used to suppress the overfitting tendency of the model in a single vertical domain and maintain its sensitivity to basic capabilities such as general visual understanding and reading comprehension. To protect the learned domain knowledge, this stage uses a significantly lower learning rate than the previous stage and strictly controls the number of training steps. Simultaneously, parameter updates for key modules and general modules in the power sector are differentiated. Through this anchoring, the final model maintains reasonable performance on general tasks while ensuring significant improvement in key power audit indicators, making it more stable and reliable in complex application scenarios.

[0185] Specifically, step 300 involves DPO alignment and generalized anchoring. In this stage, based on the model fine-tuned in step 200, direct preference optimization is performed using preference data containing explicit negative samples. Simultaneously, a small amount of general data is mixed in to anchor the model's generalization ability, ultimately forming an alignment model specifically for power drawing review. Each sample in the DPO dataset during the alignment stage contains one input and one pair of outputs: a "Chosen" answer deemed superior and an incorrect "Rejected" answer. To prevent model degradation during domain-specific alignment, a general anchoring dataset, Dataset_Anchor, is introduced, containing general samples such as text-based question answering and document understanding.

[0186] During training, power industry-specific preference samples were mixed with general anchored samples in a ratio of approximately 7:3. The configuration for DPO training differed from the SFT stage, with the learning rate significantly reduced to 2e-5 to allow for finer-grained decision boundary adjustments. The rank of LoRA was increased to 16 to enhance the model's ability to express fine-grained preferences. Loss function... It adopts a combined approach, including a weighted average of preference loss and anchoring loss.

[0187] ;

[0188] in, Direct Preference Optimization (DPO) represents preferences, and the loss is used to teach the model "which answer is better". The anchoring loss is essentially an SFT (Supervised Fine-Tuning) loss, used to preserve the model's language generation capabilities and general knowledge.

[0189] The sigmoid or bco_pair loss for preference learning and the SFT loss for general capability anchoring are calculated and balanced using weights. In terms of the training mechanism, the model weights produced in step 200 serve as both the initial state of the policy model and the benchmark of the reference model, effectively constraining the model from deviating excessively from the learned knowledge distribution during optimization.

[0190] Step 400: Deploy the alignment model specifically for power drawing review into practical applications, evaluate its effectiveness, and establish a data feedback mechanism to drive a new round of iterative cycles by continuously collecting low-confidence and rejected samples.

[0191] This closed-loop iteration phase aims to deploy the maturely trained model into actual business processes and form a closed loop with the data synthesis mechanism in step 100, thereby enabling the model to continuously evolve itself in real-world usage scenarios.

[0192] First, the alignment model specifically for power drawing review obtained in step 300, i.e., the enhanced model, is deployed as the online version to the actual power drawing review business system, and begins to receive real drawings, review instructions, and user feedback. Simultaneously, this enhanced model will undergo a role upgrade in the next round of data synthesis, being added to the "teacher model group" of step 100 or replacing the original basic model, allowing it to directly participate in the new multi-way reasoning game. As iterations proceed, the alignment model specifically for power drawing review will gradually evolve from a "student" to a "co-decision-maker" and eventually a new leading teacher within the teacher group.

[0193] To drive this evolutionary process, an online feedback-driven data feedback mechanism was established. During actual model inference, the system automatically marks low-confidence cases based on its internal uncertainty estimation or multi-round self-consistency checks, and feeds them back to step 100 as the data source for a new round of game. Simultaneously, user rejections, manual corrections, or reviews in the business system are also recorded. These rejected model outputs and corrected results are prioritized as high-value preference data or expert-tuned data for constructing new preference pairs or expert-supervised samples. This feedback data is then re-input into the multi-expert game consensus mechanism of step 100 to generate a new training dataset, and the training and alignment processes of steps 200 and 300 are executed again, forming the next version of the model. Through this continuous iteration and version evolution, the model continuously learns new drawing styles, specification changes, and rare boundary cases from real-world business, and gradually corrects high-risk error patterns in previous versions through the preference alignment mechanism, ultimately achieving a spiral increase in capabilities.

[0194] The specific implementation of step 400 is iteration and effect evaluation. It deploys the DPO model trained in step 300 to a practical application, evaluates its effect, and establishes a data feedback mechanism to drive a new round of iterations from step 100 to step 300 by continuously collecting low-confidence and rejected samples.

[0195] During deployment, the alignment model specifically designed for power drawing review is encapsulated as an online service, and the input, output, and metadata of each call are recorded. By monitoring metrics such as JSON parsing success rate and output stability, low-confidence samples are automatically identified and fed back. Simultaneously, the system allows review engineers to reject model conclusions and provide corrective suggestions; these manually corrected cases are directly constructed into high-value DPO preference pairs, enriching the preference dataset for the next round of training.

[0196] In a specific experimental evaluation, an independent evaluation set containing 309 manually reviewed samples was used to compare the performance of models at different stages. The base model, not trained using the process described in this invention, had an overall accuracy of only 50.5%, with some unparseable outputs and an extremely low recall rate of only 0.18 for the crucial "not meeting review points" label, indicating a high risk of missed reports. After SFT training in step 200, the model's output parsing success rate significantly improved, with the overall accuracy increasing to 76.0%, and the recall rate for "not meeting review points" jumping to 0.73, demonstrating the effectiveness of SFT in the structured output and basic rules of the teacher model. Finally, the SFT+DPO model trained through the complete process of steps 200 and 300 achieved an overall accuracy of 84.4%. Particularly noteworthy was the stable recall rate of 0.87 for the key indicator of "not meeting review points," representing a qualitative leap compared to the base model and significantly reducing the risk of missed reports. Meanwhile, the model also demonstrated high accuracy and recall on other labels such as "meets audit points" and "drawings are not relevant to audit points". Group statistics show that after DPO optimization, the model's accuracy on a large number of audit points was substantially improved, with some audit points even reaching 100% accuracy.

[0197] Experimental results fully demonstrate that the SFT stage in step 200 effectively improves the model's mastery of domain knowledge, while the DPO stage in step 300 successfully corrects the model's erroneous preferences by aligning the preferences of explicit negative samples, significantly reducing the probability of high-risk errors. Through the complete implementation path of "data generation, basic training, reinforcement alignment, and closed-loop iteration" described in this invention, a high-performance power drawing understanding and review model that meets the requirements of a production environment can be systematically constructed and continuously evolved.

[0198] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A self-evolutionary training method for understanding large-scale power drawing models, characterized in that, Includes the following steps: Step 100: Based on the original power engineering drawings, construct a supervised training set, a preference training set, and an expert fine-tuning set through a game and consensus mechanism of multi-model reasoning, and explicitly label negative samples; Step 200: Based on the supervised training set, the training task is deconstructed and the data is arranged. The multimodal large model of the base is trained by using efficient parameter fine-tuning and progressive injection to obtain the basic training model. Step 300: Based on the basic training model and preference training set, direct preference optimization is performed using preference data containing explicitly labeled negative samples. At the same time, general data is mixed to anchor the generalization ability of the model, and finally an alignment model for power drawing review is formed. Step 400: Deploy the alignment model specifically for power drawing review into practical applications, evaluate its effectiveness, and establish a data feedback mechanism to drive a new round of iterative cycles by continuously collecting low-confidence and rejected samples. Step 100 includes: Obtain original power engineering design drawings of different voltage levels, preprocess them to obtain power engineering drawings to be reviewed; set up multimodal tasks and configure review instructions for each task; Based on the power engineering drawings to be reviewed and their corresponding review instructions, a multi-way reasoning game is performed and standardized to obtain standardized multi-way reasoning results. This includes: for each power engineering drawing to be reviewed and its corresponding review instructions, a base model to be trained is selected as the current model, and at the same time, at least one group of models with stronger performance or different preferences are selected as the teacher model group; based on the current model and the teacher model group, parallel independent reasoning is performed on the same power engineering drawing to be reviewed and its corresponding review instructions, generating structured review results respectively; the structured review results output by the current model and the teacher model group are standardized to obtain standardized multi-way reasoning results. Based on standardized multi-way inference results, a consensus discrimination and splitting mechanism is constructed to obtain a supervised training set, a preference training set, and an expert fine-tuning set. This includes: constructing a consensus discrimination and splitting mechanism, which, based on semantic consistency and rule checks, divides power engineering design drawings into different data subsets and explicitly labels positive and negative samples; if the multi-way inference results are consistent, the power engineering design drawing is considered a high-confidence sample, and its consensus conclusion and logical chain are extracted and combined into a structured supervised sample, which is stored in the supervised training set; if the multi-way inference results are inconsistent, an automatic mechanism is used to determine their superiority or inferiority, and samples conforming to the automatic mechanism are labeled as positive samples, otherwise they are labeled as negative samples, forming the preference training set; if the multi-way inference results are inconsistent and cannot be determined by the automatic mechanism, the power engineering design drawing is labeled as a difficult sample and submitted to a human expert with experience in power drawing review for review and correction. The correction result will serve as an expert fine-tuning sample, forming the expert fine-tuning set, which is then added to the supervised training set. If there are discrepancies in the multi-way inference results, an automatic mechanism is used to determine the superiority or inferiority. Samples conforming to the automatic mechanism are marked as positive samples, and those not conforming are marked as negative samples. These two sets constitute the preference training set. The automatic mechanism includes majority voting, rule-based business verification, or external constraints. The rule-based business verification checks whether the model's output conforms to basic common sense and logical chains in power engineering, including contradictions between conclusions and reasoning processes, and errors in numerical calculation results. The external constraints refer to whether the model outputs according to a preset JSON format and whether the grounding coordinates are within the image resolution range. Step 200 includes: Based on the actual process of power drawing review, the training task is deconstructed and the data is arranged. The deconstructed task includes the recognition and description of basic visual elements of drawings, the extraction of structured information of drawings, and basic review logic reasoning. The multimodal large model of the base is trained using a combination of efficient parameter fine-tuning and progressive injection to obtain a basic training model. The efficient parameter fine-tuning includes targeted fine-tuning of the visual encoding, text decoding, and multimodal alignment modules. The progressive injection method involves dividing the entire training process into a basic stage and an advanced stage. In the basic stage, drawing recognition and structured extraction samples are used to enable the model to recognize the visual composition and text format of power drawings. In the advanced stage, the proportion of samples containing review conclusions and logical chains is gradually increased to guide the model to learn to explain and judge according to power specifications.

2. The self-evolutionary training method for a large-scale power drawing understanding model according to claim 1, characterized in that, Step 300 includes: Based on the basic training model and preference training set, by comparing the output probabilities or likelihood differences of positive and negative samples, the model's ranking ability on business preferences is directly optimized, so that it tends to produce a response close to that of the positive sample when faced with the same input. General data is introduced during the preference training process, and the general dataset and preference training set are mixed to anchor the generalization ability of the model, ultimately forming an alignment model specifically for power grid drawing review.

3. The self-evolutionary training method for a large-scale power drawing understanding model according to claim 1, characterized in that, Step 400 includes: The alignment model specifically designed for power drawing review will be upgraded in the next round of data synthesis. It will be added to the teacher model group or replace the original basic model to participate in the new multi-way reasoning game and evaluate its effectiveness. Establish a data feedback mechanism to drive a new round of iterative cycles by continuously collecting low-confidence and rejected samples.

Images