A cross-task transfer method and related equipment for collaborative optimization of a large-scale patrol and inspection model using LoRA and Bottleneck Adapter.

Abstract

This invention discloses a cross-task transfer method and related equipment for collaborative optimization of a large-scale inspection and supervision model using LoRA and Bottleneck Adapter. It solves the technical problem that current large-scale models for the inspection and supervision field struggle to achieve rapid transfer and flexible adaptation between multiple inspection and supervision tasks. The method includes: acquiring raw inspection and supervision corpora and constructing a preprocessed corpus dataset for the inspection and supervision domain; introducing LoRA and Bottleneck Adapter technologies into a general large-scale language model to construct a LoRA-Bottleneck Adapter fusion base model; constructing corresponding LoRA parameter packages with optimal gated weighting ratios based on the training and validation sets of each task classification in the preprocessed corpus dataset; modeling a structured knowledge graph for the basic knowledge base of inspection and supervision; receiving raw business data from multiple sources, calling the corresponding target LoRA parameter package, and processing the data through the fusion base model in combination with the parameter package and knowledge graph to generate standardized business processing results for inspection and supervision.

Description

Technical Field

[0001] This invention relates to the field of inspection and patrol technology, and in particular to a cross-task migration method and related equipment for the collaborative optimization of a large-scale inspection and patrol model using LoRA and Bottleneck Adapter. Background Technology

[0002] In power grid operations, inspections and audits are the core tools for strengthening internal supervision, preventing compliance risks, and regulating the exercise of power. Their business scenarios cover key tasks throughout the entire process, including legal analysis, case retrieval, judgment of violations and disciplinary infractions, generation of problem lists, and evaluation of rectification and implementation. They are characterized by diverse scenarios, complex processes, high professional requirements, and strict standard constraints, and are an important support for ensuring the safe and stable operation of power grid companies and improving the effectiveness of modern governance.

[0003] However, in practical application, power grid inspection and supervision work still faces a series of real challenges. First, the entire inspection and supervision task chain is complex and logically interconnected. It requires not only accurate interpretation of laws and policies, but also identification of violations and irregularities, full tracking of rectification progress, and comprehensive evaluation of rectification effectiveness. Each task relies on multi-dimensional information for in-depth causal reasoning, which traditional rule-based processing methods and single-task-specific models cannot meet the needs of such complex reasoning and full-process support. Second, the business logic and data characteristics of different inspection and supervision scenarios, such as political supervision, financial auditing, and power grid inspection, differ significantly. Existing intelligent models lack lightweight and efficient cross-scenario migration and adaptation mechanisms. Frequent repetitive training not only significantly increases computational resource consumption but also severely slows down business response and model deployment efficiency.

[0004] Currently, large-scale models for the inspection and supervision field generally adopt a technical approach of independent training for each task and fine-tuning of all parameters. This involves building a dedicated model or conducting large-scale parameter updates for each specific task, such as legal analysis, case retrieval, and qualitative and quantitative disciplinary analysis. This approach lacks a mechanism for parameter reuse and lightweight adaptation between tasks. The training logic and feature representations of different tasks are independent and cannot be shared. When the model switches to a new inspection task, it cannot quickly adapt based on existing training results and must be retrained or have its parameters fine-tuned on a large scale. Ultimately, this makes it difficult to achieve rapid transfer and flexible adaptation between multiple inspection and supervision tasks. Summary of the Invention

[0005] This invention provides a method and related equipment for cross-task migration of a large-scale inspection and patrol model using LoRA and Bottleneck Adapter in collaboration, which solves the technical problem that current large-scale models for the inspection and patrol field are difficult to achieve rapid migration and flexible adaptation between multiple inspection and patrol tasks.

[0006] The first aspect of this invention provides a cross-task transfer method for a large-scale patrol and inspection model that utilizes LoRA and Bottleneck Adapter in a collaborative optimization process, comprising: Obtain the original corpus of inspection and supervision, and construct a preprocessed corpus dataset for the field of inspection and supervision based on the original corpus of inspection and supervision. LoRA and Bottleneck Adapter technologies are introduced into the general large language model to construct a LoRA-Bottleneck Adapter fusion base model; Based on the training and validation sets of multiple task categories in the preprocessed corpus dataset of the inspection and patrol domain, the parameter package of the LoRA-Bottleneck Adapter fusion base model is constructed to obtain the LoRA parameter package with optimal gated weighted weight ratio for each task category. The pre-set basic knowledge base of inspection and supervision is processed by structured modeling to output a structured knowledge graph of inspection and supervision. When the original data of the multi-source business of inspection and patrol is received, the corresponding LoRA parameter package with the optimal gated weighted weight ratio is called based on the task classification of the original data of the multi-source business of inspection and patrol, and is used as the target LoRA parameter package. The LoRA-Bottleneck Adapter fusion base model is used to process the original data of the multi-source inspection and supervision business based on the target LoRA parameter package and the structured inspection and supervision knowledge graph, and to generate standardized inspection and supervision business processing results.

[0007] Optionally, the step of constructing a preprocessed corpus dataset for the field of inspection and supervision based on the original inspection and supervision corpus includes: The original corpus of patrol and inspection is sequentially deduplicated, denoised, and format-standardized to output the cleaned corpus of patrol and inspection. The cleaned patrol and inspection corpus is divided into segments, and the segmented patrol and inspection corpus set is output. The segmented corpus of inspection and supervision is labeled, and a preprocessed corpus dataset in the field of inspection and supervision is output.

[0008] Optionally, the introduction of LoRA and Bottleneck Adapter technologies into the general large language model to construct a LoRA-Bottleneck Adapter fusion base model includes: The bottleneck layer of the Transformer layer of the general large language model is embedded with the Bottleneck Adapter to obtain the basic model with the embedded Bottleneck Adapter. Configure pluggable LoRA submodules for the attention layer and feedforward layer of the embedded Bottleneck Adapter base model, and output a LoRA-Bottleneck Adapter dual-module base model; A gating weighting mechanism is introduced into the LoRA-Bottleneck Adapter dual-module basic model to output a LoRA-Bottleneck Adapter dual-module basic model with gating weighting mechanism; The backbone parameters of the LoRA-Bottleneck Adapter dual-module base model with gating weighting mechanism are frozen, and the parameters of the Bottleneck Adapter model and the LoRA model are trained together to obtain the LoRA-Bottleneck Adapter fusion base model.

[0009] Optionally, the step of constructing a parameter package for the LoRA-Bottleneck Adapter fusion base model based on the training and validation sets of multiple task classifications in the preprocessed corpus dataset of the inspection and patrol domain, to obtain a LoRA parameter package with optimal gated weighted weight ratios for each task classification, includes: Freeze the backbone parameters and BottleneckAdapter model parameters of the LoRA-Bottleneck Adapter fusion base model, and use the training sets of each task classification to jointly train the LoRA model parameters and gated weighting ratio of the LoRA-Bottleneck Adapter fusion base model to obtain the trained LoRA-Adapter fusion patrol and inspection large model corresponding to each task classification. The performance of the corresponding trained LoRA-Adapter fusion inspection and patrol model is verified using the validation set of each task category, thus obtaining the verified LoRA-Adapter fusion inspection and patrol model for each task category. LoRA parameters and optimal gating weight ratios are extracted from the verified LoRA-Adapter fusion inspection and patrol models corresponding to each task category, and the LoRA parameter package with the optimal gating weight ratio is output for each task category.

[0010] Optionally, the step of using the LoRA-Bottleneck Adapter fusion base model to process the multi-source original data of the inspection and supervision based on the target LoRA parameter package and the structured inspection and supervision knowledge graph to generate standardized inspection and supervision business processing results includes: The original data from the multi-source inspection and patrol business are sequentially subjected to optical character recognition, data cleaning, and format conversion to output structured inspection and patrol business data. The target LoRA parameter package is used to update the model parameters of the LoRA-Bottleneck Adapter fusion base model to obtain the patrol and inspection cross-task online inference model; The cross-task online reasoning model for inspection and supervision is used to perform cross-task intelligent reasoning processing on the structured inspection and supervision business data based on the structured inspection and supervision knowledge graph, and outputs the intelligent reasoning results of inspection and supervision tasks. Based on the intelligent reasoning results of the inspection and supervision tasks and the preset inspection and supervision business template, template filling and text splicing are performed to output the initial standardized inspection and supervision business document. The initial standardized inspection and supervision business documents are verified and corrected, and the standardized inspection and supervision business processing results are output.

[0011] The second aspect of this invention provides a cross-task migration system for a large-scale patrol and inspection model that utilizes LoRA and Bottleneck Adapter in a collaborative manner, comprising: The acquisition module is used to acquire the original corpus of inspection and supervision, and to construct a preprocessed corpus dataset in the field of inspection and supervision based on the original corpus of inspection and supervision. The model building module is used to introduce LoRA and Bottleneck Adapter technologies into a general large language model and build a LoRA-Bottleneck Adapter fusion base model. The parameter package construction module is used to construct the parameter package for the LoRA-Bottleneck Adapter fusion base model based on the training set and validation set of multiple task classifications in the preprocessed corpus dataset of the inspection and patrol domain, so as to obtain the LoRA parameter package with the optimal gated weighted weight ratio for each task classification. The graph modeling module is used to perform structured modeling on the preset basic knowledge base of inspection and supervision, and output a structured knowledge graph of inspection and supervision. The selection module is used to, when receiving the original data of the multi-source business of the inspection and patrol, call the corresponding LoRA parameter package with the optimal gated weighted weight ratio based on the task classification of the original data of the multi-source business of the inspection and patrol, and use it as the target LoRA parameter package; The generation module is used to perform business processing on the original multi-source business data of the inspection and supervision based on the target LoRA parameter package and the structured inspection and supervision knowledge graph using the LoRA-Bottleneck Adapter fusion base model, and generate standardized business processing results for inspection and supervision.

[0012] Optionally, the model building module is specifically used for: The bottleneck layer of the Transformer layer of the general large language model is embedded with the Bottleneck Adapter to obtain the basic model with the embedded Bottleneck Adapter. Configure pluggable LoRA submodules for the attention layer and feedforward layer of the embedded Bottleneck Adapter base model, and output a LoRA-Bottleneck Adapter dual-module base model; A gating weighting mechanism is introduced into the LoRA-Bottleneck Adapter dual-module basic model to output a LoRA-Bottleneck Adapter dual-module basic model with gating weighting mechanism; The backbone parameters of the LoRA-Bottleneck Adapter dual-module base model with gating weighting mechanism are frozen, and the parameters of the Bottleneck Adapter model and the LoRA model are trained together to obtain the LoRA-Bottleneck Adapter fusion base model.

[0013] A third aspect of the present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the cross-task migration method for the LoRA and Bottleneck Adapter collaborative optimization of the patrol and inspection large model as described above.

[0014] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the cross-task migration method for the collaborative optimization of the LoRA and Bottleneck Adapter patrol and inspection large model as described above.

[0015] The fifth aspect of the present invention provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein, when the program instructions are executed by a computer, the computer performs the steps of the cross-task migration method for the LoRA and Bottleneck Adapter collaborative optimization of the patrol and inspection large model as described above.

[0016] As can be seen from the above technical solutions, the present invention has the following advantages: The above-mentioned technical solution of the present invention provides a cross-task transfer method for collaborative optimization of a large-scale inspection and supervision model using LoRA and Bottleneck Adapter. This method involves acquiring original inspection and supervision corpora and constructing a preprocessed corpus dataset for the inspection and supervision domain based on these corpora. LoRA and Bottleneck Adapter technologies are introduced into a general large-scale language model to construct a LoRA-Bottleneck Adapter fusion base model. Based on the training and test sets of multiple task categories in the preprocessed corpus dataset for the inspection and supervision domain, parameter packages are constructed for the LoRA-Bottleneck Adapter fusion base model to obtain LoRA parameter packages with optimal gated weighting ratios for each task category. A pre-defined basic knowledge base for inspection and supervision is structured and modeled to output a structured inspection and supervision knowledge graph. When original multi-source business data from inspection and supervision is received, the corresponding LoRA parameter package with optimal gated weighting ratio is called based on the task category of the original multi-source business data and used as the target LoRA parameter package. The LoRA-Bottleneck Adapter fusion base model is then employed. The Adapter fusion base model processes the original data from multiple sources of inspection and supervision business based on the target LoRA parameter package and the structured inspection and supervision knowledge graph, generating standardized business processing results for inspection and supervision. Based on the above scheme, a standardized corpus dataset of inspection and supervision domain is constructed to lay a unified data foundation for model training. By leveraging the base model structure that integrates LoRA and Bottleneck Adapter, it breaks away from the traditional technical path of full parameter fine-tuning. It only trains for different inspection and supervision task classifications to generate lightweight LoRA parameter packages with optimal gating weight ratios, eliminating the need for repeated training of the model backbone parameters and Bottleneck Adapter parameters. In actual business applications, the corresponding LoRA parameter package can be accurately called directly according to the task classification to complete the model's rapid adaptation. Combined with the structured inspection and supervision knowledge graph, subsequent business reasoning and processing can be carried out. Throughout the process, there is no need to retrain or fine-tune the model parameters on a large scale for each inspection and supervision task. At the same time, relying on the optimal gating weight ratio corresponding to each task classification ensures the accuracy of multi-task adaptation, further enhancing the actual effect of flexible adaptation and rapid transfer of the model in multi-task inspection and supervision scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart illustrates the steps of a cross-task migration method for a large-scale patrol and inspection model that utilizes LoRA and Bottleneck Adapter in a collaborative optimization manner, as provided in Embodiment 1 of the present invention. Figure 2 This is an overall framework diagram of a cross-task migration method for a large-scale patrol and inspection model that is collaboratively optimized by LoRA and Bottleneck Adapter, as provided in Embodiment 1 of the present invention. Figure 3 This is a schematic diagram of the overall architecture of a cross-task migration method for a large-scale patrol and inspection model that is collaboratively optimized by LoRA and Bottleneck Adapter, as provided in Embodiment 1 of the present invention. Figure 4 This is a flowchart illustrating a cross-task migration method for a large-scale patrol and inspection model that utilizes LoRA and Bottleneck Adapter in a collaborative optimization manner, as provided in Embodiment 1 of the present invention. Figure 5 This is a structural block diagram of a cross-task migration system for a large-scale patrol and inspection model that utilizes LoRA and Bottleneck Adapter in collaboration, as provided in Embodiment 2 of the present invention. Detailed Implementation

[0019] This invention provides a method and related equipment for cross-task migration of a large-scale inspection and patrol model using LoRA and Bottleneck Adapter in collaboration, which solves the technical problem that current large-scale models for the inspection and patrol field are difficult to achieve rapid migration and flexible adaptation between multiple inspection and patrol tasks.

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that in the optional embodiments of the present invention, the object information and other related data involved require the permission or consent of the object when the embodiments of the present invention are applied to specific products or technologies, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. That is to say, if the embodiments of the present invention involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of relevant departments, and in compliance with relevant laws, regulations, and standards. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments also need to be implemented with the authorization and consent of the object.

[0021] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of a cross-task migration method for a large-scale patrol and inspection model that utilizes LoRA and Bottleneck Adapter in collaboration, as provided in Embodiment 1 of the present invention.

[0022] This invention provides a cross-task transfer method for collaborative optimization of a large-scale inspection and patrol model using LoRA and Bottleneck Adapter, comprising: Step 101: Obtain the original corpus of inspection and supervision, and construct a preprocessed corpus dataset for the field of inspection and supervision based on the original corpus of inspection and supervision.

[0023] The original text data of inspection and supervision refer to various original text data collected from the business scenarios of inspection and supervision, including but not limited to legal texts, historical inspection cases, rectification records, materials for determining violations of rules and regulations, rectification assessment reports and other business-related text information.

[0024] It should be noted that by standardizing and integrating the original corpus, standardized domain corpus data suitable for model training is formed, providing data support for subsequent model construction and parameter training.

[0025] Further, step 101 may include the following sub-steps: S11. The original corpus of inspection and patrol is deduplicated, noise-reduced and format-standardized in sequence, and the cleaned corpus of inspection and patrol is output. S12. Divide the cleaned patrol and inspection corpus and output the divided patrol and inspection corpus set. S13. Label the divided inspection and patrol corpus and output the preprocessed corpus dataset for the inspection and patrol field.

[0026] It should be noted that the original inspection and patrol data was sequentially deduplicated, denoised, and formatted. By removing duplicate text, filtering irrelevant and interfering information, and standardizing text encoding and layout, the data was cleaned and the cleaned inspection and patrol data was output. The cleaned inspection and patrol data was then divided into training, validation, and test sets according to a preset ratio to ensure a reasonable distribution of data in each subset, and the divided inspection and patrol data sets were output. The divided inspection and patrol data sets were then annotated with domain terminology and task type classification to clarify the business task attributes corresponding to the data, and finally, the inspection and patrol domain preprocessed data set was output. This process achieved standardization and normalization of the data, providing high-quality and highly adaptable data support for the subsequent training of the LoRA-BottleneckAdapter fusion base model and the construction of multi-task parameter packages, effectively avoiding model training bias caused by messy data and non-standard annotation.

[0027] Step 102: Introduce LoRA and Bottleneck Adapter technologies into the general large language model to construct a LoRA-Bottleneck Adapter fusion base model.

[0028] It should be noted that LoRA (Low-Rank Adaptation) and Bottleneck Adapter technologies are introduced into the general-purpose large language model. First, the Bottleneck Adapter bottleneck layer is embedded in the Transformer layer of the general-purpose large language model. Then, pluggable LoRA sub-modules are configured in its attention layer and feedforward layer. The backbone parameters of the model are frozen and the relevant parameters of the two technologies are trained in a coordinated manner. In this way, a LoRA-Bottleneck Adapter fusion base model is constructed, which provides basic model support for the subsequent construction of multi-task parameter packages and cross-task adaptation.

[0029] Furthermore, step 102 may include the following sub-steps: S21. Embed the Bottleneck Adapter bottleneck layer into the Transformer layer of the general large language model to obtain the basic model embedded with the Bottleneck Adapter; S22. Configure pluggable LoRA sub-modules for the attention layer and feedforward layer of the base model embedded with Bottleneck Adapter, and output the LoRA-Bottleneck Adapter dual-module base model. S23. Introduce a gating weighting mechanism to the LoRA-Bottleneck Adapter dual-module basic model and output a LoRA-Bottleneck Adapter dual-module basic model with gating weighting mechanism; S24. Freeze the backbone parameters of the LoRA-Bottleneck Adapter dual-module base model with gating weighting mechanism, and train the Bottleneck Adapter model parameters and LoRA model parameters together to obtain the LoRA-Bottleneck Adapter fusion base model.

[0030] It should be noted that, as Figure 2As shown, this invention introduces the LoRA module into the pre-trained inspection and patrol model (i.e., the general large language model) to perform low-rank decomposition and fine-tuning of the model's key weight matrix, thereby achieving efficient updates of the model's shared parameters. At the same time, task-specific Adapter modules (i.e., Bottleneck Adapters) are inserted into each layer of the model to carry feature representations for different business scenarios such as legal analysis, case retrieval, violation judgment, problem list generation, and rectification effect evaluation.

[0031] During model execution, this invention can dynamically load the corresponding Adapter module according to the target task and combine it with LoRA to fine-tune parameters, enabling rapid switching and seamless migration of the model between tasks. This approach not only reduces the computational overhead of updating all parameters but also improves the model's adaptation speed and generalization ability on new tasks.

[0032] This invention also provides task management, knowledge base management and intelligent generation modules to complement the model, supporting functions such as rapid retrieval of laws, regulations and historical cases, knowledge graph construction, supervision data fusion, intelligent report and rectification list generation, realizing intelligent support for the entire process of inspection and supervision business from data collection, problem identification, report generation to rectification tracking.

[0033] In summary, the overall framework of this invention consists of five parts: model adaptation and optimization module, task management module, knowledge base and knowledge graph management module, intelligent generation module, and data processing and interface module. The model adaptation and optimization module includes a LoRA submodule, an Adapter submodule, and a fusion scheduling unit. The LoRA submodule performs low-rank decomposition on the task-related weight matrix in the large model, updating only the low-rank matrix parameters while keeping most parameters frozen, achieving efficient fine-tuning and reducing computational and storage overhead. The Adapter submodule inserts small, pluggable adapter layers into each Transformer layer of the large model to carry feature representations and business logic for specific tasks (such as regulatory analysis, violation judgment, rectification assessment, etc.), achieving parameter isolation and rapid switching between tasks. The fusion scheduling unit dynamically calls the combined parameters of LoRA and Adapter according to task requirements during the inference or training phase, achieving efficient cross-task migration.

[0034] The task management module is responsible for the creation, classification, scheduling, and execution of inspection and supervision tasks. It supports the automatic generation of task key points based on regulations, policies, supervision results, and multi-source business data, and triggers corresponding model adaptation configurations. Specifically, this module manages the entire process of inspection and supervision tasks, including task creation, classification, scheduling, and execution. The system first automatically generates task key points based on regulations, policies, historical supervision results, and multi-source business data, such as identifying key areas and issues requiring inspection. Then, tasks are classified according to type and priority, and the corresponding model adaptation configuration is triggered (by calling the corresponding LoRA / Adapter parameter package). During execution, the task management module monitors task progress, records processing logs, and dynamically updates task priorities and subsequent action arrangements based on feedback. Through this process, automated orchestration and traceable management of inspection and supervision tasks are achieved.

[0035] The knowledge base and knowledge graph management module maintains a legal and policy database, a historical inspection case database, a common problem database, and a rectification plan database. It constructs a knowledge graph of entities and relationships to enhance the model's retrieval and reasoning capabilities and provide a basis for judging violations and improving problem identification. Specifically, this module maintains various inspection-related knowledge bases, including a legal and policy database (storing the latest legal and regulatory provisions), a historical inspection case database (storing problems discovered and their handling results from past inspections), a common problem database, and a rectification plan database. Simultaneously, it performs structured modeling of entities and relationships within this knowledge base, constructing a knowledge graph. Knowledge graph nodes represent entities such as legal entries, units, and problem types, while edges represent the relationships between them (e.g., applicable legal areas, institutional jurisdiction, etc.). The model can retrieve information from the knowledge graph during reasoning to provide background information for judging violations, conducting in-depth problem analysis, and making rectification suggestions. For example, when analyzing a violation, the model can query the legal graph to obtain relevant legal provisions and compare them with historical cases, thereby improving the accuracy and interpretability of the judgment. The knowledge base and knowledge graph are continuously updated to ensure the system can promptly track new regulations and institutional changes.

[0036] For the intelligent generation module, based on model output and task templates, it automatically generates inspection reports, lists of key issues, problem drafts, and rectification suggestions, ensuring the standardization and normalization of the output. Specifically, the intelligent generation module automatically writes inspection-related documents based on task templates and model outputs. This includes automatically generating inspection reports, lists of key issues, problem drafts, and rectification suggestions. This module combines the model's inference results with preset document formats, ensuring the standardization of output content through template filling or text splicing. For example, after the model identifies key issues, the report generation module converts them into written report paragraphs, attaching legal basis and rectification measures suggestions. The output text is formatted and validated to meet the requirements of legal text specifications and formats, achieving automation and standardization of inspection documents. In addition, this module can be iteratively optimized based on user feedback to improve the accuracy and professionalism of the generated results.

[0037] The data processing and interface module provides functions such as external system integration, data import and cleaning, OCR (Optical Character Recognition), and multi-source data aggregation, achieving seamless integration with business systems. Specifically, this module is responsible for interfacing with external business systems and data preprocessing, including data import, cleaning, OCR recognition, and multi-source data fusion. It converts various inspection-related data (such as electronic files, on-site scanned documents, video and audio content, etc.) into structured information that can be processed by the model. For example, it performs OCR recognition on scanned paper materials and cleans and segments the results for subsequent model calls. The module also supports real-time interfaces to share inspection data with the monitoring system, enabling synchronized updates of task progress and results. Through effective aggregation and standardized processing of multi-source data, it ensures that each module can seamlessly obtain the required information, thereby improving the overall system's response speed and reliability.

[0038] The above technical modules complement each other, forming a complete solution for the inspection system. Among them, the model adaptation and optimization module achieves efficient enhancement of the pre-trained model through complex mathematical mechanisms (low-rank decomposition, bottleneck adaptation layer, and gated weighted fusion); the task management and knowledge base modules provide task-driven configuration and data support for the model; and the intelligent generation and data processing module completes the final output generation and system integration, ensuring the automation and intelligence of the entire inspection process.

[0039] Specifically, for the model adaptation and optimization module, this module achieves efficient task-level fine-tuning and parameter isolation by inserting LoRA and Adapter structures into the Transformer layer of the general large language model. This invention introduces mathematical formulas to describe each sub-module: The LoRA submodule achieves efficient fine-tuning by adding a low-rank increment AB to the pre-trained weight matrix W0. Given the original weights W0 and the low-rank matrix AB, the fine-tuned weights are as follows: W = W0 + AB; The Adapter submodule (i.e., the Bottleneck Adapter layer): Small bottleneck layers are inserted into each sublayer of the Transformer layer (such as after the feedforward network), and their outputs are fed back into the input via residual connections. Mathematically, this can be expressed as: h out =h in +W up f (W) down h in )); Among them, h in ∈R d For the input feature vector, W down Compress the features to the bottleneck dimension b. f () is an activation function (such as ReLU), W up ∈R d ×d, then project the features back to d dimensions. In the formula, W up f (W) down h in () indicates the transformation performed by the Adapter on the input features, which is then compared with the original input h. in The residual connections are achieved through addition. This structure can effectively capture the features of a specific task while maintaining decoupling from the original model, facilitating rapid switching between different tasks.

[0040] Fusion Scheduling Unit: A gating mechanism is introduced during the inference or training phase to weightedly fuse the outputs of the LoRA path and the Adapter path. Let the current layer input be h, then a gating vector g (i.e., the gating weight ratio, with each element taking values ​​in the range [0,1]) is defined to balance the contributions of the two paths. For example, a simple gating form is: ; in, and For gating network parameters, This is the element-wise Sigmoid function. Combining the two fine-tuning paths mentioned above, we can obtain the output of this layer: h out =g⊙(ABh)+(1-g)⊙W up f(W down h); Among them, A B h represents the low-rank incremental output of the LoRA path, Wup f(W down h) represents the Adapter path output, and ⊙ represents element-wise multiplication. In simpler terms, when g is close to 1, it indicates that this dimension relies more on LoRA fine-tuning; when g is close to 0, it relies more on Adapter adjustment. Through this gated weighting mechanism, the model can adaptively select the fine-tuning path under different tasks and input features, achieving efficient combination and multi-task sharing of LoRA and Adapter, and improving the model's transferability and generalization performance in patrol and inspection scenarios.

[0041] As shown above, this module mathematically integrates the advantages of LoRA and Adapter, maintaining the lightweight nature of LoRA's low-rank fine-tuning while achieving the flexibility of Adapter's rapid switching between different tasks. Simultaneously, the introduced gating weighting mechanism ensures adaptive path selection during multi-task migration, thereby effectively improving the model's overall performance and stability in the field of inspection and patrol when facing various regulations, policies, and tasks.

[0042] It is worth mentioning that, in addition to the LoRA and Adapter fusion scheme proposed in this invention, the same inventive objective can be achieved through various alternative techniques. For example, PromptTuning / PrefixTuning can be used to add trainable cue vectors to the input or attention layer, BitFit can be used to update only the bias parameters, or (IA)³ (Intermediate Activations Scaling) can be used to introduce scaling vectors in the attention and feedforward modules. Alternatively, Compactor or PHM Adapter (Parameterized Hypercomplex Multiplication) can be used. Low-rank shared structures such as the Adapter (parameterized hypercomplex multiplication adapter) can replace traditional Adapters, or the weights can be sparsified through DiffPruning to update only some parameters. In addition, LoRA can be replaced by other low-rank decompositions or reparameterization methods (such as Kronecker decomposition and Hadamard decomposition). The Adapter can be replaced by lightweight MLP (Multilayer Perceptron), convolution or normalization enhancement modules. In terms of method steps, cross-task migration can also be achieved through multi-task prompt tuning joint training, weight merging, sparsification or parameter routing mechanisms, so as to ensure that lightweight, multi-task efficient adaptation and fast migration can be achieved under different implementation paths.

[0043] In this embodiment, a BottleneckAdapter bottleneck layer is embedded into each sub-layer of the Transformer layer of the general-purpose large language model. By building a small bottleneck structure and embedding it into the core layer of the model, the initial domain adaptation capability of the model is achieved, resulting in a basic model with an embedded BottleneckAdapter. Adaptation interfaces are reserved for the attention layer and feedforward layer of the basic model with the embedded BottleneckAdapter, and pluggable LoRA sub-modules are configured to allow the LoRA module to be flexibly connected and disassembled, outputting a LoRA-BottleneckAdapter dual-module basic model. A gating weighting mechanism is introduced into the LoRA-BottleneckAdapter dual-module basic model. By setting the gating unit to adjust the output ratio of the BottleneckAdapter and LoRA module, the collaborative adaptation of the two fine-tuning techniques is achieved, outputting a LoRA-BottleneckAdapter dual-module basic model with a gating weighting mechanism. The backbone parameters of the general-purpose large language model are frozen in the LoRA-BottleneckAdapter dual-module basic model with a gating weighting mechanism, targeting only the Bottleneck. The parameters of the Adapter model and the LoRA model are trained together to ensure that the parameters of the two modules are compatible and adapted, and finally the LoRA-Bottleneck Adapter fusion base model is obtained. This process constructs a basic model architecture that can support multi-task adaptation through hierarchical embedding and co-training of two fine-tuning techniques, avoiding the high overhead problem of traditional full parameter training, and providing a stable model carrier for the subsequent construction of parameter packages for each task classification.

[0044] Step 103: Based on the training and validation sets of multiple task classifications in the preprocessed corpus dataset of the inspection and patrol domain, construct the parameter package for the LoRA-Bottleneck Adapter fusion base model to obtain the LoRA parameter package with the optimal gated weighted weight ratio for each task classification.

[0045] It should be noted that, based on the multiple task classifications in the preprocessed corpus dataset of the inspection and supervision field, the training set and validation set corresponding to each task are extracted. The training set is input into the LoRA-Bottleneck Adapter fusion base model, the backbone parameters of the model are frozen, and only the LoRA-related parameters are trained. The optimal weight ratio is adjusted by combining the gating weighting mechanism, and the training effect is verified at the same time. Finally, the LoRA parameter package with the optimal gating weighting ratio corresponding to each task classification is constructed, providing lightweight parameter support for subsequent task adaptation.

[0046] Furthermore, step 103 may include the following sub-steps: S31. Freeze the backbone parameters and BottleneckAdapter model parameters of the LoRA-Bottleneck Adapter fusion base model, and use the training sets of each task classification to jointly train the LoRA model parameters and gated weighting ratio of the LoRA-Bottleneck Adapter fusion base model to obtain the trained LoRA-Adapter fusion patrol and inspection large model corresponding to each task classification. S32. The performance of the corresponding trained LoRA-Adapter fusion inspection and patrol model is verified using the validation set of each task category, so as to obtain the verified LoRA-Adapter fusion inspection and patrol model for each task category. S33. Extract LoRA parameters and optimal gating weight ratio from the verified LoRA-Adapter fusion inspection and patrol model corresponding to each task category, and output the LoRA parameter package with optimal gating weight ratio for each task category.

[0047] Task classification refers to the specific division of inspection and supervision tasks based on the needs of inspection and supervision business scenarios. The core is to break down inspection and supervision work into different specific business task types so as to carry out targeted model training, parameter configuration and business processing. Common task classifications include legal analysis, violation judgment, rectification tracking and document archiving, etc. Each task classification corresponds to an independent business scenario and processing requirements. Through clear task classification, the model can be accurately adapted and efficiently responded to, avoiding repeated training, thereby solving the problem of rapid adaptation in multi-task scenarios. It forms a logical closed loop with subsequent parameter package construction, task invocation and other links to ensure the orderly development and efficient progress of different tasks.

[0048] It should be noted that the backbone parameters of the LoRA-Bottleneck Adapter fusion base model and the Bottleneck Adapter model parameters are frozen, and these two types of parameters are fixed and do not participate in subsequent training. Only the training sets of each task category are input into the fusion base model to jointly train the LoRA model parameters and the gated weighting ratio, simultaneously optimizing both parameters to adapt to the business requirements of the corresponding task categories. After training, the trained LoRA-Adapter fusion inspection and patrol models for each task category are obtained. The validation sets of each task category are input into the corresponding trained LoRA-Adapter fusion inspection and patrol models. The performance indicators such as inference accuracy and adaptation stability of the models are tested through the validation sets, and models that meet the performance standards are selected to obtain the models for each task category. The corresponding validated LoRA-Adapter fusion inspection model was developed. Parameters were extracted from the validated LoRA-Adapter fusion inspection model for each task category, separating the LoRA parameters and the optimal gated weighting ratio from the model. These two parameters were then encapsulated and integrated to output the LoRA parameter package with the optimal gated weighting ratio for each task category. This process, through freezing core parameters, targeted joint training, and performance verification, ensures that the parameter package for each task category is lightweight and highly adaptable. Subsequently, the parameter package can be directly called to quickly adapt the model without repeatedly training the model backbone and Bottleneck Adapter parameters, effectively reducing the computational overhead of multi-task adaptation.

[0049] Step 104: Perform structured modeling on the pre-set basic knowledge base of inspection and supervision, and output a structured knowledge graph of inspection and supervision.

[0050] Structured inspection and supervision knowledge graph refers to a knowledge network with clear logic that can be directly used for model reasoning, formed by sorting out the relationships based on information such as laws, cases and business norms related to inspection and supervision, and can quickly call up various business-related knowledge.

[0051] The basic knowledge base for inspection and supervision refers to the collection and organization of basic data used for model training and business processing. It includes laws, cases, business norms and other content related to inspection and supervision, and is the core basis for building a knowledge graph.

[0052] It should be noted that the pre-set basic knowledge base for inspection and supervision is processed through structured modeling. First, the information such as laws and regulations, case data, and business specifications in the base is sorted out, and the data format and classification standards are unified. Then, by extracting key information and establishing relationships, the scattered knowledge is integrated to build a structured knowledge graph for inspection and supervision. Finally, the structured knowledge graph for inspection and supervision is output, which provides accurate knowledge support for subsequent business processing and cross-task adaptation. At the same time, it is ensured that the knowledge graph is linked with the pre-processed corpus and model parameters to further improve the accuracy of cross-task adaptation.

[0053] Step 105: When receiving the original data of the multi-source business of the inspection and supervision, the corresponding LoRA parameter package with the optimal gated weighted weight ratio is called based on the task classification of the original data of the multi-source business of the inspection and supervision, and is used as the target LoRA parameter package.

[0054] The multi-source original business data of inspection and supervision refers to the original business data collected from various business scenarios of inspection and supervision, including different forms of business data such as on-site verification materials, business documents, and system ledgers.

[0055] It should be noted that when receiving raw data from multiple sources of inspection and supervision business, the task attributes of this type of data are first determined to clarify its task category. Then, based on the task category results, the corresponding LoRA parameter package with the optimal gated weighted weight ratio is accurately matched and called from the generated task category parameter packages. This LoRA parameter package is determined as the target LoRA parameter package used for the current business processing. By directly matching the task category with the dedicated parameter package, the repeated fine-tuning and training of the model is eliminated, which greatly shortens the model adaptation preparation cycle in multi-task scenarios and further enhances the ability of the inspection and supervision large model to quickly switch across tasks.

[0056] Step 106: Using the LoRA-Bottleneck Adapter fusion base model, the original data of multi-source inspection and supervision are processed according to the target LoRA parameter package and the structured inspection and supervision knowledge graph to generate standardized inspection and supervision business processing results.

[0057] The standardized business processing results of inspection and supervision refer to the standardized output results generated after processing multi-source business data based on the fusion base model, target LoRA parameter package and structured knowledge graph, which conform to the business specifications and format requirements of inspection and supervision, and cover business outputs such as problem list, assessment report, and legal application conclusion.

[0058] It should be noted that the LoRA-Bottleneck Adapter fusion base model is used to load the target LoRA parameter package to complete rapid parameter adaptation. Combined with the structured inspection and supervision knowledge graph, business knowledge support is provided. Targeted business reasoning and integration processing are carried out on the multi-source business raw data of inspection and supervision, and finally, standardized business processing results of inspection and supervision are generated.

[0059] Furthermore, step 106 may include the following sub-steps: S61. Perform optical character recognition, data cleaning, and format conversion on the original data of multi-source inspection and supervision in sequence, and output structured inspection and supervision business data. S62. The LoRA-Bottleneck Adapter fusion base model is updated with the target LoRA parameter package to obtain the patrol and inspection cross-task online inference model. S63. The cross-task online reasoning model for inspection and supervision is adopted to perform cross-task intelligent reasoning processing on structured inspection and supervision business data based on the structured inspection and supervision knowledge graph, and output the intelligent reasoning results of inspection and supervision tasks. S64. Based on the intelligent reasoning results of the inspection and supervision tasks and the preset inspection and supervision business template, perform template filling and text splicing processing to output the initial standardized business document for inspection and supervision. S65. Verify and correct the initial standardized business documents for inspection and supervision, and output the standardized business processing results for inspection and supervision.

[0060] The cross-task online reasoning model for inspection and supervision refers to a specialized model that can adapt to the current task classification and supports online real-time reasoning after loading the target LoRA parameter package and updating the parameters based on the LoRA-Bottleneck Adapter fusion base model.

[0061] The inspection and supervision business template refers to a standardized document framework formulated in advance according to the inspection and supervision business specifications, which is used to standardize the text format and content structure of the business processing results.

[0062] It should be noted that the original data from multiple sources of inspection and supervision business undergoes optical character recognition (OCR), data cleaning, and format conversion sequentially. OCR transforms unstructured text, such as images, into computable text. Data cleaning removes invalid interference and unifies the data storage format, outputting structured inspection and supervision business data. A target LoRA parameter package is used to perform lightweight parameter updates on the LoRA-Bottleneck Adapter fusion base model, replacing only task-specific parameters rather than fully adjusting model weights, resulting in a cross-task online inference model for inspection and supervision. This cross-task online inference model, combined with the associated knowledge support of a structured inspection and supervision knowledge graph, performs business logic reasoning and information analysis on the structured inspection and supervision business data, outputting intelligent inference results for inspection and supervision tasks. Based on the intelligent inference results, a pre-set inspection and supervision business template is matched to complete the filling of key business information and the splicing of standardized text, outputting an initial standardized inspection and supervision business document. The initial standardized inspection and supervision business document undergoes content completeness and compliance verification, correcting expression deviations and data omissions, ultimately outputting the standardized business processing results for inspection and supervision.

[0063] For example, such as Figure 3 The diagram illustrates the fine-tuning and deployment process of a large-scale inspection and monitoring model based on LoRA and Adapter fusion optimization. First, the user uploads the model or task data to be adapted. The system identifies the adaptation strategy to determine whether to use LoRA, Adapter, or gated fusion path. In the LoRA path, a lightweight LoRA parameter package is generated through parameter injection and low-rank fine-tuning training. In the Adapter path, bottleneck structures are inserted into the network layers and adapted during training, generating the corresponding Adapter parameter package. In the gated fusion path, the system designs a fusion strategy and performs weighted calculations using gate variables to obtain the fused adapted package. Subsequently, the parameter results generated from each path are aggregated into a multi-path fusion and scheduling module, achieving collaborative optimization of different adaptation methods. This ultimately completes the model deployment and online inference, while also supporting version control and task migration.

[0064] Specifically, the data and basic model were first prepared in the computing environment (software: PyTorch 2.x + CUDA 12, hardware: NVIDIA A100 80GB or equivalent GPU (Graphics Processing Unit), mixed precision FP16 (mixed precision 16-bit floating-point arithmetic)): the corpus related to inspections and patrols (legal texts, historical cases, rectification records, etc., with an example scale of about 100,000 documents) was collected and cleaned, and the training / validation / test sets were divided into 80 / 10 / 10 sets; a general large language model based on Transformer was selected as the basis (such as LLaMA / GPT-2, etc., with the model dimension d clearly recorded).

[0065] The second step is to insert LoRA and Adapter structures into the model: inject LoRA low-rank increments into the key weight matrices (such as the attention projection matrix and feedforward layer weights), with an example low-rank r=8, initialized with zero mean and small variance; insert an Adapter bottleneck layer after each feedforward network layer, with an example bottleneck dimension b=32, activation function f(·)=ReLU, and residual connections remain unchanged.

[0066] The third step is to configure the training hyperparameters and perform joint training: using the AdamW optimizer (weight decay =0.01), LoRA learning rate lr LoRA =1×e -4 Adapter learning rate lr_Adapter = 5 × e -4 The batch size is 32, the number of training rounds is 3–10, gradient accumulation and learning rate scheduling (linear decay) are used, and dropout=0.1 and early stopping strategy are enabled; the gating network adopts a single-layer fully connected + Sigmoid, the output dimension is the same as the hidden dimension d, and it is used to perform weighted fusion of LoRA and Adapter paths by element (or by channel). Its parameters are optimized along with the training process or only fine-tuned to save costs.

[0067] The fourth step is versioning and evaluation: The trained LoRA parameter package and Adapter parameter package are saved in .pt format, managed using semantic version numbers (e.g., v1.0.0), and the training environment and random seed are recorded. Task-level evaluation metrics (accuracy of legal interpretation, recall / precision / F1 of case retrieval, accuracy of violation judgment) and system metrics (average response latency, GPU memory usage) are used to compare and verify with baselines (full fine-tuning, LoRA only, Adapter only). Specifically, this invention assigns different weights to LoRA and Bottleneck Adapter, and debugs the optimal LoRA-Adapter weight ratio (i.e., gated weighted weight ratio) for different downstream tasks. Because the Bottleneck Adapter is embedded in the model, an optimal weight ratio can be recorded for each LoRA. The overall LoRA data package is managed using version numbers.

[0068] Step 5: Deployment and Online Scheduling: Implement a dynamic loading mechanism in the inference service, loading the corresponding Adapter / LoRA package on demand according to the task type and enabling gating fusion; support canary releases, rollbacks, and A / B testing; the monitoring module collects business feedback in real time for online fine-tuning or offline retraining. Step 6: Operation, Maintenance, and Security Compliance: Define parameter package access permissions, log auditing, and model interpretability checking processes; regularly update the knowledge base and index the knowledge graph for inference enhancement; if using self-developed modules (e.g., self-developed Adapter units), provide their implementation code or operator specifications and unit test reports to ensure that those skilled in the art can reproduce and implement this invention based on the above parameters, equipment, and steps.

[0069] Furthermore, the specific equipment required for this invention includes high-performance training and inference hardware and corresponding software environment. For the training phase, it is recommended to use NVIDIA A100 80GB×4 or H100 80GB×2 GPUs paired with Intel Xeon Gold 6348×2 or AMD EPYC 9654×2 CPUs, along with 512GB of memory and high-speed NVMe SSD storage to meet the computational requirements of low-rank fine-tuning and multi-task adapter module training; historical inspection cases and knowledge base data can be stored on a 20TB enterprise-grade HDD. For the deployment and inference phase, an NVIDIA A40 48GB×2 GPU can be configured, paired with Intel Xeon Silver 4314×2 CPUs and 256GB of memory to ensure low latency and high throughput for online inference. The network must support 10 Gigabit Ethernet to meet the requirements of distributed training and multi-node inference. The software environment includes PyTorch 2.x and CUDA / cuDNN 12.x, distributed training libraries (such as NCCL), containerization tools Docker and Kubernetes, as well as OCR, data cleaning and knowledge base management tools. Combined with version management and monitoring systems, it enables efficient, controllable and traceable model training, deployment, task scheduling and data processing.

[0070] For comparison of technical effectiveness, existing technologies can be used as a reference. In power grid operations, inspection and supervision scenarios are complex, involving multiple tasks such as legal analysis, case retrieval, violation judgment, problem list generation, and rectification implementation evaluation. Under the current model, the large-scale model lacks sufficient transferability between different tasks, often requiring retraining or deep fine-tuning for each task, resulting in long development cycles, high computational costs, and low deployment efficiency. Simultaneously, inspection personnel face problems such as low efficiency, insufficient standardization, and low knowledge utilization in areas like legal retrieval, historical case utilization, and document generation, hindering the precision and routine implementation of supervision work.

[0071] Based on the above, existing methods have the following drawbacks: First, cross-task transfer capabilities are limited. When there are significant differences between tasks, the adapter's effectiveness may be limited. Cross-domain task transfer can easily lead to performance degradation, especially when there are significant differences in task requirements, data distribution, or domain knowledge. Second, computational and data requirements are high. Training and fine-tuning require a large amount of computing resources and high-quality domain data. The legal field involves a large amount of complex data collection, annotation, and processing, which increases system costs and limits its application in resource-constrained environments. Third, there are interpretability and compliance issues. Although fine-tuning techniques can improve the model's task adaptability, large language models usually have "black box" characteristics, which are particularly sensitive in fields such as legal services. The model's decision-making process lacks clear interpretability, posing challenges to decision support and compliance in the legal field. In scenarios where transparency and accuracy of legal services need to be ensured, it may also raise trust and legal liability issues.

[0072] Therefore, in the field of inspection and supervision, existing large-scale models have many technical shortcomings: limited cross-task transfer capabilities, when facing different tasks such as legal analysis, case retrieval, and qualitative and quantitative disciplinary work, it is usually necessary to retrain or fine-tune on a large scale for each task, making it difficult to achieve rapid transfer and adaptation; insufficient generalization performance, with unstable performance in new tasks outside the training tasks, unable to maintain high-precision output, affecting the actual effect of multi-task inspection and supervision work; low deployment efficiency and flexibility, lacking a lightweight task adaptation mechanism, making it difficult to quickly respond to business needs in multi-task and multi-scenario situations, limiting the efficiency and intelligence level of inspection and supervision work.

[0073] The emergence of Low-Rank Adaptation (LoRA) and Adapter techniques has provided a feasible path to improve the cross-task transferability of large models. LoRA significantly reduces the scale of parameter updates and computational overhead by performing low-rank decomposition and training on a portion of the weight matrix while keeping most of the original model's parameters frozen, making it suitable for efficient task customization in resource-constrained environments. Adapter technology, on the other hand, separates task-specific representation learning from the main body of the original model by inserting small adaptation modules into each layer of the model. This allows the model to quickly load and replace adaptation layers for different tasks, enabling rapid switching and incremental expansion between tasks.

[0074] By integrating LoRA with an Adapter, LoRA enables efficient fine-tuning of shared parameters, while the Adapter carries task-specific knowledge. This approach enhances task adaptability and transfer efficiency while maintaining model generalization capabilities. This fusion solution is particularly suitable for multi-task, multi-domain, and multi-stage business scenarios such as inspection and supervision. It significantly shortens the switching time between tasks like regulatory analysis, problem identification, and rectification assessment, reduces repetitive training costs, and improves the digitalization and intelligence of supervision work.

[0075] Therefore, this invention optimizes the large-scale inspection and patrol model by integrating LoRA and Adapter technologies, achieving a synergistic effect of low-rank fine-tuning and task-specific adaptation. This enables the model to efficiently migrate and quickly adapt across different tasks, significantly improving its generalization ability while reducing training and maintenance costs, and increasing deployment efficiency and response speed. It not only meets the application needs of multi-task and multi-scenario inspection and patrol, and enhances the intelligence level and practical application value of the large-scale model, but also provides a referable technical path for the efficient migration and lightweight fine-tuning of large-scale models in the field of inspection and patrol.

[0076] Specifically, such as Figure 4 As shown, a general-purpose large language model is first selected as the foundation, and its understanding of patrol and inspection corpora is enhanced through further pre-training in the domain. Then, LoRA modules are inserted into the attention and feedforward layers, and an Adapter bottleneck structure is added to the layer output. The two are then merged to form a composable parameter package. Subsequently, joint training is carried out based on multi-task data, and an adaptive gating mechanism is introduced to dynamically select the LoRA or Adapter path to ensure both sharing and feature consideration. In the cross-task migration and online deployment stage, only the corresponding parameter package needs to be trained when adding a new task. During inference, dynamic combination is achieved through weighting or routing, and retrieval enhancement is combined to ensure the adaptation of long texts and new regulations. Finally, the LoRA-Adapter parameter packages for different tasks are stored independently and versioned, supporting fast start-up and stop, gray-scale updates and rollback to meet compliance and traceability requirements.

[0077] In summary, the key technical point of this invention lies in integrating LoRA and Adapter into a large-scale inspection and monitoring model. While keeping most parameters frozen, LoRA enables efficient low-rank fine-tuning, while the Adapter carries task-specific feature representations. The two are combined and dynamically weighted using a gating mechanism to achieve rapid switching and efficient migration across tasks. Simultaneously, supporting modules for task management, knowledge base and knowledge graph, intelligent generation, and data interface are provided to construct a complete intelligent inspection and monitoring system. The main points to be protected include: the collaborative optimization method and gating fusion mechanism of LoRA and Adapter, the cross-task dynamic loading and migration adaptation scheme, and the systematic application architecture for inspection and monitoring scenarios.

[0078] Compared with existing technologies, this invention improves the multi-task generalization capability of a single model in a vertical domain through efficient parameter fine-tuning technology, focusing on algorithmic innovation in model compression and transfer efficiency. Its technical means mainly revolve around the fusion mechanism of LORA and Adapter, enabling a single base model in the field of inspection and patrol to flexibly adapt to sub-tasks such as clue analysis and report generation by freezing the backbone parameters and adding an adaptation layer.

[0079] Existing methods construct a multi-agent collaborative system-level solution. Their technological focus is not on optimizing model generalization capabilities, but on establishing domain-specific model matrices. This is achieved through the division of labor and collaboration of dedicated large language models for four major domains—civil, criminal, commercial, and administrative—combined with five functional matrices to decouple tasks. Each agent only needs to handle specific domain-specific tasks. Regarding knowledge enhancement, this invention relies on the model's own parameterized knowledge storage, while the prior art design uses a seven-dimensional legal knowledge base system. Through RAG (Retrieval-Augmented Generation) technology, external authoritative legal provisions and cases are dynamically injected to form a retrieval enhancement closed loop. Ultimately, the innovative value of this invention lies in improved model deployment efficiency and lightweight adaptation, while the core breakthrough of the prior art lies in constructing an end-to-end intelligent system covering all legal scenarios, realizing a complete service chain from request parsing to 3D visualization. These two approaches respectively represent different development directions in model-level fine-tuning technology and system engineering applications.

[0080] In this embodiment of the invention, a cross-task transfer method for a large-scale inspection and patrol model collaboratively optimized using LoRA and Bottleneck Adapter is provided. By organically integrating LoRA and Adapter technologies, the large-scale inspection and patrol model is optimized, achieving efficient transfer and rapid adaptation between different tasks. Simultaneously, while maintaining most of the original model's parameters unchanged, the high accuracy, stability, and robustness of the model are ensured. This technical solution can significantly enhance the model's generalization ability in diverse inspection and patrol tasks such as regulatory analysis, case retrieval, and qualitative and quantitative disciplinary investigations. It effectively solves the problem of performance degradation of traditional large-scale models in new or cross-task scenarios, thereby avoiding the high computational cost and time overhead caused by frequent full retraining.

[0081] Meanwhile, by introducing the low-rank adjustment (LORA) technique, the model can efficiently fine-tune task-specific parameters while preserving its core semantic representation capabilities. Through the Adapter technique in the task-specific adaptation layer, the model can quickly adapt to the features and requirements of new tasks, achieving lightweight and highly efficient transfer learning. The synergistic effect of these two techniques not only improves the model's response speed and deployment flexibility across multiple tasks and scenarios but also significantly reduces model training and maintenance costs, enhancing the scalability and practicality of inspection and supervision work.

[0082] Furthermore, this invention can meet the diverse and complex business needs of inspection and supervision work, enabling the model to maintain high accuracy and consistency when handling tasks such as legal analysis, case retrieval, and qualitative and quantitative disciplinary analysis. It also provides a referable technical path for cross-task migration and lightweight fine-tuning of large models in other professional fields, significantly improving the intelligence level, application efficiency, and practical value of large models, and providing an efficient and scalable model optimization solution for related fields.

[0083] Please see Figure 5 , Figure 5 This is a structural block diagram of a cross-task migration system for a large-scale patrol and inspection model that utilizes LoRA and Bottleneck Adapter in collaboration, as provided in Embodiment 2 of the present invention.

[0084] This invention provides a cross-task migration system for collaborative optimization of a large-scale patrol and inspection model using LoRA and Bottleneck Adapter, comprising: The acquisition module 501 is used to acquire the original corpus of inspection and supervision, and to construct a preprocessed corpus dataset in the field of inspection and supervision based on the original corpus of inspection and supervision. Model building module 502 is used to introduce LoRA and Bottleneck Adapter technologies into the general large language model and build a LoRA-Bottleneck Adapter fusion base model; The parameter package construction module 503 is used to construct the parameter package for the LoRA-Bottleneck Adapter fusion base model based on the training set and validation set of multiple task classifications in the preprocessed corpus dataset of the inspection and patrol domain, and obtain the LoRA parameter package with the optimal gated weighted weight ratio for each task classification. The graph modeling module 504 is used to perform structured modeling processing on the preset basic knowledge base of inspection and supervision, and output a structured knowledge graph of inspection and supervision. Module 505 is selected to call the corresponding LoRA parameter package with the optimal gated weighted weight ratio based on the task classification of the original data of the multi-source business of the inspection and patrol when the original data of the multi-source business of the inspection and patrol is received, and use it as the target LoRA parameter package. The generation module 506 is used to perform business processing on the original data of multi-source inspection and supervision based on the target LoRA parameter package and the structured inspection and supervision knowledge graph using the LoRA-Bottleneck Adapter fusion base model, and generate standardized business processing results for inspection and supervision.

[0085] Furthermore, the model building module 502 is specifically used for: By embedding the Bottleneck Adapter bottleneck layer into the Transformer layer of the general large language model, a base model with embedded Bottleneck Adapter is obtained. Configure pluggable LoRA submodules for the attention layer and feedforward layer of the base model embedded with Bottleneck Adapter, and output a LoRA-Bottleneck Adapter dual-module base model; A gating weighting mechanism is introduced into the LoRA-Bottleneck Adapter dual-module basic model to output a LoRA-Bottleneck Adapter dual-module basic model with gating weighting mechanism; The backbone parameters of the LoRA-Bottleneck Adapter dual-module base model with gating weighting mechanism are frozen, and the parameters of the Bottleneck Adapter model and the LoRA model are trained together to obtain the LoRA-Bottleneck Adapter fusion base model.

[0086] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0087] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the cross-task migration method of the LoRA and Bottleneck Adapter collaborative optimization patrol and inspection large model as described in the above embodiments.

[0088] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the cross-task migration method for the LoRA and Bottleneck Adapter collaborative optimization of the large-scale patrol and inspection model as described in the above embodiments.

[0089] This invention also provides a computer program product, including a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer performs the steps of the cross-task migration method for the LoRA and Bottleneck Adapter collaborative optimization of the patrol and inspection large model as described in the above embodiments.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0091] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0093] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-task transfer method for collaborative optimization of a large-scale patrol and inspection model using LoRA and Bottleneck Adapter, characterized in that, include: Obtain the original corpus of inspection and supervision, and construct a preprocessed corpus dataset for the field of inspection and supervision based on the original corpus of inspection and supervision. LoRA and Bottleneck Adapter technologies are introduced into the general large language model to construct a LoRA-Bottleneck Adapter fusion base model; Based on the training and validation sets of multiple task categories in the preprocessed corpus dataset of the inspection and patrol domain, the LoRA-Bottleneck Adapter fusion base model is constructed with parameter packages to obtain LoRA parameter packages with optimal gated weighted weight ratios for each task category. The pre-set basic knowledge base of inspection and supervision is processed by structured modeling to output a structured knowledge graph of inspection and supervision. When receiving the original data of multi-source inspection and patrol business, the corresponding LoRA parameter package with the optimal gated weighted weight ratio is called based on the task classification of the original data of multi-source inspection and patrol business, and is used as the target LoRA parameter package. The LoRA-Bottleneck Adapter fusion base model is used to process the original data of the multi-source inspection and supervision business based on the target LoRA parameter package and the structured inspection and supervision knowledge graph, and to generate standardized inspection and supervision business processing results.

2. The cross-task migration method for the collaborative optimization of the large-scale inspection and patrol model using LoRA and Bottleneck Adapter as described in claim 1, characterized in that, The step of constructing a preprocessed corpus dataset for the field of inspection and supervision based on the original corpus of inspection and supervision includes: The original corpus of patrol and inspection is sequentially deduplicated, denoised, and format-standardized to output the cleaned corpus of patrol and inspection. The cleaned patrol and inspection corpus is divided into segments, and the segmented patrol and inspection corpus set is output. The segmented corpus of inspection and supervision is labeled, and a preprocessed corpus dataset in the field of inspection and supervision is output.

3. The cross-task migration method for the collaborative optimization of the large-scale inspection and patrol model using LoRA and Bottleneck Adapter as described in claim 1, characterized in that, The introduction of LoRA and Bottleneck Adapter technologies into the general large language model to construct a LoRA-Bottleneck Adapter fusion base model includes: The bottleneck layer of the Transformer layer of the general large language model is embedded with the Bottleneck Adapter to obtain the basic model with the embedded Bottleneck Adapter. Configure pluggable LoRA submodules for the attention layer and feedforward layer of the embedded Bottleneck Adapter base model, and output a LoRA-Bottleneck Adapter dual-module base model; A gating weighting mechanism is introduced into the LoRA-Bottleneck Adapter dual-module basic model to output a LoRA-Bottleneck Adapter dual-module basic model with gating weighting mechanism; The backbone parameters of the LoRA-Bottleneck Adapter dual-module base model with gating weighting mechanism are frozen, and the parameters of the Bottleneck Adapter model and the LoRA model are trained together to obtain the LoRA-Bottleneck Adapter fusion base model.

4. The cross-task migration method for the collaborative optimization of the large-scale inspection and patrol model using LoRA and Bottleneck Adapter as described in claim 1, characterized in that, The LoRA-Bottleneck Adapter fusion base model is constructed using training and validation sets of multiple task classifications from the preprocessed corpus dataset in the inspection and patrol domain. This results in LoRA parameter packages for each task classification with optimal gated weighted weight ratios, including: Freeze the backbone parameters and BottleneckAdapter model parameters of the LoRA-Bottleneck Adapter fusion base model, and use the training sets of each task classification to jointly train the LoRA model parameters and gated weighting ratio of the LoRA-Bottleneck Adapter fusion base model to obtain the trained LoRA-Adapter fusion patrol and inspection large model corresponding to each task classification. The performance of the corresponding trained LoRA-Adapter fusion inspection and patrol model is verified using the validation set of each task category, thus obtaining the verified LoRA-Adapter fusion inspection and patrol model for each task category. LoRA parameters and optimal gating weight ratios are extracted from the verified LoRA-Adapter fusion inspection and patrol models corresponding to each task category, and the LoRA parameter package with the optimal gating weight ratio is output for each task category.

5. The cross-task migration method for the collaborative optimization of the large-scale inspection and patrol model using LoRA and Bottleneck Adapter as described in claim 1, characterized in that, The LoRA-Bottleneck Adapter fusion base model is used to process the multi-source raw data of the inspection and supervision based on the target LoRA parameter package and the structured inspection and supervision knowledge graph, generating standardized inspection and supervision business processing results, including: The original data from the multi-source inspection and patrol business are sequentially subjected to optical character recognition, data cleaning, and format conversion to output structured inspection and patrol business data. The target LoRA parameter package is used to update the model parameters of the LoRA-Bottleneck Adapter fusion base model to obtain the patrol and inspection cross-task online inference model; The cross-task online reasoning model for inspection and supervision is used to perform cross-task intelligent reasoning processing on the structured inspection and supervision business data based on the structured inspection and supervision knowledge graph, and outputs the intelligent reasoning results of inspection and supervision tasks. Based on the intelligent reasoning results of the inspection and supervision tasks and the preset inspection and supervision business template, template filling and text splicing are performed to output the initial standardized inspection and supervision business document. The initial standardized inspection and supervision business documents are verified and corrected, and the standardized inspection and supervision business processing results are output.

6. A cross-task migration system for collaborative optimization of a large-scale patrol and inspection model using LoRA and Bottleneck Adapter, characterized in that, include: The acquisition module is used to acquire the original corpus of inspection and supervision, and to construct a preprocessed corpus dataset in the field of inspection and supervision based on the original corpus of inspection and supervision. The model building module is used to introduce LoRA and Bottleneck Adapter technologies into a general large language model and build a LoRA-Bottleneck Adapter fusion base model. The parameter package construction module is used to construct the parameter package for the LoRA-Bottleneck Adapter fusion base model based on the training set and validation set of multiple task classifications in the preprocessed corpus dataset of the inspection and patrol domain, so as to obtain the LoRA parameter package with the optimal gated weighted weight ratio for each task classification. The graph modeling module is used to perform structured modeling on the preset basic knowledge base of inspection and supervision, and output a structured knowledge graph of inspection and supervision. The selection module is used to, when receiving the original data of the multi-source business of the inspection and patrol, call the corresponding LoRA parameter package with the optimal gated weighted weight ratio based on the task classification of the original data of the multi-source business of the inspection and patrol, and use it as the target LoRA parameter package; The generation module is used to perform business processing on the original multi-source business data of the inspection and supervision based on the target LoRA parameter package and the structured inspection and supervision knowledge graph using the LoRA-Bottleneck Adapter fusion base model, and generate standardized business processing results for inspection and supervision.

7. The cross-task migration system for the collaborative optimization of the large-scale patrol and inspection model using LoRA and Bottleneck Adapter as described in claim 6, characterized in that, The model building module is specifically used for: The bottleneck layer of the Transformer layer of the general large language model is embedded with the Bottleneck Adapter to obtain the basic model with the embedded Bottleneck Adapter. Configure pluggable LoRA submodules for the attention layer and feedforward layer of the embedded Bottleneck Adapter base model, and output a LoRA-Bottleneck Adapter dual-module base model; A gating weighting mechanism is introduced into the LoRA-Bottleneck Adapter dual-module basic model to output a LoRA-Bottleneck Adapter dual-module basic model with gating weighting mechanism; The backbone parameters of the LoRA-Bottleneck Adapter dual-module base model with gating weighting mechanism are frozen, and the parameters of the Bottleneck Adapter model and the LoRA model are trained together to obtain the LoRA-Bottleneck Adapter fusion base model.

8. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the cross-task migration method for the LoRA and Bottleneck Adapter collaborative optimization of the large-scale patrol and inspection model as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the cross-task migration method for the collaborative optimization of the large-scale patrol and inspection model using LoRA and Bottleneck Adapter as described in any one of claims 1-5.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein, when the program instructions are executed by a computer, the computer performs the steps of the cross-task migration method for the LoRA and BottleneckAdapter collaborative optimization of the patrol and inspection large model as described in any one of claims 1-5.

Images